A precise positioning maintenance inspection system and method based on road network monitoring

Abstract

The present invention discloses a precise positioning maintenance inspection system and method based on road network monitoring. First, the video monitoring equipment along the roadside of the highway network is initialized and the pre-position parameters are calibrated, the mileage pile numbers along the roadside of the highway network are matched with the video monitoring screen, and the video monitoring equipment collects road network monitoring pictures and videos. Finally, based on the collected road network monitoring pictures and videos, the road network is intelligently inspected according to demand. Based on the existing monitoring equipment of the road network, the present invention completes the intelligent maintenance inspection of the highway through pre-position setting and image acquisition and splicing, significantly improves the conventional road network management, the maintenance inspection monitoring field scene range, strengthens the coordination level of abnormal road network monitoring, significantly improves the regional road network operation status overall management and refined maintenance inspection capabilities, effectively supports accurate small-scale mileage pile number section operation observation, improves the management and maintenance inspection video polling frequency, and effectively reduces the road network management and maintenance investment.

Description

Technical Field

[0001] The present invention belongs to the field of road traffic monitoring and maintenance, and specifically relates to a precise positioning maintenance inspection system and method based on road network monitoring. Background Art

[0002] With the development of economic and technological levels, the video surveillance of ordinary national and provincial trunk roads has been further encrypted and the video quality has been further improved. The use of roadside surveillance cameras for inspection has become an effective means of road inspection and monitoring. At present, the average density of video surveillance points on major national and provincial trunk roads in key cities across the country is high, and the construction and application conditions are relatively complete. Through the collection and artificial intelligence analysis of fixed roadside videos, event detection work is carried out. Event detection information can support road maintenance inspections, and can realize rapid detection and alarm of road spills and accumulations, realizing a closed loop of event discovery, event confirmation, and event processing.

[0003] With the continuous encryption of national and provincial trunk video surveillance facilities, the drawbacks of the traditional road network video monitoring mode are becoming increasingly prominent. In the traditional mode, the on-duty personnel regularly check the road network video points in the jurisdiction area through manual operation of the control keyboard. Due to the large number of fixed video points in the road network, and the fact that each point has two directions, uplink and downlink, and each video has multiple preset positions, the task is very heavy. For scenes such as highway reconstruction and expansion construction, maintenance construction, traffic congestion, and traffic accidents, due to the lack of quantitative matching relationship between the spatial position of the camera and the highway, and the lack of collaborative linkage mechanism between adjacent cameras, it is difficult to fully and accurately perceive the operating status of the above abnormal scenes, and it is difficult to effectively support the efficient operation of emergency command and dispatch and other businesses, and it is difficult to effectively tap the potential of existing video facilities, and it is impossible to provide all-round video monitoring information services for road network management, maintenance inspections, and maintenance projects. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a precise positioning maintenance inspection system and method based on road network monitoring, utilize camera preset positions to tap the potential of existing roadside video monitoring facilities, explore traffic incident detection under multiple camera preset positions, innovate active road network video monitoring methods, and significantly improve highway video monitoring efficiency.

[0005] The specific technical solution for achieving the purpose of the present invention is:

[0006] A precise positioning maintenance inspection system based on road network monitoring, including a roadside video monitoring preset position calibration module, a roadside video image precise matching module, a roadside video monitoring image acquisition module, and a highway inspection module;

[0007] The roadside video surveillance preset position calibration module is used to initialize the video surveillance equipment along the roadside of the highway network and calibrate the preset position parameters;

[0008] The roadside video image accurate matching module is used to match the mileage pile numbers along the roadside of the highway network with the video surveillance images;

[0009] The roadside video surveillance picture acquisition module is used to collect road network monitoring pictures and videos based on video surveillance equipment;

[0010] The highway inspection module performs intelligent inspections of the road network based on collected road network monitoring images and videos as needed.

[0011] Furthermore, the roadside video surveillance preset position calibration module performs parameter calibration according to the model and performance of the video surveillance equipment of the highway network and the height and angle of equipment installation;

[0012] The camera imaging principle is used to determine the maximum monitoring distance of existing cameras.

[0013] Furthermore, the preset position parameter calibration of the video surveillance equipment includes camera model, performance parameters, camera installation height, initial installation pitch angle, installation longitude and latitude, and maximum rotation angle of the camera.

[0014] Furthermore, the process of determining the farthest monitoring distance of the camera is as follows:

[0015] The maximum viewing distance of the camera is determined by the maximum focal length of the camera under the condition that the minimum resolution pixel of the test target can be clearly seen in the video screen. The maximum monitoring distance of the camera is determined according to the location of the test target and the preset position parameters of the camera:

[0016]

[0017] dis1=(dis0 2 -h0 2 ) 0.5 +o(δ3)=h0·tan(α-α0)+o(δ3)

[0018] Where dis0 represents the distance between the target and the camera; dis1 represents the farthest monitoring distance of the camera; U represents the actual height of the target; h represents the width of the camera target image; H represents the actual width of the target; u represents the height of the camera target image; f represents the focal length of the camera; h0 represents the initial height of the camera; α0 represents the initial installation angle; o(δ1), o(δ2), and o(δ3) represent error terms;

[0019] Then the monitoring range of the surveillance camera at the Kth highway mileage post is [K-dis1,K+dis1].

[0020] Furthermore, the coverage of video surveillance is first divided, and for each video point, between the upper and lower limit preset positions, i.e., the observation range of the farthest preset position, preset positions are gradually set according to the requirement that there is enough intersection of the road section monitoring range between the images taken by adjacent preset positions, which are called intermediate preset positions;

[0021] Based on the camera imaging principle, a corresponding relationship is established between each intermediate preset position and the mileage pile number with longitude and latitude as the intermediate node, and based on the distance dis between each intermediate preset position and the surveillance camera, the pitch angle α, horizontal angle β and focal length f required to adjust the camera to the intermediate preset position are determined:

[0022]

[0023] Where R represents the radius of the earth; (x, y) represents the longitude and latitude coordinates of the target center point in the geodetic coordinate system; h0 represents the initial height of the surveillance camera; α0 represents the initial pitch angle of the surveillance camera installation; β0 represents the initial horizontal angle of the surveillance camera installation; (x0, y0) represents the longitude and latitude coordinates of the geodetic coordinate system of the initial installation point of the surveillance camera; U represents the height of the preset target; h represents the image width of the surveillance camera target surface; H represents the width of the target; u represents the image height of the surveillance camera target surface; o'(δ) represents the error term, k represents the number of lanes, and N represents the width of a single lane;

[0024] Relying on the latitude and longitude position information, an accurate matching relationship between the preset position of the camera's specific angle and focal length parameters and the mileage post number is established. That is, by using the roadside mileage posts, signs or traffic facilities and equipment containing obvious mileage post information near the route post number as reference objects, the real-time video or image of the accurate mileage post number information at each center preset position is obtained, thereby achieving an accurate match between the real-time picture of the camera preset position and the mileage post number.

[0025] Furthermore, the roadside video surveillance image acquisition module sets a fixed time every day according to the patrol requirements, and within a smaller period of time, simultaneously and in parallel dispatches different video points to shoot videos or images in sequence according to the set preset positions, thereby obtaining video surveillance pictures and videos covering the entire road section.

[0026] Furthermore, the highway inspection and maintenance module edits and splices images based on the monitoring pictures or videos collected by the roadside video monitoring picture collection module to form a complete road network monitoring inspection picture or video;

[0027] Among them, for the uplink or downlink starting preset position of the same video point, combined with the invariant-based SURF feature detection algorithm, the pixel-level fusion splicing of the uplink and downlink starting preset position images is achieved;

[0028] For different intermediate preset positions of the same video point, the image is combined along the direction of increasing or decreasing the highway pile number;

[0029] For the extreme preset positions where there is an overlapping monitoring range between adjacent video points, if the optical zoom ratios are the same, the images with opposite perspectives are fused and spliced ​​at the pixel level; if the optical zoom ratios are inconsistent, the images of the extreme preset positions of adjacent video points are combined between the frames.

[0030] The present invention also provides a method for accurately positioning a maintenance inspection system based on road network monitoring, comprising the following steps:

[0031] Step 1: Roadside video surveillance preset position calibration, initializing and calibrating the parameters of the video surveillance equipment along the roadside of the highway network;

[0032] Step 2: Accurately match the roadside video images, and match the mileage posts and calibration objects along the roadside of the highway network with the real-time images of the video surveillance equipment;

[0033] Step 3: roadside video surveillance image collection, collecting road network monitoring images and videos based on video surveillance equipment;

[0034] Step 4: Highway inspection and maintenance: Based on the collected road network monitoring images and videos, the road network is inspected and maintained as needed.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) At present, the research and application of preset position and pan / tilt control technology have been relatively mature. There are many research results that apply preset position and pan / tilt control to the field of automatic target tracking. The pan / tilt and preset position control progress can reach 0.1°, which can better support preset position calibration and polling control, and provide a certain technical reserve and application basis for roadside video access and automatic polling, information sharing with collaborative patrol systems, and highway longitude and mileage pile number matching. On this basis, the solution of the present invention uses camera preset positions to explore the potential of existing roadside video monitoring facilities, explore traffic event detection under multiple camera preset positions, innovate active road network video monitoring methods, and significantly improve highway video monitoring efficiency;

[0037] (2) The solution of the present invention can significantly improve the scope of conventional road network management, maintenance inspection and monitoring vision scenes, and strengthen the coordination level of abnormal road network monitoring. It can significantly improve the overall management and control of regional road network operation status, refined maintenance inspection, and accurate perception of abnormal scenes. It can effectively support accurate observation of road operation sections with small mileage pile numbers, increase the frequency of road management and maintenance inspection video polling, and effectively reduce the labor intensity of road network management and maintenance inspection personnel;

[0038] (3) The solution of the present invention can provide multi-form video inspection for daily monitoring of road networks. In daily status, the time, route, pile number range and frequency of daily automatic inspection can be set independently. According to the line output, the monitoring view of the road section at that time point can be completed by simply viewing the map. In emergencies such as traffic congestion and traffic accidents, there is no need to find a camera near the pile number from the facilities. Only the line pile number needs to be entered to automatically retrieve the most matching camera for observation, which greatly improves the response time. Multi-form video inspection can be provided for maintenance project management such as overhaul, medium repair and minor repair. For daily safety and progress management of maintenance projects, project management personnel can independently set the time, route, pile number range and frequency of daily automatic inspection according to the reported maintenance project information. According to the line output, the monitoring view of the road section at that time point can be completed by simply viewing the map. For the construction site of the maintenance project that needs to be monitored, the management personnel only need to enter the pile number of the corresponding position to automatically retrieve the most matching camera for observation, which significantly improves the management efficiency of the maintenance project.

[0039] The present invention is further described below in conjunction with specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the architecture of the precise positioning maintenance inspection system based on road network monitoring of the present invention.

[0041] Figure 2 It is a schematic diagram of preset position calibration in an embodiment of the present invention.

[0042] Figure 3 Schematic diagram of calculating the maximum monitoring distance of a camera in an embodiment of the present invention.

[0043] Figure 4 It is a schematic diagram of calculating the intermediate preset position in an embodiment of the present invention.

[0044] Figure 5 Schematic diagram of road network monitoring in an embodiment of the present invention.

[0045] Figure 6 Schematic diagram of video inspection in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] A precise positioning maintenance inspection system based on road network monitoring, including a roadside video monitoring preset position calibration module, a roadside video image precise matching module, a roadside video monitoring image acquisition module, and a highway inspection module;

[0047] The roadside video surveillance preset position calibration module is used to initialize the video surveillance equipment along the roadside of the highway network and calibrate the preset position parameters;

[0048] The roadside video image accurate matching module is used to match the mileage pile numbers along the roadside of the highway network with the video surveillance images;

[0049] The roadside video surveillance picture acquisition module is used to collect road network monitoring pictures and videos based on video surveillance equipment;

[0050] The highway inspection module performs intelligent inspections of the road network based on collected road network monitoring images and videos as needed.

[0051] Furthermore, the roadside video surveillance preset position calibration module performs parameter calibration according to the model and performance of the video surveillance equipment of the highway network and the height and angle of equipment installation;

[0052] The camera imaging principle is used to determine the maximum monitoring distance of existing cameras.

[0053] Furthermore, the preset position parameter calibration of the video surveillance equipment includes camera model, performance parameters, camera installation height, initial installation pitch angle, installation longitude and latitude, and maximum rotation angle of the camera.

[0054] In addition, the camera preset position needs to be calibrated. It is necessary to select typical highway targets based on the conditions of the video surveillance facilities along the route and the video sharing software platform. According to the camera brand model and initial parameters such as installation height and angle, adjust the focal length of the high-definition ball camera and rotate the pan / tilt to different viewing angles. Based on the camera imaging principle, quantitatively analyze the farthest monitoring distance of the existing cameras, and further quantitatively analyze the video surveillance blind spots between adjacent cameras.

[0055] The camera's farthest monitoring distance is when a long-distance vehicle target is used as the calibration target. The vehicle target is recognizable or distinguishable in the image. It is required that in an environment with a brightness of not less than 2000lx and a visibility of not less than 2000m, the vehicle in the image taken at different preset positions is not less than 400*400 pixels, the non-motor vehicle is not less than 200*200 pixels, and the pedestrian is not less than 100*100 pixels. The road area is located in the middle of the image and occupies more than 60% of the image. Specifically, the determination process is as follows:

[0056] The maximum viewing distance of the camera is determined by the maximum focal length of the camera under the condition that the minimum resolution pixel of the test target can be clearly seen in the video screen. The maximum monitoring distance of the camera is determined according to the location of the test target and the preset position parameters of the camera:

[0057]

[0058] dis1=(dis0 2 -h0 2 )0.5 +o(δ3)=h0·tan(α-α0)+o(δ3)

[0059] Wherein, dis0 represents the distance between the target and the camera; dis1 represents the farthest monitoring distance of the camera; U represents the actual height of the target; h represents the width of the camera target image; H represents the actual width of the target; u represents the height of the camera target image; f represents the focal length of the camera; h0 represents the initial height of the camera; α0 represents the initial installation angle; o(δ1), o(δ2), and o(δ3) represent error terms;

[0060] Then the monitoring range of the surveillance camera at the Kth highway mileage post is [K-dis1,K+dis1].

[0061] By calibrating the camera preset positions, analyzing the online video monitoring blind spots, and providing a reference basis for the encryption of road network video monitoring facilities

[0062] Furthermore, the accurate matching of the roadside video images is achieved after quantitatively analyzing the maximum observation range of the roadside camera. In order to further realize accurate positioning and inspection based on road network monitoring facilities, the matching technology of different preset positions and route pile numbers is studied. By combining the mileage piles between adjacent video points and kilometer piles with known mileage pile numbers, traffic signs, light poles, markings, road intersections and other targets, the accurate positioning of the camera preset position and the route pile number is achieved, and the pile number range is calibrated for the road section covered by the video images taken at different preset positions. It is convenient for road network managers to input the pile number range to automatically mobilize the corresponding video point to rotate to the corresponding pan-tilt position, and automatically adjust to the corresponding focal length to shoot video images covering the input pile number range, thereby improving the efficiency of road network management, maintenance inspections, and emergency command and dispatch work. The specific process is as follows:

[0063] First, the coverage of video surveillance is divided. For each video point, between the upper and lower limit preset positions, i.e., the observation range of the farthest preset position, preset positions are gradually set according to the requirement that there is enough intersection of road monitoring range between the images taken by adjacent preset positions, which are called intermediate preset positions.

[0064] Based on the camera imaging principle, a corresponding relationship is established between each intermediate preset position and the mileage pile number with longitude and latitude as the intermediate node, and based on the distance dis between each intermediate preset position and the surveillance camera, the pitch angle α, horizontal angle β and focal length f required to adjust the camera to the intermediate preset position are determined:

[0065] Based on the camera imaging principle, a corresponding relationship is established between each intermediate preset position and the mileage pile number with longitude and latitude as the intermediate node, and based on the distance dis between each intermediate preset position and the surveillance camera, the pitch angle α, horizontal angle β and focal length f required to adjust the camera to the intermediate preset position are determined:

[0066]

[0067] Where R represents the radius of the earth; (x, y) represents the longitude and latitude coordinates of the target center point in the geodetic coordinate system; h0 represents the initial height of the surveillance camera; α0 represents the initial pitch angle of the surveillance camera installation; β0 represents the initial horizontal angle of the surveillance camera installation; (x0, y0) represents the longitude and latitude coordinates of the geodetic coordinate system of the initial installation point of the surveillance camera; U represents the height of the preset target; h represents the image width of the surveillance camera target surface; H represents the width of the target; u represents the image height of the surveillance camera target surface; o'(δ) represents the error term, k represents the number of lanes, and N represents the width of a single lane;

[0068] Relying on the latitude and longitude position information, the preset position of the camera's specific angle and focal length parameters is precisely matched with the mileage pile number. That is, the roadside mileage piles, signs, or traffic facilities with obvious mileage pile information near the route pile number are used as reference objects to obtain the real-time video or image of the precise mileage pile number information under each center preset position, so as to achieve the precise matching of the real-time picture of the camera preset position with the mileage pile number. The roadside video surveillance picture acquisition module sets a fixed time every day according to the patrol requirements. Within a certain smaller time period, it simultaneously and parallelly dispatches different video points to shoot videos or images in sequence according to the preset positions, thereby obtaining video surveillance pictures and videos covering the entire road section.

[0069] The highway inspection and maintenance module clips and splices images based on the monitoring pictures or videos collected by the roadside video monitoring picture collection module to form a complete road network monitoring inspection picture or video;

[0070] Among them, for the uplink or downlink starting preset position of the same video point, combined with the invariant-based SURF feature detection algorithm, the pixel-level fusion splicing of the uplink and downlink starting preset position images is achieved;

[0071] For different intermediate preset positions of the same video point, the image is combined along the direction of increasing or decreasing the highway pile number;

[0072] For the extreme preset positions where there is an overlapping monitoring range between adjacent video points, if the optical zoom ratios are the same, the images with opposite perspectives are fused and spliced ​​at the pixel level; if the optical zoom ratios are inconsistent, the images of the extreme preset positions of adjacent video points are combined between the frames.

[0073] The present invention also provides a method for accurately positioning a maintenance inspection system based on road network monitoring, comprising the following steps:

[0074] Step 1: Roadside video surveillance preset position calibration, initializing and calibrating the parameters of the video surveillance equipment along the roadside of the highway network;

[0075] Among them, parameter calibration is carried out according to the model and performance of the video monitoring equipment of the highway network and the height and angle of the equipment installation, and the maximum monitoring distance of the existing camera is determined according to the camera imaging principle;

[0076] The process of determining the farthest monitoring distance of the camera is as follows:

[0077] The maximum viewing distance of the camera is determined by the maximum focal length of the camera under the condition that the minimum resolution pixel of the test target can be clearly seen in the video screen. The maximum monitoring distance of the camera is determined according to the location of the test target and the preset position parameters of the camera:

[0078]

[0079] dis1=(dis0 2 -h0 2 ) 0.5 +o(δ3)=h0·tan(α-α0)+o(δ3)

[0080] Where dis0 represents the distance between the target and the camera; dis1 represents the farthest monitoring distance of the camera; U represents the actual height of the target; h represents the width of the camera target image; H represents the actual width of the target; u represents the height of the camera target image; f represents the focal length of the camera; h0 represents the initial height of the camera; α0 represents the initial installation angle; o(δ1), o(δ2), and o(δ3) represent error terms;

[0081] Then the monitoring range of the surveillance camera at the Kth highway mileage post is [K-dis1,K+dis1].

[0082] Step 2: Accurately match the roadside video images, and match the mileage posts and calibration objects along the roadside of the highway network with the real-time images of the video surveillance equipment:

[0083] First, the coverage of video surveillance is divided. For each video point, between the upper and lower limit preset positions, i.e., the observation range of the farthest preset position, preset positions are gradually set according to the requirement that there is enough intersection of road monitoring range between the images taken by adjacent preset positions, which are called intermediate preset positions.

[0084] Based on the camera imaging principle, a corresponding relationship is established between each intermediate preset position and the mileage pile number with longitude and latitude as the intermediate node, and based on the distance dis between each intermediate preset position and the surveillance camera, the pitch angle α, horizontal angle β and focal length f required to adjust the camera to the intermediate preset position are determined:

[0085]

[0086] Where R represents the radius of the earth; (x, y) represents the longitude and latitude coordinates of the target center point in the geodetic coordinate system; h0 represents the initial height of the surveillance camera; α0 represents the initial pitch angle of the surveillance camera installation; β0 represents the initial horizontal angle of the surveillance camera installation; (x0, y0) represents the longitude and latitude coordinates of the geodetic coordinate system of the initial installation point of the surveillance camera; U represents the height of the preset target; h represents the image width of the surveillance camera target surface; H represents the width of the target; u represents the image height of the surveillance camera target surface; o'(δ) represents the error term, k represents the number of lanes, and N represents the width of a single lane;

[0087] Relying on the latitude and longitude location information, establish the precise matching relationship between the preset position of the camera's specific angle and focal length parameters and the mileage pile number, that is, use the roadside mileage piles, signs or traffic facilities with obvious mileage pile information near the route pile number as reference objects to obtain the real-time video or image of the precise mileage pile number information under each center preset position, so as to achieve the precise matching between the real-time picture of the camera preset position and the mileage pile number. Step 3: Roadside video surveillance picture acquisition, based on the video surveillance equipment to collect road network monitoring pictures and videos;

[0088] Step 4: Highway inspection and maintenance: Based on the collected road network monitoring images and videos, the road network is inspected and maintained as needed.

[0089] Example

[0090] Combination Figure 1 , a precise positioning maintenance inspection system based on road network monitoring, including a roadside video monitoring preset position calibration module, a roadside video image precise matching module, a roadside video monitoring image acquisition module, and a highway inspection module;

[0091] The roadside video surveillance preset position calibration module is used to initialize the video surveillance equipment along the roadside of the highway network and calibrate the preset position parameters;

[0092] The roadside video image accurate matching module is used to match the mileage pile numbers along the roadside of the highway network with the video surveillance images;

[0093] The roadside video surveillance picture acquisition module is used to collect road network monitoring pictures and videos based on video surveillance equipment;

[0094] The highway inspection module performs intelligent inspections of the road network based on collected road network monitoring images and videos as needed.

[0095] The roadside video surveillance preset position calibration module performs parameter calibration according to the model and performance of the video surveillance equipment of the highway network and the height and angle of the equipment installation;

[0096] The camera imaging principle is used to determine the maximum monitoring distance of existing cameras.

[0097] In addition, the camera preset position needs to be calibrated. It is necessary to select typical highway targets based on the conditions of the video surveillance facilities along the route and the video sharing software platform. According to the camera brand model and initial parameters such as installation height and angle, adjust the focal length of the high-definition ball camera and rotate the pan / tilt to different viewing angles. Based on the camera imaging principle, quantitatively analyze the farthest monitoring distance of the existing cameras, and further quantitatively analyze the video surveillance blind spots between adjacent cameras, such as Figure 2 As shown;

[0098] The preset position parameter calibration of the video surveillance equipment includes camera model, performance parameters, camera installation height, initial installation pitch angle, installation latitude and longitude, and maximum rotation angle of the camera. When the camera's farthest monitoring distance is a long-distance vehicle target as the calibration target, the vehicle target is recognizable or distinguishable in the picture. It is required that in an environment with a brightness of not less than 2000lx and visibility of not less than 2000m, the vehicle in the image taken at different preset positions is not less than 400*400 pixels, the non-motor vehicle is not less than 200*200 pixels, and the pedestrian is not less than 100*100 pixels. The road area is located in the middle of the image and occupies more than 60% of the image length. Specifically, the determination process is:

[0099] The maximum monitoring distance of the camera is based on the projection relationship between the object in the three-dimensional space and the imaging plane. The method of analytic geometry and trigonometric function is used to determine the target point in the ground object coordinate space and its spatial relationship for calculation. The error caused by the inclination of the road surface, the error caused by the internal conversion of the grating, the error of the imaging principle model system, the position offset error during the rotation of the lens are ignored, and the calculation accuracy is 0.1.

[0100] Combination Figure 3 , under the condition that the minimum resolution pixel of the test target can be clearly seen in the video screen, the maximum focal length of the camera determines the maximum viewing distance of the camera, and the maximum monitoring distance of the camera is determined according to the location of the test target and the preset position parameters of the camera:

[0101]

[0102] dis1=(dis0 2 -h0 2 ) 0.5 +o(δ3)=h0·tan(α-α0)+o(δ3)

[0103] Where dis0 represents the distance between the target and the camera; dis1 represents the farthest monitoring distance of the camera; U represents the actual height of the target; h represents the width of the camera target image; H represents the actual width of the target; u represents the height of the camera target image; f represents the focal length of the camera; h0 represents the initial height of the camera; α0 represents the initial installation angle; o(δ1), o(δ2), and o(δ3) represent error terms;

[0104] Then the monitoring range of the surveillance camera at the Kth highway mileage post is [K-dis1,K+dis1].

[0105] In this embodiment, the camera used is a 4-megapixel high-definition ball camera, which supports 360° horizontal rotation and automatic flipping from -20° to 90°. Therefore, the installation angle limitation is not considered, and the square difference formula is used to calculate the theoretical coverage range. For example, the camera target surface is a 1 / 1.8-inch CCD sensor, the target surface size is 7.2*5.4 mm, the focal length is 6~240 mm automatic focus, and it is installed on a 12-meter pole. The target object is a standard car (size is 3.8*1.8*1.5 meters). Considering the extreme tilt state of the car in the image, the target car is measured at 3.8*3.8 meters. The basic requirement for the car to be clearly visible is not less than 400*400 pixels, so:

[0106]

[0107] dis1=(dis0 2 -h0 2 ) 0.5 +o(δ3)

[0108] =(729.6 2 -12 2 ) 0.5 +o(δ)

[0109] ≈729.5≈730m

[0110] Therefore, the theoretical coverage range of the surveillance camera is about 730 meters on one side.

[0111] By calibrating the camera preset positions, analyzing the online video monitoring blind spots, and providing a reference basis for the encryption of road network video monitoring facilities

[0112] The precise matching of roadside video images is carried out after quantitatively analyzing the maximum observation range of the roadside camera. In order to further realize the precise positioning and inspection based on the road network monitoring facilities, the matching technology of different preset positions and route pile numbers is studied. By combining the mileage piles between adjacent video points and kilometer piles with known mileage pile numbers, traffic signs, light poles, markings, road intersections and other targets, the precise positioning of the camera preset position and the route pile number is realized, and the pile number range is calibrated for the road section covered by the video images taken at different preset positions. It is convenient for road network managers to input the pile number range to automatically mobilize the corresponding video point to rotate to the corresponding pan-tilt position, and automatically adjust to the corresponding focal length to shoot video images covering the input pile number range, thereby improving the efficiency of road network management, maintenance inspections, and emergency command and dispatch work. The specific process is as follows:

[0113] First, the coverage of video surveillance is divided. For each video point, the observation range of the upper and lower limit preset positions is determined according to the requirement that there is enough intersection of the road monitoring range between the images taken by adjacent preset positions. The preset positions are called intermediate preset positions.

[0114] Combination Figure 4 ,

[0115] Based on the camera imaging principle, a corresponding relationship is established between each intermediate preset position and the mileage pile number with longitude and latitude as the intermediate node, and based on the distance dis between each intermediate preset position and the surveillance camera, the pitch angle α, horizontal angle β and focal length f required to adjust the camera to the intermediate preset position are determined:

[0116]

[0117] Where R represents the radius of the earth; (x, y) represents the longitude and latitude coordinates of the target center point in the geodetic coordinate system; h0 represents the initial height of the surveillance camera; α0 represents the initial pitch angle of the surveillance camera installation; β0 represents the initial horizontal angle of the surveillance camera installation; (x0, y0) represents the longitude and latitude coordinates of the geodetic coordinate system of the initial installation point of the surveillance camera; U represents the height of the preset target; h represents the image width of the surveillance camera target surface; H represents the width of the target; u represents the image height of the surveillance camera target surface; o'(δ) represents the error term, k represents the number of lanes, and N represents the width of a single lane;

[0118] Relying on the latitude and longitude position information, the preset position of the camera with specific angle and focal length parameters is precisely matched with the mileage pile number. That is, the roadside mileage piles, signs, or traffic facilities with obvious mileage pile information near the route pile number are used as reference objects to obtain the real-time video or image of the precise mileage pile number information under each center preset position, so as to achieve the precise matching of the real-time picture of the camera preset position with the mileage pile number. In actual situations, the coordinates of the observation center point are generally unknown, and the angle between the roadside camera and the horizontal line is small enough, so dis and the road width N can also be used to approximate β.

[0119] In this embodiment, the coverage of video surveillance is split, and for each video point, the observation range between the upper and lower limit preset positions is considered, and according to the requirement that there is enough intersection of the road monitoring range between the images taken by adjacent preset positions, a corresponding calculation relationship is established between each split preset position and the mileage pile number with longitude and latitude as the intermediate node, and the intermediate preset positions are calibrated one by one. For example, assuming that the camera is installed at K1, the initial pitch angle is 0°, the initial horizontal angle is 0°, and the highway is a two-way six-lane road, the calculation formula for the intermediate preset position is as follows:

[0120] Table 1 Preset position calculation table

[0121]

[0122] Relying on the longitude and latitude position information, establish an accurate matching relationship between the route pile number and the mileage pile number. That is, use the roadside mileage piles, signs or traffic facilities with obvious mileage pile information near the route pile number as reference objects to obtain the accurate mileage pile number information under each center preset position, so as to achieve accurate matching between the camera preset position and the mileage pile number. Connect the video stream and the PTZ control of the video device through the video platform API interface or SDK, and then automatically track and turn according to the longitude and latitude, altitude of the camera, and the longitude and latitude of the pile number. On the web side of the matching tool, you can manually control the PTZ to obtain the corresponding position according to the specific video, compare the automatically acquired image with the manual image, and fine-tune the parameters according to the differences in the comparison process to eliminate the error terms in the calculation process. Its overall architecture is as follows: Figure 5 shown.

[0123] The roadside video surveillance image acquisition module sets a fixed time every day according to the patrol requirements. Within a smaller period of time, it simultaneously and parallelly dispatches different video points to shoot videos or images in sequence according to the set preset positions, thereby obtaining video surveillance images and videos covering the entire road section.

[0124] The highway inspection and maintenance module clips and splices the images based on the monitoring pictures or videos collected by the roadside video monitoring picture collection module to form a complete road network monitoring inspection picture or video, such as Figure 6 As shown;

[0125] Among them, for the uplink or downlink starting preset position of the same video point, combined with the invariant-based SURF feature detection algorithm, the pixel-level fusion splicing of the uplink and downlink starting preset position images is achieved;

[0126] For different intermediate preset positions of the same video point, the image is combined along the direction of increasing or decreasing the highway pile number;

[0127] For the extreme preset positions where there is an overlapping monitoring range between adjacent video points, if the optical zoom ratios are the same, the images with opposite perspectives are fused and spliced ​​at the pixel level; if the optical zoom ratios are inconsistent, the images of the extreme preset positions of adjacent video points are combined between the frames.

[0128] The above embodiments show and describe the basic principles and main features of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A precise positioning maintenance inspection system based on road network monitoring, characterized in that: It includes roadside video surveillance preset position calibration module, roadside video image accurate matching module, roadside video surveillance image acquisition module, and highway inspection module; The roadside video surveillance preset position calibration module is used to initialize the video surveillance equipment along the roadside of the highway network and calibrate the preset position parameters; The roadside video surveillance preset position calibration module performs parameter calibration according to the model and performance of the video surveillance equipment of the highway network and the height and angle of the equipment installation; The maximum monitoring distance of existing cameras is determined based on the camera imaging principle, specifically: The maximum viewing distance of the camera is determined by the maximum focal length of the camera under the condition that the minimum resolution pixel of the test target can be clearly seen in the video screen. The maximum monitoring distance of the camera is determined according to the location of the test target and the preset position parameters of the camera: dis1=(dis0 2 -h0 2 ) 0.5 +o(δ3)=h0·tan(α-α0)+o(δ3) Where dis0 represents the distance between the target and the camera; dis1 represents the farthest monitoring distance of the camera; U represents the actual height of the target; h represents the width of the camera target image; H represents the actual width of the target; u represents the height of the camera target image; f represents the focal length of the camera; h0 represents the initial height of the camera; α0 represents the initial installation angle; ο(δ1), ο(δ2), ο(δ3) represent error terms; Then the monitoring range of the surveillance camera at the Kth highway mileage pile is [K-dis1, K+dis1]; The roadside video image accurate matching module is used to match the mileage pile numbers along the roadside of the highway network with the video surveillance images, specifically: First, the coverage of video surveillance is divided. For each video point, between the upper and lower limit preset positions, i.e., the observation range of the farthest preset position, preset positions are gradually set according to the requirement that there is enough intersection of road monitoring range between the images taken by adjacent preset positions, which are called intermediate preset positions. Based on the camera imaging principle, a corresponding relationship is established between each intermediate preset position and the mileage pile number with longitude and latitude as the intermediate node, and based on the distance dis between each intermediate preset position and the surveillance camera, the pitch angle α, horizontal angle β and focal length f required to adjust the camera to the intermediate preset position are determined: Wherein, R represents the radius of the earth; (x, y) represents the latitude and longitude coordinates of the target center point in the geodetic coordinate system; h0 represents the initial height of the surveillance camera; α0 represents the initial pitch angle of the surveillance camera installation; β0 represents the initial horizontal angle of the surveillance camera installation; (x0, y0) represents the latitude and longitude coordinates of the geodetic coordinate system of the initial installation point of the surveillance camera; U represents the height of the preset target; h represents the width of the surveillance camera target surface image; H represents the width of the target; u represents the height of the surveillance camera target surface image; o'(δ) represents the error term, represents the road width, k represents the number of lanes, and N represents the width of a single lane; Relying on the latitude and longitude position information, the preset position of the camera with specific angle and focal length parameters is precisely matched with the mileage pile number. That is, the roadside mileage piles, signs or traffic facilities with obvious mileage pile information near the route pile number are used as reference objects to obtain the real-time video or image of the precise mileage pile number information at each central preset position, so as to achieve precise matching between the real-time picture of the camera preset position and the mileage pile number. The roadside video surveillance picture acquisition module is used to collect road network monitoring pictures and videos based on video surveillance equipment; The highway inspection module performs intelligent inspections of the road network based on the collected road network monitoring images and videos according to needs.

2. According to the road network monitoring-based precise positioning maintenance inspection system of claim 1, it is characterized in that: The preset position parameter calibration of the video surveillance equipment includes the camera model, performance parameters, camera installation height, initial installation pitch angle, installation longitude and latitude, and maximum rotation angle of the camera.

3. The precise positioning maintenance inspection system based on road network monitoring according to claim 1 is characterized in that: The roadside video surveillance image acquisition module sets a fixed time every day according to the patrol requirements. Within a certain period of time, different video points are simultaneously and parallelly scheduled to shoot videos or images in sequence according to the set preset positions, thereby obtaining video surveillance images and videos covering the entire road section.

4. The precise positioning maintenance inspection system based on road network monitoring according to claim 1 is characterized in that: The highway inspection and maintenance module clips and splices images based on the monitoring pictures or videos collected by the roadside video monitoring picture collection module to form a complete road network monitoring inspection picture or video; Among them, for the uplink or downlink starting preset position of the same video point, combined with the invariant-based SURF feature detection algorithm, the pixel-level fusion splicing of the uplink and downlink starting preset position images is achieved; For different intermediate preset positions of the same video point, the image is combined along the direction of increasing or decreasing the highway pile number; For the extreme preset positions where there is an overlapping monitoring range between adjacent video points, if the optical zoom ratios are the same, the images with opposite perspectives are fused and spliced ​​at the pixel level; if the optical zoom ratios are inconsistent, the images of the extreme preset positions of adjacent video points are combined between the frames.

5. A method for accurately positioning maintenance inspection system based on road network monitoring, characterized in that: The following steps are involved: Step 1: Roadside video surveillance preset position calibration, initializing and calibrating the parameters of the video surveillance equipment along the roadside of the highway network; The parameter calibration is carried out according to the model and performance of the video monitoring equipment of the highway network and the height and angle of the equipment installation, and the maximum monitoring distance of the existing cameras is determined according to the camera imaging principle; The process of determining the farthest monitoring distance of the camera is as follows: The maximum viewing distance of the camera is determined by the maximum focal length of the camera under the condition that the minimum resolution pixel of the test target can be clearly seen in the video screen. The maximum monitoring distance of the camera is determined according to the location of the test target and the preset position parameters of the camera: dis1=(dis0 2 -h0 2 ) 0.5 +o(δ3)=h0·tan(α-α0)+o(δ3) Where dis0 represents the distance between the target and the camera; dis1 represents the farthest monitoring distance of the camera; U represents the actual height of the target; h represents the width of the camera target image; H represents the actual width of the target; u represents the height of the camera target image; f represents the focal length of the camera; h0 represents the initial height of the camera; α0 represents the initial installation angle; ο(δ1), ο(δ2), ο(δ3) represent error terms; Then the monitoring range of the surveillance camera at the Kth highway mileage pile is [K-dis1, K+dis1]; Step 2: Accurately match the roadside video images, and match the mileage posts and calibration objects along the roadside of the highway network with the real-time images of the video surveillance equipment: First, the coverage of video surveillance is divided. For each video point, between the upper and lower limit preset positions, i.e., the observation range of the farthest preset position, preset positions are gradually set according to the requirement that there is enough intersection of road monitoring range between the images taken by adjacent preset positions, which are called intermediate preset positions. Based on the camera imaging principle, a corresponding relationship is established between each intermediate preset position and the mileage pile number with longitude and latitude as the intermediate node, and based on the distance dis between each intermediate preset position and the surveillance camera, the pitch angle α, horizontal angle β and focal length f required to adjust the camera to the intermediate preset position are determined: Where R represents the radius of the earth; (x, y) represents the longitude and latitude coordinates of the target center point in the geodetic coordinate system; h0 represents the initial height of the surveillance camera; α0 represents the initial pitch angle of the surveillance camera installation; β0 represents the initial horizontal angle of the surveillance camera installation; (x0, y0) represents the longitude and latitude coordinates of the geodetic coordinate system of the initial installation point of the surveillance camera; U represents the height of the preset target; h represents the width of the surveillance camera target surface image; H represents the width of the target; u represents the height of the surveillance camera target surface image; o'(δ) represents the error term, k represents the number of lanes, and N represents the width of a single lane; Relying on the latitude and longitude position information, the preset position of the camera with specific angle and focal length parameters is precisely matched with the mileage pile number. That is, the roadside mileage piles, signs or traffic facilities with obvious mileage pile information near the route pile number are used as reference objects to obtain the real-time video or image of the precise mileage pile number information at each central preset position, so as to achieve precise matching between the real-time picture of the camera preset position and the mileage pile number. Step 3: roadside video surveillance image collection, collecting road network monitoring images and videos based on video surveillance equipment; Step 4: Highway inspection and maintenance: Based on the collected road network monitoring images and videos, the road network is inspected and maintained as needed.

Images