Road disease positioning method and system

Abstract

The invention discloses a road disease positioning method and system, and the method comprises the specific steps: S1, collecting the track information of road disease equipment through employing a positioning image collection device, and obtaining the track information of a collection terminal; timestamps are added to the collected road disease videos; s2, splitting the road disease video into frames, and determining the time of each frame of image; s3, positioning the position and course angle of each frame of image according to the time of each frame of image in combination with the trajectory information; s4, calculating the distance and angle between the road disease and the shooting point according to the position of the disease on the image; and S5, calculating a final disease position through the position and the course angle of each frame of picture in combination with the distance and the angle between the disease and the shooting point. According to the positioning method and system, the positioning accuracy of the road disease is improved, and the deviation of the position of the GPS positioning acquisition device and the position of the image acquisition device is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of road management, and in particular relates to a method and system for locating road damage. Background Art

[0002] Judging from the current development of highway maintenance, the identification of highway pavement defects has been carried out by digital and intelligent means. In the entire continuous process of road defect discovery, repair, evaluation and decision-making, the location of road defects is an important link, such as in the deduplication of road defects, file creation, repair, and evaluation of the effect after repair.

[0003] The following problems often occur when accurately locating defects. Problem 1: The location of road defects is collected using positioning systems such as GPS or Beidou. The location output by the positioning system is accurate, but the acquisition frequency cannot be consistent with the video acquisition frequency. The video can collect 25 frames per second, and the GPS / Beidou positioning system can only achieve continuous coordinate acquisition at the second level. Problem 2: The location of road defects collected in the video / picture is not the direct location of the GPS / Beidou positioning system. If the positioning data is used directly, it will inevitably lead to large deviations.

[0004] If the road defects cannot be accurately located, it is easy to waste the money and time invested later. In the existing technology, there is an inaccurate positioning problem when discovering road defects. If there are multiple defects in one frame of the picture, or one defect exists in multiple frames of photos, there is no better way to file, remove duplicates, and repair the road defects, resulting in the need for manual participation and reduced efficiency.

[0005] Chinese patent document CN 118032807 A discloses a method and system for high-precision positioning of road surface defects, the method comprising: determining the horizontal rotation angle and vertical rotation angle of the road surface defect sight line relative to the camera optical axis; according to the three-dimensional information data of the road where the camera is located, the horizontal rotation angle and vertical rotation angle of the road surface defect sight line relative to the camera optical axis, calculating along the center line of the road to determine the corresponding pile number point of the road surface defect and the distance value from the projection point of the road surface defect on the camera optical axis to the optical center; based on the current camera focal length, attitude value, position value in the world coordinate system and the distance value from the projection point of the road surface defect on the camera optical axis to the optical center, after conversion calculation from the camera coordinate system to the world coordinate system, the current position value of the road surface defect in the world coordinate system is obtained. The present invention can realize rapid and high-precision positioning of urban road surface defects.

[0006] Therefore, it is necessary to provide a method for locating road damages, which can improve the accuracy of locating road damages. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a method for locating road damage, which improves the accuracy of locating road damage and reduces the position deviation of GPS positioning acquisition and image acquisition devices.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is: the method for locating road damage specifically comprises the following steps:

[0009] S1: Using a positioning image acquisition device to collect the trajectory information of the road disease equipment, obtain the trajectory information of the acquisition terminal; and add a timestamp to the collected road disease video;

[0010] S2: split the road damage video into frames and determine the time of each frame image;

[0011] S3: Accurately locate the position and heading angle of each frame image by combining the time of each frame image with the trajectory information;

[0012] S4: Calculate the distance and angle between the road disease and the shooting point according to the location of the disease on the image;

[0013] S5: The final precise location of the disease is calculated through the position and heading angle of each frame of the image, combined with the distance and angle between the disease and the shooting point.

[0014] By adopting the above technical solution, the trajectory information of the acquisition terminal is collected by the positioning image acquisition device, the road disease video is split, the image position and heading angle are calculated by combining the time and trajectory of the image, and the angle is corrected to calculate the final precise location of the disease. The positioning accuracy of road diseases is improved and the deviation between the GPS positioning acquisition and the position of the image acquisition device is reduced.

[0015] Preferably, the positioning image acquisition device in step S1 is a device that integrates an image acquisition device and a position acquisition module; the image acquisition device in step S1 is a video acquisition module, and the position acquisition module is a positioning module; the video acquisition module is used to mark the time in each frame of the video during the video recording process, thereby obtaining a road disease video with each frame marked with time; at the same time, the positioning module acquires the movement trajectory of the video acquisition module to obtain trajectory data.

[0016] Preferably, the specific steps of step S2 are: identifying the time of each frame of the image through OCR software, extracting frame data of the same time and arranging them in order of frame numbers, and calculating the precise time of each frame of the image through linear interpolation, thereby determining the time of each frame of the image.

[0017] Preferably, the specific steps of step S3 are:

[0018] S31: taking out the exact time of a frame of image and naming it as a collection frame, finding out the previous track point data of the collection frame image and naming it as a previous track point;

[0019] S32: Find the next track point data of the acquisition frame image and name it as the next track point;

[0020] S33: Compare the heading angle deviations of the front track point and the rear track point; and then calculate the position and heading angle of the acquired frame image using different methods according to the heading angle deviations;

[0021] S34: when the heading angle deviation is less than or equal to 1 degree, the position and heading angle of the frame image are calculated using a linear interpolation method;

[0022] S35: When the heading angle deviation is greater than 1 degree, it is assumed that the motion trajectory of the image acquisition device is a standard circular arc from the front trajectory point to the rear trajectory point; the trajectory arc of the image acquisition device is calculated by the position of the front trajectory point, the heading angle of the front trajectory point, the position of the rear trajectory point, and the heading angle of the rear trajectory point; the position of this frame image is calculated by the linear difference method on the arc, and the heading angle of the acquired frame image is determined by the tangent direction of the arc at the position.

[0023] Preferably, in step S34, when the heading angle deviation is less than or equal to 1 degree, it is determined that the positioning image acquisition device moves in a straight line, and the position and heading angle of the frame image are calculated using a linear interpolation method, specifically:

[0024] According to the linear interpolation formula, the latitude, longitude and azimuth of the acquisition frame are expressed as:

[0025]

[0026] Among them, Δt is the time difference between the front track point and the rear track point; Δt1 is the time difference between the acquisition frame and the front track point; the position of the front track point is (lon1, lat1, bearing1); the position of the rear track point is (lon2, lat2, bearing2); lon3 is the longitude of the acquisition frame; lat3 is the latitude of the acquisition frame; bearing3 is the heading angle of the acquisition frame, then the position of the acquisition frame is (lon3, lat3, bearing3).

[0027] Preferably, the specific steps in step S35 are:

[0028] S351: Calculate the equation of the straight line passing through the previous track point and perpendicular to the heading angle of the previous track point. The formula is:

[0029]

[0030] Calculate the equation of the straight line that passes through the back track point and is perpendicular to the heading angle of the back track point. The formula is:

[0031]

[0032] Solving the system of equations for these two lines gives:

[0033]

[0034] Thus, we get the intersection point of the two straight lines mentioned above, the intersection point (x 0 ,y 0 ) is the center of the arc;

[0035] S352: Calculate the angle between the two straight lines; specifically: 0 ,y 0 ) is the center of the circle. The front trajectory point and the rear trajectory point are located on the circle respectively. By calculating the vector (x 0 -lon1,y 0 -lat1) and (x 0 -lon2,y 0 -lat2), and get the angle between the two straight lines;

[0036] in,

[0037] The formula for calculating the vector front trajectory position is: 1 =(x 0 -lon1,y 0 -lat1);

[0038] The formula for the trajectory position after calculating the vector is: 2 =(x 0 -lon2,y 0 -lat2);

[0039] The formula for calculating the angle α between the front and rear trajectory position vectors is:

[0040] Calculate the rotation angle α from the estimated point to the frame position before 13 The formula is:

[0041]

[0042] Among them, Δt1 is the acquisition frame shooting time minus the time of the previous track point; Δt is the time difference between the previous track point and the next track point, so as to calculate the center of the circle (x 0 ,y 0 );

[0043] S353: Calculate the center of the circle (x 0 ,y 0), and then rotate the angle α towards the direction of the backward track point as the starting point. 13 The point coordinates obtained are the acquisition position (lon3, lat3) of the image point in this frame, and the tangent direction of the arc point at this position is the acquisition heading angle (bearing3) of the acquisition frame image, thereby obtaining the position (lon, lat, bearing) of the positioning image acquisition device.

[0044] Preferably, the specific steps of step S4 are:

[0045] S41: First, it is necessary to take a sample image in a standard location after fixing the angle of the image acquisition device;

[0046] S42: measuring the horizontal distance and angle between all disease center points and the image acquisition device;

[0047] S43: measuring the angle and pixel difference (dx, dy) between all diseases and the center point of the bottom of the image in the captured image;

[0048] S44: using a cubic spline interpolation method, constructing a function of the longitudinal pixel difference (dy) and the distance, called a distance calculation function, and constructing an angle calculation function of the pixel difference (dx, dy) and the actual angle;

[0049] S45: When the defect point is found in the image, the difference between the pixel coordinates of the center point and the center point at the bottom of the image is used to calculate the distance x from the defect to the video acquisition module through the distance calculation function, and then the angle y between the defect and the heading of the video acquisition module is calculated through the angle calculation function, so that the relative position of the road defect and the shooting point is obtained as (x, y).

[0050] Preferably, the specific steps of step S44 are:

[0051] S441: The cubic spline function is constructed between every two adjacent data points (dis i ,dy i ) and (dis i+1 ,dy i+1 ), define a cubic polynomial S i (dy) to approximate the relationship between the distance dis and the pixel difference dy; each spline function S i (d) shall be in the form of:

[0052] S i (dy) = a i +b i (dy-dy i )+c i (dy-dy i )2 +d i (dy-dy i ) 3 ;

[0053] Among them, a i 、b i 、c i d i are the coefficients of each segment, which need to be obtained by solving a set of equations;

[0054] To ensure the smoothness and continuity of the curve, the cubic spline function S i (dis)The conditions that need to be met include:

[0055] Interpolation condition: The spline function passes through all data points, that is:

[0056] S i (dy i )=dis i ;

[0057] S i (dy i+1 )=dis i+1 ;

[0058] Continuity condition: At each interior point dis i At , the function value, derivative, and second-order derivative of the spline are continuous:

[0059] S i (dy i )=S i+1 (dy i+1 );

[0060] S i ′ (dy i+1 )=S i ′ +1 (dy i+1 );

[0061] S i ″ (dy i+1 )=S i ″ +1 (dy i+1 );

[0062] Boundary conditions: There are two types of boundary conditions, namely natural boundary conditions and fixed boundary conditions;

[0063] The natural boundary condition is that the second-order derivative of the spline at both ends is zero:

[0064] S′ 1 ′ (dy 1 )=0;

[0065] S ′ n ′ -1 (dy n )=0;

[0066] Fixed boundary conditions: specify the slope at both ends of the spline, that is, the first-order derivative value;

[0067] S442: Finally, solve the coefficients: organize the interpolation conditions, continuity conditions and boundary conditions into a linear equation system, and solve each segment of the spline function S i The coefficient a of (dy) i 、b i 、c i d i ; After the coefficients are found, the interpolation function S(d) is obtained, and the formula is:

[0068] S(dy)=S i (dy)dy i ≤dy≤dy i+1 .

[0069] Preferably, the specific steps of step S5 are: the position (lon, lat, bearing) of the positioning image acquisition device is calculated by step S3, and the deviation angle θ of the road defect point from the acquisition distance dis is calculated by step S4; the direction angle of the road defect point is the heading angle bearing of the positioning image acquisition device plus the deviation angle θ; specifically:

[0070] S51: Assume the direction angle of the disease point is θ total , the calculation formula is:

[0071] θ total = bearing + θ;

[0072] S52: The distance and angle are then converted into plane coordinate offsets, that is, on the curved surface of the earth, the distance dis and the direction θ total The plane coordinates converted into relative positions are the easting offset dx and the northing offset dy, and the formula is:

[0073] dx = dis·cos(θ total );

[0074] dy=dis·sin(θ total );

[0075] S53: Convert the plane offset to the latitude and longitude offset, that is, use the conversion formula of the geodetic coordinate system, assume that the radius of the earth is R=6378137 meters (WGS-84 standard), and convert the offsets dx and dy into latitude and longitude offsets; the calculation formula of the latitude offset Δlat is:

[0076]

[0077] The calculation formula for the longitude offset Δlon is:

[0078]

[0079] S54: Finally, the longitude and latitude of the disease point are calculated, that is, the offset is added to the original longitude and latitude of the device to obtain the longitude and latitude of the road disease point. The formula is:

[0080] lat1=lat+Δlat;

[0081] lon1=lon+Δlon;

[0082] Thus the final location of the road damage point can be obtained.

[0083] Another technical problem to be solved by the present invention is to provide a road damage positioning system, which improves the positioning accuracy of road damage and reduces the position deviation of GPS positioning acquisition and image acquisition devices.

[0084] In order to solve the above technical problems, the technical solution adopted by the present invention is: the road disease positioning system at least includes a positioning image acquisition device and a cloud server, and the positioning image acquisition device is connected to the cloud server through a network; the positioning image acquisition device includes a position acquisition module and a video acquisition module, and the position acquisition module and the video acquisition module are integrated into one; the position acquisition module is used to collect position information, the video acquisition module is responsible for collecting pictures and video data of the road surface, and the cloud server performs precise positioning by collecting data from the positioning image acquisition device.

[0085] By adopting the above technical solution, the position acquisition module and the video acquisition module are integrated into one. Since the position acquisition module and the video acquisition module are integrated into one and their positions are in an upper and lower relationship, the final positioning deviation caused by the position deviation between the two modules can be reduced; that is, the position deviation between the GPS positioning acquisition and the image acquisition device and the position deviation between the GPS acquisition and the video acquisition module can be reduced.

[0086] Compared with the prior art, the present invention has the following beneficial effects: using a positioning image acquisition device that integrates a position acquisition module and a video acquisition module can reduce the deviation between the image acquired by the video acquisition module and the positioning data acquired by the position acquisition module, which helps to improve positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 Schematic diagram of the road disease positioning system of the present invention; wherein, 1- cloud server; 2- positioning image acquisition device; 201- position acquisition module; 202- video acquisition module;

[0088] Figure 2 A flow chart of a method for locating road hazards of the present invention;

[0089] Figure 3 A schematic diagram of obtaining a road disease reference sample for a road disease location method of the present invention;

[0090] Figure 4 A schematic diagram of location information collected by the method for locating road hazards of the present invention;

[0091] Figure 5 A time diagram is marked on each frame of the image of the road damage positioning method of the present invention;

[0092] Figure 6 A schematic diagram of the location of each frame of image acquisition based on a straight line in the method for locating road damage of the present invention;

[0093] Figure 7 A schematic diagram of the location of each frame of image acquisition based on an arc in the method for locating road damages of the present invention;

[0094] Figure 8 This is a schematic diagram of calculating the precise location of a road defect based on known collection points according to the method for locating road defects of the present invention. DETAILED DESCRIPTION

[0095] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.

[0096] Example: Figure 2 As shown, the method for locating road damage specifically includes the following steps:

[0097] S1: Use a positioning image acquisition device to collect the trajectory information of the road disease equipment and obtain the trajectory information of the acquisition terminal; and add a timestamp to the collected road disease video; the positioning image acquisition device in step S1 is a device that integrates an image acquisition device and a position acquisition module; the image acquisition device in step S1 is a video acquisition module, and the position acquisition module is a positioning module; use the video acquisition module to mark the time in each frame of the video during the video recording process, so as to obtain a road disease video with a time in each frame; at the same time, the positioning module acquires the movement trajectory of the video acquisition module to obtain trajectory data; in this embodiment, the video acquisition module is a camera, and the position acquisition module is a GPS or Beidou or any module or device that can collect such information; through the OBS video recording software, connect the camera to record the video, and add a timestamp to each frame of the video; such as Figure 4 The table shows the collected trajectory information, where the collected trajectory information includes time (datetime), location information (longitude, latitude), heading angle (bearing), and speed (speed); generally, the collected location information is a continuous record in seconds;

[0098] S2: split the road disease video into frames and determine the time of each frame; the specific steps of step S2 are: since the video recording software only supports second-level time tags, the time corresponding to the video within one second is the same; identify the time of each frame of the image through OCR software, extract the frame data of the same time and arrange them in order of frame numbers, and calculate the precise time of each frame of the image through linear interpolation, so as to determine the time of each frame of the image; Figure 5 As shown, all frame data within one second, a total of 25 frames, are taken out, and the time difference of each frame image is 0.04 seconds through linear interpolation method, and the precise time of each frame is calculated in turn;

[0099] S3: Accurately locate the position and heading angle of each frame image by combining the time of each frame image with the trajectory information;

[0100] The specific steps of step S3 are:

[0101] S31: taking out the exact time of a frame of image and naming it as a collection frame, finding out the previous track point data of the collection frame image and naming it as a previous track point;

[0102] S32: Find the next track point data of the acquisition frame image and name it as the next track point;

[0103] S33: Compare the heading angle deviations of the front track point and the rear track point; and then calculate the position and heading angle of the acquired frame image using different methods according to the heading angle deviations;

[0104] S34: when the heading angle deviation is less than or equal to 1 degree, the position and heading angle of the frame image are calculated using a linear interpolation method;

[0105] In step S34, when the heading angle deviation is less than or equal to 1 degree, it is determined that the positioning image acquisition device moves in a straight line, and the position and heading angle of the frame image are calculated using linear interpolation, specifically:

[0106] According to the linear interpolation formula, the latitude, longitude and azimuth of the acquisition frame are expressed as:

[0107]

[0108] Among them, Δt is the time difference between the front track point and the rear track point; Δt1 is the time difference between the acquisition frame and the front track point; the position of the front track point is (lon1, lat1, bearing1); the position of the rear track point is (lon2, lat2, bearing2); lon3 is the longitude of the acquisition frame; lat3 is the latitude of the acquisition frame; bearing3 is the heading angle of the acquisition frame, then the position of the acquisition frame is (lon3, lat3, bearing3); Figure 6 As shown, the frame time is: 10:31:01.36, through which the front track point "time: 10:31:01.00; position: (lon1, lat1, bearing1)" and the back track point "time: 10:31:02.00; position: (lon2, lat2, bearing2)" are found;

[0109] The time difference between the front track point and the back track point: Δt = 10:31:02.00-10:31:01.00 = 1 second;

[0110] The time difference between the acquisition frame and the previous trajectory point: Δt1 = 10:31:01.36-10:31:01.00 = 0.36 seconds;

[0111] Will Substituting the result into the linear interpolation formula of the acquisition position, we can calculate (lon3, lat3, bearing3);

[0112]

[0113] lon3=lon1+(lon2-lon1)×0.36;

[0114] lat3=lat1+(lat2-lat1)×0.36;

[0115] bearing3=bearing1+(bearing2-bearing1)×0.36;

[0116] S35: When the heading angle deviation is greater than 1 degree, it is assumed that the motion trajectory of the image acquisition device is a standard arc from the front trajectory point to the rear trajectory point; the trajectory arc of the image acquisition device is calculated by the position of the front trajectory point, the heading angle of the front trajectory point, the position of the rear trajectory point, and the heading angle of the rear trajectory point; the position of this frame image is calculated by the linear difference method on the arc, and the heading angle of the acquired frame image is determined by the tangent direction of the arc where the position is located; Figure 7 As shown, the frame time is: 10:31:01.36. Through this time, we can find the front track point "time: 10:31:01.00; position: (lon1, lat1, bearing1)" and the back track point "time: 10:31:02.00; position: (lon2, lat2, bearing2)". We need to calculate the point acquisition device to acquire the frame position (lon3, lat3, bearing3).

[0117] The specific steps in step S35 are:

[0118] S351: Calculate the equation of the straight line passing through the previous track point and perpendicular to the heading angle of the previous track point. The formula is:

[0119]

[0120] Calculate the equation of the straight line that passes through the back track point and is perpendicular to the heading angle of the back track point. The formula is:

[0121]

[0122] Solving the system of equations for these two lines gives:

[0123]

[0124] Thus, we get the intersection point of the two straight lines mentioned above, the intersection point (x 0 ,y 0 ) is the center of the arc;

[0125] S352: Calculate the angle between the two straight lines; specifically: 0 ,y 0 ) is the center of the circle. The front trajectory point and the rear trajectory point are located on the circle respectively. By calculating the vector (x 0 -lon1,y 0 -lat1) and (x 0 -lon2,y 0 -lat2), and get the angle between the two straight lines;

[0126] in,

[0127] The formula for calculating the vector front trajectory position is: 1 =(x 0 -lon1,y 0 -lat1);

[0128] The formula for the trajectory position after calculating the vector is: 2 =(x 0 -lon2,y 0 -lat2);

[0129] The formula for calculating the angle α between the front and rear trajectory position vectors is:

[0130] Calculate the rotation angle α from the estimated point to the frame position before 13 The formula is:

[0131]

[0132] Among them, Δt1 is the acquisition frame shooting time minus the time of the previous track point; Δt is the time difference between the previous track point and the next track point;

[0133] Δt = 10:31:02.00-10:31:01.00 = 1 second;

[0134] Δt1=10:31:01.36-10:31:01.00=0.36 seconds;

[0135] Substitute Δt1 = 0.36 seconds and Δt = 1 second;

[0136] In this embodiment, specifically:

[0137] S353: Calculate the center of the circle (x 0 ,y 0 ), and then rotate the angle α towards the direction of the backward track point as the starting point. 13 The point coordinates are obtained as the acquisition position (lon3, lat3) of the image point in this frame, and the tangent direction of the arc point at this position is the acquisition heading angle (bearing3) of the acquisition frame image, thereby obtaining the position (lon, lat, bearing) of the positioning image acquisition device;

[0138] S4: Calculate the distance and angle between the road disease and the shooting point based on the position of the disease on the image; determine the relative position between the road disease and the shooting point;

[0139] The specific steps of step S4 are:

[0140] S41: First, it is necessary to take a sample image in a standard location after fixing the angle of the image acquisition device; Figure 3 The figure shows a schematic diagram of a standard site, which includes the location of the acquisition equipment and preset marking points. The longitudinal distance and angle from each marking point to the camera position are measured in advance. The acquisition equipment is set up at the camera acquisition position and a picture is taken.

[0141] S42: measuring the horizontal distance and angle between the center points of all diseases and the image acquisition device;

[0142] S43: measuring the angle and pixel difference (dx, dy) between all diseases and the center point of the bottom of the image in the captured image;

[0143] S44: using a cubic spline interpolation method, constructing a function of the longitudinal pixel difference (dy) and the distance, called a distance calculation function, and constructing an angle calculation function of the pixel difference (dx, dy) and the actual angle;

[0144] Specifically: according to the pixel difference between the marker point and the bottom center point [(dx1, dy1), (dx2, dy2), ..., (dxn, dyn)], measure the longitudinal distance [dis1, dis2, ..., disn] between the marker point and the bottom center point; use the cubic spline interpolation method to calculate the relationship between the pixel difference and the distance point;

[0145] Stand on the road and shoot forward for a distance. Find several marking points on the road and measure the distance between the marking points and the shooting points [dis1, dis2, ..., disn]. Measure the pixel difference between the marking points and the bottom of the shot [dy1, dy2, ..., dyn)] and use the cubic spline interpolation method to calculate the relationship between the pixel difference and the distance point.

[0146] The specific steps of step S44 are:

[0147] S441: The cubic spline function is constructed between every two adjacent data points (dis i ,dy i ) and (dis i+1 ,dy i+1 ), define a cubic polynomial S i (dy) to approximate the relationship between the distance dis and the pixel difference dy; each spline function S i (d) shall be in the form of:

[0148] S i(dy) = a i +b i (dy-dy i )+c i (dy-dy i ) 2 +d i (dy-dy i ) 3 ;

[0149] Among them, a i , b i 、c i d i are the coefficients of each segment, which need to be obtained by solving a set of equations;

[0150] To ensure the smoothness and continuity of the curve, the cubic spline function S i (dis)The conditions that need to be met include:

[0151] Interpolation condition: The spline function passes through all data points, that is:

[0152] S i (dy i )=dis i ;

[0153] S i (dy i+1 )=dis i+1 ;

[0154] Continuity condition: At each interior point dis i At , the function value, derivative, and second-order derivative of the spline are continuous:

[0155] S i (dy i )=S i+1 (dy i+1 );

[0156] S i ′ (dy i+1 )=S i ′ +1 (dy i+1 );

[0157] S i ″ (dy i+1 )=S i ″ +1 (dy i+1 );

[0158] Boundary conditions: There are two types of boundary conditions, namely natural boundary conditions and fixed boundary conditions;

[0159] The natural boundary condition is that the second-order derivative of the spline at both ends is zero:

[0160] S ′ 1 ′ (dy 1 )=0;

[0161] S ′ n ′ -1 (dy n )=0;

[0162] Fixed boundary conditions: specify the slope at both ends of the spline, that is, the first-order derivative value;

[0163] S442: Finally, solve the coefficients: organize the interpolation conditions, continuity conditions and boundary conditions into a linear equation system, and solve each segment of the spline function S i The coefficient a of (dy) i , b i 、c i d i ; After the coefficients are found, the interpolation function S(d) is obtained, and the formula is:

[0164] S(dy)=S i (dy)dy i ≤dy≤dy i+1 ;

[0165] When the defect point is found in the image, the difference between the pixel coordinates of the center point and the center point at the bottom of the image is used to calculate the distance x from the defect to the video acquisition module through the distance calculation function, and the angle y between the defect and the heading of the video acquisition module is calculated through the angle calculation function; the relative position of the road defect and the shooting point is (x, y);

[0166] S45: When the defect point is found in the image, the distance x from the defect to the video acquisition module is calculated by using the difference between the pixel coordinates of the center point and the center point at the bottom of the image through a distance calculation function, and then the angle y between the defect and the heading of the video acquisition module is calculated through an angle calculation function, so that the relative position of the road defect and the shooting point is obtained as (x, y);

[0167] S5: The final precise location of the disease is calculated by combining the position and heading angle of each frame of the image with the distance and angle between the disease and the shooting point;

[0168] The specific steps of step S5 are: the position (lon, lat, bearing) of the positioning image acquisition device is calculated by step S3, and the deviation angle θ of the road defect point from the acquisition distance dis is calculated by step S4; the direction angle of the road defect point is the heading angle bearing of the positioning image acquisition device plus the deviation angle θ; specifically:

[0169] S51: Assume the direction angle of the disease point is θ total , the calculation formula is:

[0170] θ total = bearing + θ;

[0171] S52: The distance and angle are then converted into plane coordinate offsets, that is, on the curved surface of the earth, the distance dis and the direction θ total The plane coordinates converted into relative positions are the easting offset dx and the northing offset dy, and the formula is:

[0172] dx = dis·cos(θ total );

[0173] dy=dis·sin(θ total );

[0174] S53: Convert the plane offset to the latitude and longitude offset, that is, use the conversion formula of the geodetic coordinate system, assume that the radius of the earth is R=6378137 meters (WGS-84 standard), and convert the offsets dx and dy into latitude and longitude offsets; the calculation formula of the latitude offset Δlat is:

[0175]

[0176] The calculation formula for the longitude offset Δlon is:

[0177]

[0178] S54: Finally, the longitude and latitude of the disease point are calculated, that is, the offset is added to the original longitude and latitude of the device to obtain the longitude and latitude of the road disease point. The formula is:

[0179] lat1=lat+Δlat;

[0180] lon1=lon+Δlon;

[0181] Thus, the final location of the road damage point can be obtained, such as Figure 8 shown.

[0182] like Figure 1As shown, the road disease positioning system at least includes a positioning image acquisition device 2 and a cloud server 1, and the positioning image acquisition device 2 is connected to the cloud server 1 through a network; the positioning image acquisition device includes a position acquisition module 201 and a video acquisition module 202, and the position acquisition module 201 and the video acquisition module 202 are integrated into one; the position acquisition module 201 is used to collect position information, and the video acquisition module 202 is responsible for collecting pictures and video data of the road surface. The cloud server 1 performs precise positioning by collecting data from the positioning image acquisition device. In this embodiment, the video acquisition module 202 is a camera, and the position acquisition module 201 is a GPS or Beidou or any module or device that can collect such information.

[0183] For ordinary technicians in this field, the specific embodiments are only illustrative descriptions of the present invention. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concepts and technical solutions of the present invention, or the concepts and technical solutions of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A method for locating road damage, characterized in that: The specific steps include: S1: Using a positioning image acquisition device to collect the trajectory information of the road disease equipment, obtain the trajectory information of the acquisition terminal; and add a timestamp to the collected road disease video; S2: split the road damage video into frames and determine the time of each frame image; S3: locate the position and heading angle of each frame image by combining the time of each frame image with the trajectory information; S4: Calculate the distance and angle between the road disease and the shooting point according to the location of the disease on the image; S5: The final disease location is calculated by combining the position and heading angle of each frame of the image with the distance and angle between the disease and the shooting point.

2. The method for locating road damage according to claim 1, characterized in that: The positioning image acquisition device in step S1 is a device that integrates an image acquisition device and a position acquisition module; the image acquisition device is a video acquisition module, and the position acquisition module is a positioning module; the video acquisition module is used to mark the time in each frame of the video during the video recording process, thereby obtaining a road disease video with each frame marked with time; at the same time, the positioning module collects the movement trajectory of the video acquisition module to obtain trajectory data.

3. The method for locating road damage according to claim 2, characterized in that: The specific steps of step S2 are: identifying the time of each frame of the image through OCR software, extracting frame data of the same time and arranging them in order of frame numbers, and calculating the precise time of each frame of the image through linear interpolation, thereby determining the time of each frame of the image.

4. The method for locating road damage according to claim 2, characterized in that: The specific steps of step S3 are: S31: taking out the exact time of a frame of image and naming it as a collection frame, finding out the previous track point data of the collection frame image and naming it as a previous track point; S32: Find the next track point data of the acquisition frame image and name it as the next track point; S33: Compare the heading angle deviations of the front track point and the rear track point; and then calculate the position and heading angle of the acquired frame image using different methods according to the heading angle deviations; S34: when the heading angle deviation is less than or equal to 1 degree, the position and heading angle of the frame image are calculated using a linear interpolation method; S35: When the heading angle deviation is greater than 1 degree, it is assumed that the motion trajectory of the image acquisition device moves from the front trajectory point to the rear trajectory point in a standard circular arc; The trajectory arc of the image acquisition device is calculated by the position of the front trajectory point, the heading angle of the front trajectory point, the position of the rear trajectory point, and the heading angle of the rear trajectory point; then the position of this frame image is calculated by the linear difference method on the arc, and the heading angle of the acquired frame image is determined by the tangent direction of the arc at the position.

5. The method for locating road damage according to claim 2, characterized in that: In step S34, when the heading angle deviation is less than or equal to 1 degree, it is determined that the positioning image acquisition device moves in a straight line, and the position and heading angle of the frame image are calculated using linear interpolation, specifically: According to the linear interpolation formula, the latitude, longitude and azimuth of the acquisition frame are expressed as: Among them, Δt is the time difference between the front track point and the rear track point; Δt1 is the time difference between the acquisition frame and the front track point; the position of the front track point is (lon1, lat1, bearing1); the position of the rear track point is (lon2, lat2, bearing2); lon3 is the longitude of the acquisition frame; lat3 is the latitude of the acquisition frame; bearing3 is the heading angle of the acquisition frame, then the position of the acquisition frame is (lon3, lat3, bearing3).

6. The method for locating road damage according to claim 5, characterized in that: The specific steps in step S35 are: S351: Calculate the equation of the straight line passing through the previous track point and perpendicular to the heading angle of the previous track point. The formula is: Calculate the equation of the straight line that passes through the back track point and is perpendicular to the heading angle of the back track point. The formula is: Solving the system of equations for these two lines gives: Thus, the intersection point of the two straight lines is obtained, and the intersection point (x0, y0) is the center of the arc; S352: Calculate the angle between the two straight lines. Specifically, since the intersection point (x0, y0) is the center of the circle, the front trajectory point and the rear trajectory point are respectively located on the circle, and the angle between the vectors (x0-lon1, y0-lat1) and (x0-lon2, y0-lat2) is calculated to obtain the angle between the two straight lines. in, The formula for calculating the trajectory position before the vector is: v1 = (x0-lon1, y0-lat1); The formula for the trajectory position after calculating the vector is: v2 = (x0-lon2, y0-lat2); The formula for calculating the angle α between the front and rear trajectory position vectors is: Calculate the rotation angle α from the estimated point to the frame position before 13 The formula is: Wherein, Δt1 is the acquisition frame shooting time minus the time of the previous track point; Δt is the time difference between the previous track point and the next track point, so as to calculate the center of the circle (x0, y0); S353: After calculating the center of the circle (x0, y0), the previous trajectory point is used as the starting point to rotate toward the direction of the backward trajectory point by an angle α 13 The point coordinates obtained are the acquisition position (lon3, lat3) of the image point in this frame, and the tangent direction of the arc point at this position is the acquisition heading angle (bearing3) of the acquisition frame image, thereby obtaining the position (lon, lat, bearing) of the positioning image acquisition device.

7. The method for locating road damage according to claim 5, characterized in that: The specific steps of step S4 are: S41: First, it is necessary to take a sample image in a standard location after fixing the angle of the image acquisition device; S42: measuring the horizontal distance and angle between all disease center points and the image acquisition device; S43: measuring the angle and pixel difference (dx, dy) between all diseases and the center point at the bottom of the image in the captured image; S44: using a cubic spline interpolation method, constructing a function of the longitudinal pixel difference (dy) and the distance, called a distance calculation function, and constructing an angle calculation function of the pixel difference (dx, dy) and the actual angle; S45: When the defect point is found in the image, the difference between the pixel coordinates of the center point and the center point of the bottom of the image is used to calculate the distance x from the defect to the video acquisition module through the distance calculation function, and then the angle y between the defect and the heading of the video acquisition module is calculated through the angle calculation function, so that the relative position of the defect and the shooting point is obtained as (x, y).

8. The method for locating road damage according to claim 5, characterized in that: The specific steps of step S44 are: S441: The cubic spline function is constructed between every two adjacent data points (dis i ,dy i ) and (dis i+1 ,dy i+1 ), define a cubic polynomial S i (dy) to approximate the relationship between the distance dis and the pixel difference dy; each spline function S i (d) shall be in the form of: S i (dy)=a i +b i (to i )+c i (to i ) 2 +d i (to i ) 3 4 Among them, a i 、b i 、c i d i are the coefficients of each segment, which need to be obtained by solving a set of equations; To ensure the smoothness and continuity of the curve, the cubic spline function S i (dis)The conditions that need to be met include: Interpolation condition: The spline function passes through all data points, that is: S i (of i )=this i 4 S i (of i+1 )=this i+1 4 Continuity condition: At each interior point dis i At , the function value, derivative, and second-order derivative of the spline are continuous: S i (of i )=S i+1 (of i+1 ): S i ′ (of i+1 )=S i ′ +1 (of i+1 ): S i "(of i+1 )=S i ″ +1 (of i+1 ): Boundary conditions: There are two types of boundary conditions, namely natural boundary conditions and fixed boundary conditions; The natural boundary condition is that the second-order derivative of the spline at both ends is zero: S ′ 1 ′ (dy1)=0; S ′ n ′ -1 (of n )=0: Fixed boundary conditions: specify the slope at both ends of the spline, that is, the first-order derivative value; S442: Finally, solve the coefficients: organize the interpolation conditions, continuity conditions and boundary conditions into a linear equation system, and solve each segment of the spline function S i The coefficient a of (dy) i 、b i 、c i d i ; After the coefficients are found, the interpolation function S(d) is obtained, and the formula is: S(of)=S i (to) to i ≤dy≤dy i+1 。 9. The method for locating road damage according to claim 5, characterized in that: The specific steps of step S5 are: the position (lon, lat, bearing) of the positioning image acquisition device is calculated by step S3, and the deviation angle θ of the defect point from the acquisition distance dis is calculated by step S4; the direction angle of the defect point is the heading angle bearing of the positioning image acquisition device plus the deviation angle θ; specifically: S51: Assume the direction angle of the disease point is θ total , the calculation formula is: i total =bearing+θ; S52: The distance and angle are then converted into plane coordinate offsets, that is, on the curved surface of the earth, the distance dis and the direction θ total The plane coordinates converted into relative positions are the easting offset dx and the northing offset dy, and the formula is: dx=dis·cos(θ total ); dysdis·sin(θ total )4 S53: Convert the plane offset into longitude and latitude offsets, that is, use the conversion formula of the geodetic coordinate system, assume that the radius of the earth is R=6378137 meters, and convert the offsets dx and dy into latitude and longitude offsets; the calculation formula of the latitude offset Δlat is: The calculation formula for the longitude offset Δlon is: S54: Finally, the longitude and latitude of the disease point are calculated, that is, the offset is added to the original longitude and latitude of the device to obtain the longitude and latitude of the disease point. The formula is: lat1=lat+Δlat; lon1=lon+Δlon; Thus the final location of the disease point is obtained.

10. A road disease positioning system, characterized in that: The system at least comprises a positioning image acquisition device and a cloud server, wherein the positioning image acquisition device is connected to the cloud server via a network; the positioning image acquisition device comprises a position acquisition module and a video acquisition module, and the position acquisition module and the video acquisition module are integrated into one; the position acquisition module is used to acquire position information, the video acquisition module is responsible for acquiring pictures and video data of the road surface, and the cloud server performs positioning by collecting data from the positioning image acquisition device.

Images