Road disaster damage real-time detection method and system based on unmanned aerial vehicle remote sensing

Abstract

The invention relates to the technical field of detection and positioning, and discloses a road disaster damage real-time detection method and system based on unmanned aerial vehicle remote sensing. According to the method, road image data of a to-be-detected area are collected, the collected road image data are processed, a coordinate system and a reference point are set at the same time, and based on the set coordinate system and the reference point, an unmanned aerial vehicle is accurately positioned in a real-time positioning mode; after positioning is completed, image data of a to-be-detected area are collected in real time through the unmanned aerial vehicle, the collected real scene image data are corrected, meanwhile, feature extraction is conducted on the obtained corrected image data and processed road image data, and a road disaster damage area is preliminarily determined in a feature comparison mode; according to the method, the unmanned aerial vehicle is used to determine the road disaster and damage area, the image collected by the unmanned aerial vehicle in real time in the preliminarily determined road disaster and damage area is selected, the selected image is analyzed through an image analysis method, the road disaster and damage position in the analyzed image is positioned, and the reliability of road disaster and damage real-time detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of detection and positioning, and in particular to a real-time detection method and system for road damage based on unmanned aerial vehicle remote sensing. Background Art

[0002] Roads are the lifeline of transportation, but current road detection still mainly uses traditional video acquisition and visual interpretation. This method has poor timeliness and low efficiency and requires a lot of manpower.

[0003] The existing publicly applied patent CN105842707A, this method uses drones to collect remote sensing images of the monitored grassland area in real time, and calculates the grassland coverage data and grass layer height data of the monitored grassland area based on the acquired drone remote sensing images; and obtains the aboveground biomass per unit area of ​​grassland based on the acquired grassland coverage data and grass layer height data; but because the method only uses drones to collect images and calculate the biological information in the images, the dynamic nature of the biological targets is ignored, resulting in inaccurate measurements and certain limitations. Summary of the invention

[0004] 1. Technical issues to be resolved In view of the shortcomings of the existing technology, the present invention provides a real-time detection method and system for road damage based on UAV remote sensing, which has the advantages of accuracy, real-time and high efficiency, and solves the problems of poor timeliness and low efficiency of traditional video acquisition and visual interpretation.

[0005] (II) Technical solution In order to solve the above-mentioned technical problems of poor timeliness and low efficiency of traditional video acquisition and visual interpretation, the present invention provides the following technical solutions: The present invention discloses a method for real-time detection of road damage based on unmanned aerial vehicle remote sensing, which specifically comprises the following steps: S1, collecting road image data of the area to be detected, and processing the collected road image data to obtain processed road image data; S2. Setting a coordinate system and a reference point based on the processed road image data; S3, based on the set coordinate system and reference points, accurately locate the drone through real-time positioning; S4, collecting image data of the area to be detected in real time through the drone, and correcting the collected real scene image data to obtain corrected image data; S5, extracting features from the corrected image data and the processed road image data, and preliminarily determining the road damage area by feature comparison; S6. Select images collected by drones in real time within the preliminarily determined road damage area, analyze the selected images using an image analysis method, and locate the road damage position in the analyzed images.

[0006] The present invention collects road image data of the area to be detected, processes the collected road image data, sets a coordinate system and a reference point based on the processed road image data, and accurately positions the unmanned aerial vehicle (UAV) through a real-time positioning method based on the set coordinate system and reference point; after the positioning is completed, the image data of the area to be detected is collected in real time by the UAV, and the collected real scene image data is corrected, and features are extracted from the corrected image data and the processed road image data, and the road damage area is preliminarily determined through a feature comparison method, and the image collected by the UAV in real time within the preliminarily determined road damage area is selected, the selected image is analyzed through an image analysis method, and the road damage position in the analyzed image is located, thereby improving the reliability of real-time detection of road damage.

[0007] Preferably, the collecting of the road image data of the area to be detected and processing the collected road image data to obtain the processed road image data comprises the following steps: S11, setting a window, wherein the window includes the road image pixels to be processed in the collected road image data and the neighborhood pixels around the road image; S12, for each pixel in the window, calculating the weight of each pixel; S13, calculating the weight of each pixel respectively; S14, multiplying the weight of each pixel by its corresponding pixel value, and summing the weighted values ​​of all pixels; S15, dividing the weighted sum by the total weight to obtain smoothed road image data; The smoothed road image data is set as the processed road image data.

[0008] Preferably, for each pixel in the window, calculating the weight of each pixel comprises the following steps: The pixel weight calculation formula is as follows: ; ; in, represents the Gaussian distribution function, Represents the horizontal coordinate of the pixel point, Represents the vertical coordinate of the pixel point. represents the pixel weight, represents the natural logarithm, Represents the horizontal coordinate of the center pixel, Represents the vertical coordinate of the center pixel, Represents the horizontal coordinate of the central neighborhood pixel, Represents the ordinate of the central neighborhood pixel, Represents the Gaussian function parameters.

[0009] The present invention processes the collected road image data by setting a window, and completes the smoothing process of the collected road image data by calculating the weight of each pixel in the window, thereby improving the reliability of the road image data.

[0010] Preferably, the setting of the coordinate system and the reference point based on the processed road image data comprises the following steps: S21, setting a reference point in the area to be detected; Based on the processed road image data collected in the area to be detected, the intersection of the main road and the non-main road is set as a reference point; When there is only one main road or only one non-main road in the area to be detected, the starting point is set as the first reference point, and the distance threshold between the reference points is set. According to the set distance threshold, the reference points are set in equal proportion; S22, based on all the reference points in the area to be detected, number each reference point by setting a UID and save it; S23. Establish a three-dimensional coordinate system based on the first reference point, set the first reference point as the coordinate origin, record the coordinate position of each reference point corresponding to the number, and save it.

[0011] The present invention sets reference points according to environmental parameters in the area to be detected, establishes a three-dimensional coordinate system based on all reference points in the area to be detected, and records the coordinates of each reference point, thereby improving the accuracy of detecting the area to be detected.

[0012] Preferably, the method of accurately positioning the drone in real-time positioning based on the set coordinate system and reference points includes the following steps: S31, select the GPS positioning satellite and receive the UAV positioning information data; The drone positioning information data includes: GPS positioning information and ultrasonic measurement information; Send GPS positioning information through the drone signal transmitter and wait for feedback from the GPS positioning satellite; Set the first set of received GPS positioning signals as the GPS positioning information of the current drone; S32, calculating the horizontal position information of the drone based on the received GPS positioning information; Assuming the propagation speed of GPS positioning information is the speed of light, the formula for calculating the horizontal position information of the drone through the time difference between transmitting GPS positioning information and receiving GPS positioning information is as follows: ; in, Indicates the horizontal position information of the drone. represents the speed of light, Indicates the time of receiving GPS positioning information. Indicates the time when GPS positioning information is transmitted; S33, calculating the vertical height information of the UAV based on the received ultrasonic measurement information; The formula for calculating the vertical height information of the drone based on the time difference between the drone transmitting the ultrasonic measurement information and receiving the ultrasonic measurement information bounced off the ground is as follows: ; in, Indicates the vertical height information of the drone. represents the speed of sound, Indicates the time of receiving ultrasonic measurement information. Indicates the time of transmitting ultrasonic measurement information; S34, the obtained UAV horizontal position information and UAV vertical height information are expressed in a set coordinate system.

[0013] The present invention selects a GPS positioning satellite and receives positioning information data of a UAV, calculates the horizontal position information of the UAV based on the received GPS positioning information, calculates the vertical height information of the UAV based on the received ultrasonic measurement information, and represents the obtained horizontal position information and vertical height information of the UAV in a set coordinate system, thereby realizing real-time positioning of the UAV.

[0014] Preferably, the real-time collection of image data of the area to be detected by the drone and correction of the collected real scene image data to obtain the corrected image data include the following steps: S41, selecting a reference point in the image data of the area to be detected collected in real time, and recording the coordinates of the reference point in the image data; S42, correcting the collected real scene image data based on the saved reference point coordinates and the drone positioning coordinates.

[0015] Preferably, the correcting of the collected real scene image data based on the saved reference point coordinates and the drone positioning coordinates comprises the following steps: The correction transformation formula is as follows: ; ; in, represents the coordinates of the reference point in the corrected image data, Indicates the saved The coordinates of the reference points, Indicates The drone’s positioning coordinates at the moment, represents the horizontal correction factor, Indicates the vertical correction factor; The coordinates of the reference point in the corrected image data are set as the corrected image data.

[0016] The present invention selects reference points in the image data of the area to be detected in real time, records the coordinates of the reference points in the image data, and corrects the collected real scene image data based on the saved reference point coordinates and the drone positioning coordinates, and obtains the corrected image data through a correction transformation formula, thereby improving the accuracy of drone image data collection.

[0017] Preferably, the extracting features from the corrected image data and the processed road image data, and preliminarily determining the road damage area by feature comparison comprises the following steps: S51, making one-to-one correspondence between the obtained corrected image data and the processed road image data, dividing the two corresponding sets of data into 16×16 image data blocks, and inputting the divided image data blocks into a convolutional neural network; S52, extracting features from the divided image data blocks through a convolutional neural network; The convolutional neural network is set to include: input layer, convolution layer, pooling layer and fully connected layer; S521, after receiving the divided image data blocks, the input layer transmits the received image data blocks to the convolution layer, and the convolution layer extracts local features in the image data blocks through convolution operations; The convolution operation formula is as follows:

[0018] in, represents the input image data block, represents the weight of the corresponding convolution kernel, b Indicates the bias value, f represents the output features; S522, after the convolution layer extracts the local features in the image data block, the local features in the extracted image data block are processed by the pooling layer, and the features are downsampled by the pooling layer to reduce the data dimension; S523, extracting features from the divided image data blocks through continuous convolution and pooling until the extracted features converge, the convolution stops, and the extracted features are summarized and input into the fully connected layer; S524, the fully connected layer integrates the extracted features and outputs the final feature extraction result; S53, comparing the two sets of extracted feature data, setting a comparison threshold of the feature data, and when the comparison result of the two sets of feature data is less than the set comparison threshold, it indicates that there is a road damage area in the corresponding image data.

[0019] The present invention makes one-to-one correspondence between the obtained corrected image data and the processed road image data, and inputs them into a convolutional neural network for feature extraction. After the feature extraction is completed, the road damage area is preliminarily determined by feature comparison, thereby improving the efficiency of real-time detection of road damage.

[0020] Preferably, the selecting of images collected in real time by drones in the preliminarily determined road damage area, analyzing the selected images by an image analysis method, and locating the road damage position in the analyzed images comprises the following steps: S61, analyzing the selected image by background separation method to extract the image target; Set the total number of pixels in the image to N and divide it into class A. For the center point of each class, Indicates that point j belongs to the i-th category, then the objective function and constraint conditions are as follows: ; ; Where m is the blur index, which is the blur degree in the captured image. represents the center point of the i-th class, represents the sample points collected, Represents point j to the center point of the i-th category The distance, J represents the objective function; Perform iterative clustering operations on similar pixels to separate the image target from the background; S62, calculating the distance between the image target and the nearest reference point, and locating the road damage position in the image based on the calculated distance; Assume that the longitude and latitude of two reference points are C(lat1,lng1) and D(lat2,lng2), the radius of the earth is R, and the distance between the two points is DL; ; Calculate the distance DL1 between two reference points in the image data, and calculate the distance represented by each pixel in the image data by the ratio of DL to DL1; Based on the distance represented by each pixel and the position of the reference point, the road damage position in the image is located.

[0021] The present invention selects images collected by drones in real time within a preliminarily determined road damage area, analyzes the selected images through an image analysis method, further determines the road damage area through background separation, and simultaneously determines the scale and location of the road damage area through a scale method to improve the real-time performance of road damage positioning.

[0022] This embodiment also discloses a real-time road damage detection system based on drone remote sensing, which is used to implement a real-time road damage detection method based on drone remote sensing. The system includes: a data acquisition module, a data processing module, a drone positioning module, an image correction module, a feature extraction module, and a road damage location positioning module; The data acquisition module is used to collect road image data of the area to be detected; The data processing module is used to process the collected road image data; The UAV positioning module is used to accurately position the UAV; The image correction module is used to correct the collected real scene image data according to the positioning of the UAV and the collected road image data; The feature extraction module is used to extract features from the corrected image data and preliminarily determine the road damage area; The road damage location positioning module is used to analyze and locate the preliminarily determined road damage area through image analysis.

[0023] (III) Beneficial effects Compared with the prior art, the present invention provides a method and system for real-time detection of road damage based on UAV remote sensing, which has the following beneficial effects: 1. The invention collects road image data of the area to be detected and processes the collected road image data. At the same time, based on the processed road image data, a coordinate system and reference points are set, and based on the set coordinate system and reference points, the drone is accurately positioned by real-time positioning; after the positioning is completed, the image data of the area to be detected is collected by the drone in real time, and the collected real scene image data is corrected, and features are extracted from the corrected image data and the processed road image data. At the same time, the road damage area is preliminarily determined by feature comparison, and the image collected by the drone in real time within the preliminarily determined road damage area is selected, the selected image is analyzed by an image analysis method, and the road damage position in the analyzed image is located, thereby improving the reliability of real-time detection of road damage.

[0024] 2. The invention processes the collected road image data by setting a window, and completes the smoothing of the collected road image data by calculating the weight of each pixel in the window, thereby improving the reliability of the road image data.

[0025] 3. The invention sets reference points according to environmental parameters in the area to be detected, establishes a three-dimensional coordinate system based on all reference points in the area to be detected, and records the coordinates of each reference point, thereby improving the accuracy of detection of the area to be detected.

[0026] 4. The invention selects GPS positioning satellites and receives UAV positioning information data. At the same time, it calculates the horizontal position information of the UAV based on the received GPS positioning information and the vertical height information of the UAV based on the received ultrasonic measurement information. The obtained horizontal position information and vertical height information of the UAV are represented in a set coordinate system, thereby realizing real-time positioning of the UAV.

[0027] 5. The invention selects reference points in the image data of the area to be detected that is collected in real time, records the coordinates of the reference points in the image data, and corrects the collected real-scene image data based on the saved reference point coordinates and the drone positioning coordinates. The corrected image data is obtained through a correction transformation formula, thereby improving the accuracy of drone image data collection.

[0028] 6. The invention makes one-to-one correspondence between the corrected image data and the processed road image data, and inputs them into a convolutional neural network for feature extraction. After the feature extraction is completed, the road damage area is preliminarily determined by feature comparison, thereby improving the efficiency of real-time detection of road damage.

[0029] 7. The invention selects images collected by drones in real time to preliminarily determine the road damage area, analyzes the selected images through image analysis methods, further determines the road damage area through background separation, and determines the scale and location of the road damage area through a scale method to improve the real-time positioning of the road damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the structure of the real-time road damage detection process of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Example 1 See also Figure 1 This embodiment discloses a method for real-time detection of road damage based on drone remote sensing, which specifically includes the following steps: S1, collecting road image data of the area to be detected, and processing the collected road image data to obtain processed road image data; S2. Setting a coordinate system and a reference point based on the processed road image data; S3, based on the set coordinate system and reference points, accurately locate the drone through real-time positioning; S4, collecting image data of the area to be detected in real time through the drone, and correcting the collected real scene image data to obtain corrected image data; S5, extracting features from the corrected image data and the processed road image data, and preliminarily determining the road damage area by feature comparison; S6, selecting images collected by drones in real time within the preliminarily determined road damage area, analyzing the selected images using an image analysis method, and locating the road damage position in the analyzed images; For further information, see Figure 1 , collecting road image data of the area to be detected, and processing the collected road image data to obtain the processed road image data includes the following steps: S11, setting a window, wherein the window includes the road image pixels to be processed in the collected road image data and the neighborhood pixels around the road image; S12, for each pixel in the window, calculating the weight of each pixel; The pixel weight calculation formula is as follows: ; ; in, represents the Gaussian distribution function, Represents the horizontal coordinate of the pixel point, Represents the vertical coordinate of the pixel point. represents the pixel weight, represents the natural logarithm, Represents the horizontal coordinate of the center pixel, Represents the vertical coordinate of the center pixel, Represents the horizontal coordinate of the central neighborhood pixel, Represents the ordinate of the central neighborhood pixel, represents the Gaussian function parameters; S13, calculating the weight of each pixel respectively; S14, multiplying the weight of each pixel by its corresponding pixel value, and summing the weighted values ​​of all pixels; S15, dividing the weighted sum by the total weight to obtain smoothed road image data; Further, the smoothed road image data is set as the processed road image data; For further information, see Figure 1 , based on the processed road image data, setting the coordinate system and reference points includes the following steps: S21, setting a reference point in the area to be detected; Based on the processed road image data collected in the area to be detected, the intersection of the main road and the non-main road is set as a reference point; Furthermore, when there is only one main road or only one non-main road in the area to be detected, the starting point is set as the first reference point, and a distance threshold between the reference points is set, and reference points are set in equal proportion according to the set distance threshold; S22, based on all the reference points in the area to be detected, number each reference point by setting a UID and save it; S23, establishing a three-dimensional coordinate system with the first reference point as a reference, setting the first reference point as the coordinate origin, recording and saving the coordinate positions of the reference points corresponding to the numbers; For further information, see Figure 1 Based on the set coordinate system and reference points, the precise positioning of the drone through real-time positioning includes the following steps: S31, select the GPS positioning satellite and receive the UAV positioning information data; The drone positioning information data includes: GPS positioning information and ultrasonic measurement information; Send GPS positioning information through the drone signal transmitter and wait for feedback from the GPS positioning satellite; Set the first set of received GPS positioning signals as the GPS positioning information of the current drone; S32, calculating the horizontal position information of the drone based on the received GPS positioning information; Assuming the propagation speed of GPS positioning information is the speed of light, the formula for calculating the horizontal position information of the drone through the time difference between transmitting GPS positioning information and receiving GPS positioning information is as follows: ; in, Indicates the horizontal position information of the drone. represents the speed of light, Indicates the time of receiving GPS positioning information. Indicates the time when GPS positioning information is transmitted; S33, calculating the vertical height information of the UAV based on the received ultrasonic measurement information; The formula for calculating the vertical height information of the drone based on the time difference between the drone transmitting the ultrasonic measurement information and receiving the ultrasonic measurement information bounced off the ground is as follows: ; in, Indicates the vertical height information of the drone. represents the speed of sound, Indicates the time of receiving ultrasonic measurement information. Indicates the time of transmitting ultrasonic measurement information; S34, expressing the obtained UAV horizontal position information and UAV vertical height information in a set coordinate system; For further information, see Figure 1 , using a drone to collect image data of the area to be detected in real time, and correcting the collected real scene image data to obtain the corrected image data, including the following steps: S41, selecting a reference point in the image data of the area to be detected collected in real time, and recording the coordinates of the reference point in the image data; S42, correcting the collected real scene image data based on the saved reference point coordinates and the drone positioning coordinates; The correction transformation formula is as follows: ; ; in, represents the coordinates of the reference point in the corrected image data, Indicates the saved The coordinates of the reference points, Indicates The drone’s positioning coordinates at the moment, represents the horizontal correction factor, Indicates the vertical correction factor; Setting the coordinates of the reference point in the corrected image data as the corrected image data; For further information, see Figure 1 , extracting features from the corrected image data and processed road image data, and preliminarily determining the road damage area by feature comparison, including the following steps: S51, making one-to-one correspondence between the obtained corrected image data and the processed road image data, dividing the two corresponding sets of data into 16×16 image data blocks, and inputting the divided image data blocks into a convolutional neural network; S52, extracting features from the divided image data blocks through a convolutional neural network; The convolutional neural network is set to include: input layer, convolution layer, pooling layer and fully connected layer; S521, after receiving the divided image data blocks, the input layer transmits the received image data blocks to the convolution layer, and the convolution layer extracts local features in the image data blocks through convolution operations; The convolution operation formula is as follows:

[0033] in, represents the input image data block, represents the weight of the corresponding convolution kernel, b Indicates the bias value, f represents the output features; S522, after the convolution layer extracts the local features in the image data block, the local features in the extracted image data block are processed by the pooling layer, and the features are downsampled by the pooling layer to reduce the data dimension; S523, extracting features from the divided image data blocks through continuous convolution and pooling until the extracted features converge, the convolution stops, and the extracted features are summarized and input into the fully connected layer; S524, the fully connected layer integrates the extracted features and outputs the final feature extraction result; S53, comparing the two sets of extracted feature data, setting a comparison threshold of the feature data, and when the comparison result of the two sets of feature data is less than the set comparison threshold, it indicates that the corresponding image data has a road damage area; For further information, see Figure 1 , selecting images collected by drones in real time in the road damage area that is preliminarily determined, analyzing the selected images by image analysis methods, and locating the road damage position in the analyzed images includes the following steps: S61, analyzing the selected image by background separation method to extract the image target; Set the total number of pixels in the image to N and divide it into class A. For the center point of each class, Indicates that point j belongs to the i-th category, then the objective function and constraint conditions are as follows: ; ; Where m is the blur index, which is the blur degree in the captured image. represents the center point of the i-th class, represents the sample points collected, Represents point j to the center point of the i-th category The distance, J represents the objective function; Perform iterative clustering operations on similar pixels to separate the image target from the background; S62, calculating the distance between the image target and the nearest reference point, and locating the road damage position in the image based on the calculated distance; Assume that the longitude and latitude of two reference points are C(lat1,lng1) and D(lat2,lng2), the radius of the earth is R, and the distance between the two points is DL; ; Further, the distance DL1 between the two reference points in the image data is calculated, and the distance represented by each pixel in the image data is calculated by the ratio of DL to DL1; Based on the distance represented by each pixel and the position of the reference point, the road damage position in the image is located; Example 2 See also Figure 1 , this embodiment also discloses a real-time road damage detection system based on drone remote sensing, which is used to implement a real-time road damage detection method based on drone remote sensing. The system includes: a data acquisition module, a data processing module, a drone positioning module, an image correction module, a feature extraction module, and a road damage location positioning module; The data acquisition module is used to collect road image data of the area to be detected; The data processing module is used to process the collected road image data; The UAV positioning module is used to accurately position the UAV; The image correction module is used to correct the collected real scene image data according to the positioning of the UAV and the collected road image data; The feature extraction module is used to extract features from the corrected image data and preliminarily determine the road damage area; The road damage location positioning module is used to analyze and locate the preliminarily determined road damage area through image analysis.

[0034] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time road damage detection method based on UAV remote sensing, characterized in that: The following steps are involved: S1, collecting road image data of the area to be detected, and processing the collected road image data to obtain processed road image data; S2. Setting a coordinate system and a reference point based on the processed road image data; S3, based on the set coordinate system and reference points, accurately locate the drone through real-time positioning; S4, collecting image data of the area to be detected in real time through the drone, and correcting the collected real scene image data to obtain corrected image data; S5, extracting features from the corrected image data and the processed road image data, and preliminarily determining the road damage area by feature comparison; S6. Select images collected by drones in real time within the preliminarily determined road damage area, analyze the selected images using an image analysis method, and locate the road damage position in the analyzed images.

2. The method for real-time detection of road damage based on drone remote sensing according to claim 1 is characterized in that The collecting of the road image data of the area to be detected and processing the collected road image data to obtain the processed road image data comprises the following steps: S11, setting a window, wherein the window includes the road image pixels to be processed in the collected road image data and the neighborhood pixels around the road image; S12, for each pixel in the window, calculating the weight of each pixel; S13, calculating the weight of each pixel respectively; S14, multiplying the weight of each pixel by its corresponding pixel value, and summing the weighted values ​​of all pixels; S15, dividing the weighted sum by the total weight to obtain smoothed road image data; The smoothed road image data is set as the processed road image data.

3. The method for real-time detection of road damage based on drone remote sensing according to claim 2 is characterized in that ,For each pixel in the window, calculating the weight of each pixel includes the following steps: The pixel weight calculation formula is as follows: ； ； in, represents the Gaussian distribution function, Represents the horizontal coordinate of the pixel point, Represents the vertical coordinate of the pixel point. represents the pixel weight, represents the natural logarithm, Represents the horizontal coordinate of the center pixel, Represents the ordinate of the center pixel. Represents the horizontal coordinate of the central neighborhood pixel, Represents the ordinate of the central neighborhood pixel, Represents the Gaussian function parameters.

4. The method for real-time detection of road damage based on drone remote sensing according to claim 1 is characterized in that , the setting of the coordinate system and the reference point based on the processed road image data includes the following steps: S21, setting a reference point in the area to be detected; Based on the processed road image data collected in the area to be detected, the intersection of the main road and the non-main road is set as a reference point; S22, based on all the reference points in the area to be detected, number each reference point by setting a UID and save it; S23. Establish a three-dimensional coordinate system based on the first reference point, set the first reference point as the coordinate origin, record the coordinate position of each reference point corresponding to the number, and save it.

5. The method for real-time detection of road damage based on drone remote sensing according to claim 1 is characterized in that ,Based on the set coordinate system and reference points, the accurate positioning of the UAV through real-time positioning includes the following steps: S31, select the GPS positioning satellite and receive the UAV positioning information data; S32, calculating the horizontal position information of the drone based on the received GPS positioning information; Assuming the propagation speed of GPS positioning information is the speed of light, the formula for calculating the horizontal position information of the drone through the time difference between transmitting GPS positioning information and receiving GPS positioning information is as follows: ； in, Indicates the horizontal position information of the drone. represents the speed of light, Indicates the time of receiving GPS positioning information. Indicates the time when GPS positioning information is transmitted; S33, calculating the vertical height information of the UAV based on the received ultrasonic measurement information; The formula for calculating the vertical height information of the drone based on the time difference between the drone transmitting the ultrasonic measurement information and receiving the ultrasonic measurement information bounced off the ground is as follows: ； in, Indicates the vertical height information of the drone. represents the speed of sound, Indicates the time of receiving ultrasonic measurement information. Indicates the time of transmitting ultrasonic measurement information; S34, the obtained UAV horizontal position information and UAV vertical height information are expressed in a set coordinate system.

6. The method for real-time detection of road damage based on drone remote sensing according to claim 1 is characterized in that The method of collecting image data of the area to be detected by the drone in real time and correcting the collected real scene image data to obtain the corrected image data includes the following steps: S41, selecting a reference point in the image data of the area to be detected collected in real time, and recording the coordinates of the reference point in the image data; S42: Correct the collected real scene image data based on the saved reference point coordinates and the drone positioning coordinates.

7. A method for real-time detection of road damage based on drone remote sensing according to claim 6, characterized in that ,The correction of the collected real scene image data based on the saved reference point coordinates and the drone positioning coordinates includes the following steps: The correction transformation formula is as follows: ； ； in, represents the coordinates of the reference point in the corrected image data, Indicates the saved The coordinates of the reference points, Indicates The drone’s positioning coordinates at the moment, represents the horizontal correction factor, Indicates the vertical correction factor; The coordinates of the reference point in the corrected image data are set as the corrected image data.

8. The method for real-time detection of road damage based on UAV remote sensing according to claim 1 is characterized in that The method of extracting features from the corrected image data and processed road image data and preliminarily determining the road damage area by feature comparison includes the following steps: S51, making one-to-one correspondence between the obtained corrected image data and the processed road image data, dividing the two corresponding sets of data into 16×16 image data blocks, and inputting the divided image data blocks into a convolutional neural network; S52, extracting features from the divided image data blocks through a convolutional neural network; The convolutional neural network is set to include: input layer, convolution layer, pooling layer and fully connected layer; S521, after receiving the divided image data blocks, the input layer transmits the received image data blocks to the convolution layer, and the convolution layer extracts local features in the image data blocks through convolution operations; S522, after the convolution layer extracts the local features in the image data block, the local features in the extracted image data block are processed by the pooling layer, and the features are downsampled by the pooling layer to reduce the data dimension; S523, extracting features from the divided image data blocks through continuous convolution and pooling until the extracted features converge, the convolution stops, and the extracted features are summarized and input into the fully connected layer; S524, the fully connected layer integrates the extracted features and outputs the final feature extraction result; S53, comparing the two sets of extracted feature data, setting a comparison threshold of the feature data, and when the comparison result of the two sets of feature data is less than the set comparison threshold, it indicates that there is a road damage area in the corresponding image data.

9. The method for real-time detection of road damage based on drone remote sensing according to claim 1 is characterized in that ,The ,selecting images collected in real time by UAVs in the road disaster area preliminarily ,analyzing the selected images by image analysis methods, and locating the ,road disaster position in the analyzed images includes the following steps: S61, analyzing the selected image by background separation method to extract the image target; Set the total number of pixels in the image to N and divide it into class A. For each class’s center point, Indicates that point j belongs to the i-th category, then the objective function and constraint conditions are as follows: ； ； Where m is the blur index, which is the blur degree in the captured image. represents the center point of the i-th class, represents the sample points collected, Represents point j to the center point of the i-th category The distance, J represents the objective function; Perform iterative clustering operations on similar pixels to separate the image target from the background; S62, calculating the distance between the image target and the nearest reference point, and locating the road damage position in the image based on the calculated distance; Assume that the longitude and latitude of two reference points are C(lat1,lng1) and D(lat2,lng2), the radius of the earth is R, and the distance between the two points is DL; ； Calculate the distance DL1 between two reference points in the image data, and calculate the distance represented by each pixel in the image data by the ratio of DL to DL1; Based on the distance represented by each pixel and the position of the reference point, the road damage position in the image is located.

10. A system for implementing the method for real-time detection of road damage based on UAV remote sensing as described in any one of claims 1 to 9, characterized in that: It includes data acquisition module, data processing module, drone positioning module, image correction module, feature extraction module and road damage location positioning module: The data acquisition module is used to collect road image data of the area to be detected; The data processing module is used to process the collected road image data; The UAV positioning module is used to accurately position the UAV; The image correction module is used to correct the collected real scene image data according to the positioning of the UAV and the collected road image data; The feature extraction module is used to extract features from the corrected image data and preliminarily determine the road damage area; The road damage location positioning module is used to analyze and locate the preliminarily determined road damage area through image analysis.

Images