Complex pavement damage condition detection method and device fused with unmanned aerial vehicle

Abstract

The invention discloses a complex pavement damage condition detection method integrated with an unmanned aerial vehicle. The method specifically comprises the following steps: S1, an unmanned aerial vehicle platform and a sensor system; s2, image preprocessing and feature extraction; the invention relates to the technical field of intelligent traffic. According to the complex road surface damage condition detection method and device fused with the unmanned aerial vehicle, the unmanned aerial vehicle is high in inspection speed and wide in coverage range, the detection period is greatly shortened, the detection cost is reduced, and artificial intelligence technologies such as deep learning and machine learning are combined, so that accurate recognition and classification of road surface damage conditions are realized, and the detection precision is improved; a comprehensive detection report and three-dimensional visual analysis are provided, a scientific basis is provided for road maintenance decision, traffic safety management and urban planning, road maintenance is promoted to develop towards the intelligent and precise direction, the road maintenance efficiency and quality are improved, traffic safety is guaranteed, an interactive decision support system is developed, and the development of the road maintenance system is facilitated. And the user operation convenience and the decision-making efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a method and device for detecting complex road damage conditions by integrating unmanned aerial vehicles. Background Art

[0002] Unmanned aerial vehicle, abbreviated as "UAV", is an unmanned aircraft controlled by radio remote control equipment and self-contained program control device. UAV is actually a general term for unmanned aerial vehicles. From a technical point of view, it can be divided into several categories: unmanned helicopters, unmanned fixed-wing aircraft, unmanned multi-rotor aircraft, unmanned airships, and unmanned paragliders. UAVs mainly have five key technologies, namely body structure design technology, body material technology, flight control technology, wireless communication remote control technology, and wireless image return technology. These five technologies support the development and improvement of modern intelligent UAVs.

[0003] In recent years, with the rapid development of drone technology, its application in various fields has become more and more extensive. In the field of road detection, traditional automobile assisted driving technology mainly perceives road damage through various sensors installed on the car, but this method has problems such as insufficient accuracy and inability to provide complete road damage information. The introduction of drone detection technology can make up for the disadvantages of sensors by identifying road damage through image processing technology. However, the existing drone detection technology only uses image processing to obtain road damage results and cannot obtain accurate road damage information. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method and device for detecting complex road damage conditions by integrating unmanned aerial vehicles, which solves the problems mentioned in the above background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a complex road damage detection method integrating drones, specifically comprising the following steps: S1. UAV platform and sensor system: Design and manufacture a UAV platform with high-precision positioning, autonomous obstacle avoidance, and stable flight capabilities, equipped with a high-resolution multispectral camera, infrared thermal imager, and 3D laser scanner; S2. Image preprocessing and feature extraction: (1) Image preprocessing: The autoencoder network in deep learning is used to denoise and enhance the multispectral image, and histogram equalization and color space conversion techniques are used to improve the image contrast; (2) Feature extraction: Use the ResNet or EfficientNet architecture in the convolutional neural network to extract features from the preprocessed image, and combine the temperature gradient characteristics in the infrared thermal imaging data to construct a road damage feature vector; S3. Damage classification and quantitative assessment: Classification algorithm: Adopt ensemble learning method and support vector machine for multi-classification to realize accurate identification of road damage types; Quantitative evaluation: Use semantic segmentation technology in deep learning to perform pixel-level segmentation on the damaged area after classification, calculate parameters such as damaged area, perimeter, depth, etc. At the same time, combine with the road damage index calculation model to quantitatively evaluate the degree of damage; S4, 3D modeling and visualization analysis: (1) 3D modeling: Use the point cloud registration algorithm to stitch together the 3D point cloud data of multiple scanning points to construct a high-precision 3D model of the road surface; (2) Visualization analysis: In the 3D model, the damaged area is highlighted, annotated, and measured. At the same time, multispectral images and infrared thermal imaging data are combined to perform color rendering and temperature distribution analysis on the damaged area to improve the visualization effect. S5. Test report generation and decision support: Based on damage classification, quantitative assessment and 3D modeling results, automatically generate a test report containing information such as damage location, type, size, PCI value, 3D visualization image, etc.; associate the test report with the road maintenance database, and combine historical data, traffic flow, climate conditions and other factors to provide a scientific basis and decision support for the formulation of road maintenance plans, priority sorting, resource allocation and traffic safety management; develop an interactive decision support system that allows users to view test reports, adjust maintenance plans, optimize resource allocation, etc. through a visual interface.

[0006] Preferably, in S1, the UAV platform adopts a high-performance electric multi-rotor design and is equipped with a GPS / INS dual-mode navigation system to ensure high-precision positioning and autonomous flight capabilities. The UAV has autonomous take-off, landing, route planning, obstacle avoidance and emergency braking functions to adapt to complex and changeable flight environments.

[0007] Preferably, in S1, the high-resolution multispectral camera adopts a CMOS sensor with a resolution of more than 40 million pixels, supports multiple spectral channels such as RGB, NIR, SWIR, etc., and is used to capture characteristic information such as road material, color, texture, etc. The infrared thermal imager adopts uncooled micro-bolometer technology with a resolution of 640x480 pixels and a temperature measurement range of -20°C to +650°C, which is used to detect abnormal road surface temperature and identify potential damaged areas. The three-dimensional laser scanner adopts a phase or pulse ranging principle, with a scanning rate of ≥1 million points / second and a ranging accuracy of ≤2mm, and is used to obtain high-precision three-dimensional point cloud data of the road surface.

[0008] Preferably, in S1, the high-resolution multispectral camera adopts a CMOS sensor with a resolution of more than 40 million pixels, supports multiple spectral channels such as RGB, NIR, SWIR, etc., and is used to capture characteristic information such as road material, color, texture, etc. The infrared thermal imager adopts uncooled micro-bolometer technology with a resolution of 640x480 pixels and a temperature measurement range of -20°C to +650°C, which is used to detect abnormal road surface temperature and identify potential damaged areas. The three-dimensional laser scanner adopts a phase or pulse ranging principle, with a scanning rate of ≥1 million points / second and a ranging accuracy of ≤2mm, and is used to obtain high-precision three-dimensional point cloud data of the road surface.

[0009] Preferably, in S3 (1), the ensemble learning method adopts one of a random forest and a gradient boosting tree, and the road surface damage types include cracks, potholes, rutting and spalling.

[0010] Preferably, in S4 (1), the point cloud data is converted into a three-dimensional mesh model using a Poisson surface reconstruction or a Marching Cubes algorithm to achieve three-dimensional visualization of the road surface.

[0011] The present invention also discloses a complex road damage detection device integrated with a drone, comprising a memory and a processor, wherein: A memory for storing computer programs that can be run on the processor; The processor is used to execute the steps of the above-mentioned complex road damage detection method integrating drone when running the computer program.

[0012] Beneficial Effects The present invention provides a complex road surface damage detection method and device integrated with drones. Compared with the prior art, the method and device have the following beneficial effects: the complex road surface damage detection method and device integrated with drones specifically include the following steps: S1, drone platform and sensor system; S2, image preprocessing and feature extraction; S3, damage classification and quantitative evaluation; S4, three-dimensional modeling and visual analysis; S5, detection report generation and decision support; drone inspection has fast speed and wide coverage, greatly shortens the detection cycle, reduces the detection cost, combines deep learning, machine learning and other artificial intelligence technologies, realizes accurate identification and classification of road surface damage, improves detection accuracy, provides comprehensive detection reports and three-dimensional visual analysis, provides scientific basis for road maintenance decision-making, traffic safety management and urban planning, promotes road maintenance to develop in the direction of intelligence and precision, improves road maintenance efficiency and quality, ensures traffic safety, develops interactive decision support systems, and improves user operation convenience and decision-making efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] See also Figure 1 The present invention provides a technical solution: a method for detecting complex road damage conditions by integrating an unmanned aerial vehicle, which specifically includes the following steps: S1. UAV platform and sensor system: Design and manufacture a UAV platform with high-precision positioning, autonomous obstacle avoidance, and stable flight capabilities, equipped with a high-resolution multispectral camera, infrared thermal imager, and 3D laser scanner; S2. Image preprocessing and feature extraction: (1) Image preprocessing: The autoencoder network in deep learning is used to denoise and enhance the multispectral image, and histogram equalization and color space conversion techniques are used to improve the image contrast; (2) Feature extraction: Use the ResNet or EfficientNet architecture in the convolutional neural network to extract features from the preprocessed image, and combine the temperature gradient characteristics in the infrared thermal imaging data to construct a road damage feature vector; S3. Damage classification and quantitative assessment: Classification algorithm: Adopt ensemble learning method and support vector machine for multi-classification to realize accurate identification of road damage types; Quantitative evaluation: Use semantic segmentation technology in deep learning to perform pixel-level segmentation on the damaged area after classification, calculate parameters such as damaged area, perimeter, depth, etc. At the same time, combine with the road damage index calculation model to quantitatively evaluate the degree of damage; S4, 3D modeling and visualization analysis: (1) 3D modeling: Use the point cloud registration algorithm to stitch together the 3D point cloud data of multiple scanning points to construct a high-precision 3D model of the road surface; (2) Visualization analysis: In the 3D model, the damaged area is highlighted, annotated, and measured. At the same time, multispectral images and infrared thermal imaging data are combined to perform color rendering and temperature distribution analysis on the damaged area to improve the visualization effect. S5. Test report generation and decision support: Automatically generate a test report containing information such as damage location, type, size, PCI value, and 3D visualization image based on damage classification, quantitative assessment, and 3D modeling results; associate the test report with the road maintenance database, and combine historical data, traffic flow, climate conditions, and other factors to provide a scientific basis and decision support for the formulation of road maintenance plans, priority sorting, resource allocation, and traffic safety management; develop an interactive decision support system that allows users to view test reports, adjust maintenance plans, and optimize resource allocation through a visual interface.

[0016] In the present invention, in S1, the UAV platform adopts a high-performance electric multi-rotor design and is equipped with a GPS / INS dual-mode navigation system to ensure high-precision positioning and autonomous flight capabilities. The UAV has autonomous take-off, landing, route planning, obstacle avoidance and emergency braking functions to adapt to complex and changeable flight environments.

[0017] In the present invention, in S1, the high-resolution multispectral camera adopts a CMOS sensor with a resolution of more than 40 million pixels, supports multiple spectral channels such as RGB, NIR, SWIR, etc., and is used to capture characteristic information such as road material, color, texture, etc. The infrared thermal imager adopts uncooled micro-bolometer technology with a resolution of 640x480 pixels and a temperature measurement range of -20°C to +650°C, which is used to detect abnormal road surface temperature and identify potential damaged areas. The three-dimensional laser scanner adopts a phase or pulse ranging principle, with a scanning rate of ≥1 million points / second and a ranging accuracy of ≤2mm, which is used to obtain high-precision three-dimensional point cloud data of the road surface.

[0018] In the present invention, in S1, the high-resolution multispectral camera adopts a CMOS sensor with a resolution of more than 40 million pixels, supports multiple spectral channels such as RGB, NIR, SWIR, etc., and is used to capture characteristic information such as road material, color, texture, etc. The infrared thermal imager adopts uncooled micro-bolometer technology with a resolution of 640x480 pixels and a temperature measurement range of -20°C to +650°C, which is used to detect abnormal road surface temperature and identify potential damaged areas. The three-dimensional laser scanner adopts a phase or pulse ranging principle, with a scanning rate of ≥1 million points / second and a ranging accuracy of ≤2mm, which is used to obtain high-precision three-dimensional point cloud data of the road surface.

[0019] In the present invention, in S3 (1), the ensemble learning method adopts one of a random forest or a gradient boosting tree, and the road surface damage types include cracks, potholes, rutting and spalling.

[0020] In the present invention, in S4 (1), the point cloud data is converted into a three-dimensional mesh model using a Poisson surface reconstruction or a Marching Cubes algorithm to achieve three-dimensional visualization of the road surface.

[0021] The present invention also discloses a complex road damage detection device integrated with a drone, comprising a memory and a processor, wherein: A memory for storing computer programs that can be run on the processor; The processor is used to execute the steps of the above-mentioned complex road damage detection method integrating drone when running the computer program.

[0022] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0023] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0024] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting complex road damage using drones, characterized in that: The specific steps include: S1. UAV platform and sensor system: Design and manufacture a UAV platform with high-precision positioning, autonomous obstacle avoidance, and stable flight capabilities, equipped with a high-resolution multispectral camera, infrared thermal imager, and 3D laser scanner; S2. Image preprocessing and feature extraction: (1) Image preprocessing: The autoencoder network in deep learning is used to denoise and enhance the multispectral image, and histogram equalization and color space conversion techniques are used to improve the image contrast; (2) Feature extraction: Use the ResNet or EfficientNet architecture in the convolutional neural network to extract features from the preprocessed image, and combine the temperature gradient characteristics in the infrared thermal imaging data to construct a road damage feature vector; S3. Damage classification and quantitative assessment: S4. 3D modeling and visualization analysis: (1) 3D modeling: Use the point cloud registration algorithm to stitch together the 3D point cloud data of multiple scanning points to construct a high-precision 3D model of the road surface; (2) Visualization analysis: In the 3D model, the damaged area is highlighted, annotated, and measured. At the same time, multispectral images and infrared thermal imaging data are combined to perform color rendering and temperature distribution analysis on the damaged area to improve the visualization effect. S5. Test report generation and decision support: Based on damage classification, quantitative assessment and 3D modeling results, a test report containing information such as damage location, type, size, PCI value, 3D visualization image, etc. is automatically generated; the test report is associated with the road maintenance database, combined with historical data, traffic flow, climate conditions and other factors, to provide scientific basis and decision support for the formulation of road maintenance plans, priority sorting, resource allocation and traffic safety management; Develop an interactive decision support system that allows users to view inspection reports, adjust maintenance plans, optimize resource allocation, etc. through a visual interface.

2. The complex road damage detection method integrating drone according to claim 1 is characterized by: In the S1, the drone platform adopts a high-performance electric multi-rotor design and is equipped with a GPS / INS dual-mode navigation system to ensure high-precision positioning and autonomous flight capabilities. The drone has autonomous take-off, landing, route planning, obstacle avoidance and emergency braking functions to adapt to complex and changeable flight environments.

3. The complex road damage detection method integrating drone according to claim 1 is characterized by: In the S1, the high-resolution multispectral camera adopts a CMOS sensor with a resolution of more than 40 million pixels, supports multiple spectral channels such as RGB, NIR, SWIR, etc., and is used to capture characteristic information such as road material, color, texture, etc. The infrared thermal imager adopts uncooled micro-bolometer technology with a resolution of 640x480 pixels and a temperature measurement range of -20°C to +650°C, which is used to detect abnormal road surface temperature and identify potential damaged areas. The three-dimensional laser scanner adopts a phase or pulse ranging principle, with a scanning rate of ≥1 million points / second and a ranging accuracy of ≤2mm, and is used to obtain high-precision three-dimensional point cloud data of the road surface.

4. The complex road damage detection method integrating drones according to claim 1 is characterized by: In S2 (2), features of the image are extracted, including edge features, texture features and color features.

5. The complex road damage detection method integrating drone according to claim 1 is characterized by: In S3 (1), the ensemble learning method adopts one of random forest and gradient boosting tree, and the road surface damage types include cracks, potholes, rutting and spalling.

6. The complex road damage detection method integrating drones according to claim 1 is characterized by: In S4 (1), the point cloud data is converted into a three-dimensional mesh model using a Poisson surface reconstruction or a Marching Cubes algorithm to achieve three-dimensional visualization of the road surface.

7. A complex road damage detection device integrated with an unmanned aerial vehicle, characterized in that: comprising a memory and a processor, wherein: A memory for storing computer programs that can be run on the processor; A processor is used to execute the steps of the complex road damage detection method integrating drone as described in any one of claims 1 to 6 when running the computer program.

Images