Road crack intelligent identification and accurate repair method and system based on air-ground cooperation

Abstract

The invention provides a road crack intelligent identification and precise repair method and system based on air-ground cooperation. A target area is subjected to large-range inspection through an unmanned aerial vehicle; detecting and identifying a to-be-analyzed area in which cracks possibly exist in real time through the unmanned aerial vehicle, and judging whether the position of the to-be-analyzed area is in a low position or not; if so, triggering the crawler to execute path planning and automatically navigate to the to-be-analyzed area so as to obtain a high-resolution image of the to-be-analyzed area, and judging whether a crack exists or not; if the crack exists, finely segmenting the crack image, evaluating and analyzing the shape, length and width of the crack through a built-in algorithm of a tracked vehicle, and judging whether the crack needs to be repaired or not; and if the crack needs to be repaired, the crawler is triggered to temporarily repair the crack. Through air-ground separation type collaborative design, the unmanned aerial vehicle and the tracked vehicle perform own functions and complement each other, cross-space whole-process operation is achieved, meanwhile, an on-site automatic repairing mechanism is introduced, leaking stoppage is executed immediately after crack evaluation, and a closed-loop process from detection to temporary repairing is formed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road crack detection, in particular to a road crack intelligent identification and accurate repair method and system based on land-air cooperation. BACKGROUND

[0002] At present, the mainstream scheme of road crack detection is: (1) manual detection, which requires experienced engineers to visually inspect and approach the crack to detect and calculate the crack width by using a crack width card, and for the position of the bridge bottom, special equipment such as a bridge detection vehicle is needed to assist; (2) rotary-wing unmanned aerial vehicle detection, which can quickly patrol high-altitude targets such as bridges and roads at a long distance; (3) tracked vehicle detection, which detects the road at a close distance, and for the bridge, it performs wall adhesion operation through a negative pressure suction mechanism.

[0003] In the above detection methods, the manual detection method and equipment cannot keep up with the new trend of the development of infrastructure structure detection towards high altitude, refinement and intelligence, and the detection process highly depends on the experience and subjective judgment of engineers, and long-time operation is easy to cause fluctuations and errors in the measurement results; the rotary-wing unmanned aerial vehicle has large coverage capacity, but due to the limitation of load and endurance, it is difficult to approach the surface of complex structures for millimeter-level fine measurement, and it also cannot perform subsequent maintenance tasks; the tracked vehicle detection has slow operation speed, high energy consumption, and poor adaptability to irregular, rough or structure corner areas, and the overall applicable range is limited.

[0004] Patent CN120148238A discloses a highway cooperative monitoring method and system based on unmanned aerial vehicle and tracked vehicle, which discloses the cooperative operation mechanism of unmanned aerial vehicle and tracked vehicle, but mainly focuses on timed inspection and fault inspection, and cannot realize the complete operation closed loop of crack detection, fine analysis and temporary repair. SUMMARY

[0005] Therefore, the present application aims to provide a road crack intelligent identification and accurate repair method and system based on land-air cooperation to realize the complete operation closed loop of crack detection, fine analysis and temporary repair.

[0006] In a first aspect, the present application provides a road crack intelligent identification and accurate repair method based on land-air cooperation, which comprises the following steps: controlling the unmanned aerial vehicle to autonomously fly over the target area, constructing an environment map based on a laser radar and a depth camera, and performing large-scale inspection on the target area during flight; detecting and identifying the area to be analyzed which may have cracks in real time through an aerial visual recognition algorithm deployed in the unmanned aerial vehicle, and judging whether the area to be analyzed is located at a low position, the low position being a position that can be reached and detected by the tracked vehicle; If in low position, the environment map and the position of the area to be analyzed are transmitted to the tracked vehicle and the crack health analysis platform through wireless communication; According to the environment map and the position of the area to be analyzed, the tracked vehicle is triggered to perform path planning and automatically navigate to the area to be analyzed to obtain high-resolution images of the area to be analyzed and determine whether there is a crack; If there is a crack, the crack image is finely segmented, and the shape, length and width of the crack are evaluated and analyzed by the algorithm built-in the tracked vehicle, and it is determined whether it needs to be repaired; If it needs to be repaired, the tracked vehicle is triggered to temporarily repair the crack, and the repair information is transmitted to the crack health analysis platform.

[0007] Preferably, the road crack intelligent identification and accurate repair method based on land-air cooperation further comprises: If the position of the area to be analyzed is not in low position, the high-resolution image of the area to be analyzed is obtained by the unmanned aerial vehicle, and it is determined whether there is a crack; If there is a crack, the crack image is finely segmented, and the shape, length and width of the crack are evaluated and analyzed by the algorithm built-in the unmanned aerial vehicle, and it is determined whether it needs to be repaired; If it needs to be repaired, the environment map and the position of the area to be analyzed are transmitted to the tracked vehicle and the crack health analysis platform through wireless communication, the tracked vehicle is triggered to perform path planning and automatically navigate to the area to be analyzed, the crack is temporarily repaired, and the repair information is transmitted to the crack health analysis platform.

[0008] Preferably, before the crack is temporarily repaired, the road crack intelligent identification and accurate repair method based on land-air cooperation further comprises: The volume of the crack is detected by laser scanning to determine the amount of glue needed for the crack.

[0009] Preferably, the road crack intelligent identification and accurate repair method based on land-air cooperation further comprises: After the tracked vehicle completes the crack repair task, the repair information is transmitted to the crack health analysis platform through wireless means; The repair information is processed and visually displayed by the crack health analysis platform, and a reference suggestion is given in combination with the evaluation model to assist the user in the later maintenance of the crack.

[0010] In a second aspect, the present application provides a road crack intelligent identification and accurate repair system based on land-air cooperation, comprising an unmanned aerial vehicle, a tracked vehicle, and a crack health analysis platform for controlling the unmanned aerial vehicle and the tracked vehicle through wireless communication: The top of the tracked vehicle is provided with a landing pad, and the landing pad is provided with a two-dimensional code mark for identification and visual positioning of the unmanned aerial vehicle; The top of the tracked vehicle is provided with a glue supplementing assembly, the front end is provided with a rotatable depth camera, and the two side walls are provided with a searchlight and a laser radar.

[0011] Preferably, the glue supplementing assembly comprises a mechanical arm mounted on the tracked vehicle, an air pressure pump, a liquid storage tank for storing glue, and an injection gun and a camera mounted on the execution end of the mechanical arm. The camera is used for close-range high-precision image acquisition and morphological analysis of the crack, and the mounting axes of the camera and the injection gun are arranged in parallel. The injection gun, the air pressure pump and the liquid storage tank are connected through a hose.

[0012] Preferably, the execution end of the mechanical arm is provided with a laser scanning piece, and after the laser scanning piece detects the volume of the crack, the operation time of the air pressure pump is triggered to determine the amount of glue required for the crack.

[0013] Preferably, the repeatability of the mechanical arm is ±0.1mm, and the maximum working radius is 850mm.

[0014] Compared with the prior art, the beneficial effects of the present application are: Through land-air separation type cooperative design, the unmanned aerial vehicle and the tracked vehicle perform their respective functions and complement each other, realizing cross-space full-process operation. This cooperative mode breaks through the limitations of traditional system coupling rigidity and function fragmentation, and realizes efficient and flexible task cooperation. An automatic on-site repair mechanism is introduced, the tracked vehicle is equipped with a pneumatic injection gun, and after crack evaluation, the injection gun is immediately executed to plug the crack, forming a closed-loop process from detection to temporary repair, solving the problem that the existing system can only detect but cannot handle cracks on site. An integrated crack health analysis platform is integrated, which visualizes the image, detection parameters and maintenance suggestions, unifies detection, evaluation and decision-making in one platform, and improves the information closed-loop capability, user interaction experience and engineering application value of the system. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 It is a process schematic diagram of the road crack intelligent identification and accurate repair method based on land-air cooperation of the present application; Figure 2 It is a structure schematic diagram of the road crack intelligent identification and accurate repair system based on land-air cooperation of the present application; Figure 3 It is Figure 2 It is a structure schematic diagram of the glue supplementing assembly; Figure 4 It is a comparison diagram of crack detection results generated by the tracked vehicle under different models.

[0016] Main element symbol explanation: 11 - unmanned aerial vehicle; 12 - tracked vehicle; 121 - parking apron; 122 - depth camera; 123 - searchlight; 124 - laser radar; 13 - glue replenishment assembly; 131 - mechanical arm; 132 - air pressure pump; 133 - liquid storage tank; 134 - injection gun; 135 - camera; 136 - hose.

[0017] The following detailed description will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0018] The following description is provided to enable any person skilled in the art to practice the present application. The preferred embodiments described in the following description are only examples of the present application and other obvious variants can be conceived by those skilled in the art.

[0019] Please refer to Figure 1 In an embodiment of the present application, a road crack intelligent identification and accurate repair method based on land-air cooperation is provided, comprising the following steps: The unmanned aerial vehicle is controlled to fly autonomously above the target area, an environment map is constructed based on the laser radar and the depth camera, and a wide range of inspection is performed on the target area during flight; Through the air visual recognition algorithm deployed in the unmanned aerial vehicle, the region to be analyzed that may have cracks is detected and recognized in real time, and it is judged whether the region to be analyzed is located at a low position, the low position being a position that can be reached and detected by the tracked vehicle; If it is at a low position, the environment map and the position of the region to be analyzed are synchronously transmitted to the tracked vehicle and the crack health analysis platform through wireless communication; According to the environment map and the position of the region to be analyzed, the tracked vehicle is triggered to perform path planning and automatically navigate to the region to be analyzed to obtain a high-resolution image of the region to be analyzed, and it is judged whether there are cracks; If there are cracks, the crack image is finely segmented, and the shape, length and width of the crack are evaluated and analyzed by the algorithm built-in the tracked vehicle, and it is judged whether repair is needed; If repair is needed, the tracked vehicle is triggered to temporarily repair the crack, and the repair information is transmitted to the crack health analysis platform.

[0020] It should be noted that in this embodiment, for the low position, the tracked vehicle is used to segment the crack.

[0021] Please refer to Figure 1 In a preferred embodiment of the present application, the road crack intelligent identification and accurate repair method based on land-air cooperation further comprises: If the position of the region to be analyzed is not at a low position, a high-resolution image of the region to be analyzed is obtained by the unmanned aerial vehicle, and it is judged whether there are cracks; If there is a crack, the crack image is finely segmented, and the shape, length and width of the crack are evaluated and analyzed by the built-in algorithm of the unmanned aerial vehicle, and it is judged whether it needs to be repaired; If it needs to be repaired, the environment map and the position of the area to be analyzed are sent to the tracked vehicle and the crack health analysis platform through wireless communication, triggering the tracked vehicle to perform path planning and automatically navigate to the area to be analyzed, temporarily repair the crack, and transmit the repair information to the crack health analysis platform.

[0022] It should be noted that in this embodiment, the crack is segmented by the unmanned aerial vehicle.

[0023] In a preferred embodiment of the present application, before temporarily repairing the crack, the road crack intelligent identification and accurate repair method based on land-air cooperation further comprises: The volume of the crack is detected by laser scanning to determine the amount of glue needed for the crack, so as to avoid wasting materials.

[0024] In a preferred embodiment of the present application, the road crack intelligent identification and accurate repair method based on land-air cooperation further comprises: After the tracked vehicle completes the crack repair task, the repair information is sent to the crack health analysis platform through wireless means; The repair information is processed and visually displayed by the crack health analysis platform, and a reference suggestion is given in combination with the evaluation model to assist the user in the later maintenance of the crack.

[0025] It should be noted that in the present application, the crack health analysis platform equipped in the system backend receives the multi-source data transmitted by the unmanned aerial vehicle and the tracked vehicle, automatically extracts crack length, width, area and other parameters, and gives health evaluation and maintenance suggestions. Different from the existing data output type platform, the present application supports decision support and visual presentation, further improving the value of the system.

[0026] Please refer to Figure 2 and Figure 3 , the present application provides a road crack intelligent identification and accurate repair system based on land-air cooperation, which comprises an unmanned aerial vehicle 11, a tracked vehicle 12, and a crack health analysis platform for controlling the unmanned aerial vehicle 11 and the tracked vehicle 12 through wireless communication: The top of the tracked vehicle 12 is provided with a parking apron 121, and the parking apron 121 is provided with a two-dimensional code mark for identification and visual positioning of the unmanned aerial vehicle 11; The top of the tracked vehicle 12 is provided with a glue supplementing assembly 13, the front end is provided with a rotatable depth camera 122, and the two side walls are provided with a searchlight 123 and a laser radar 124.

[0027] It needs to be said that in the present application, the crack health analysis platform is a controllable terminal responsible for remote input management (such as specifying the detection area), motion control (such as controlling the motion of the unmanned aerial vehicle and the tracked vehicle), and sensor data processing (such as processing the data transmitted by the unmanned aerial vehicle and the tracked vehicle), such as a computer, a tablet, or a mobile phone. The depth camera 122 is an Intel RealSense D455 stereo camera, which provides high-precision RGB-D data for close-range obstacle detection, road monitoring, and three-dimensional crack preliminary modeling; the laser radar 124 is a 270° wide-view 2D laser radar, which realizes high-precision ranging in the rear and side rear, and can realize nearly 360° environmental perception in cooperation with the depth camera 122.

[0028] Please refer to Figure 2 and Figure 3 In a preferred embodiment of the present application, the glue supplementing assembly 13 includes a mechanical arm 131 installed on the tracked vehicle 12, an air pressure pump 132, a liquid storage tank 133 for storing gel, and an injection gun 134 and a camera 135 installed on the execution end of the mechanical arm 131; The camera 135 is used for close-range high-precision image acquisition and morphological analysis of cracks, and the mounting axes of the camera 135 and the injection gun 134 are arranged in parallel. The injection gun 134, the air pressure pump 132, and the liquid storage tank 133 are connected through a hose 136.

[0029] In a preferred embodiment of the present application, a laser scanning piece is provided on the execution end of the mechanical arm 131, which triggers the running time of the air pressure pump 132 after detecting the volume of the crack to determine the amount of glue needed for the crack, thereby avoiding waste of materials.

[0030] In another preferred embodiment of the present application, the repeatability of the mechanical arm 131 is ±0.1 mm, and the maximum working radius is 850 mm. The camera 135 is a 1080p high-definition camera that can obtain detailed visual data. The mechanical arm 131 is a UR5 six-degree-of-freedom mechanical arm, which can flexibly adjust the posture through its multi-joint structure, adapt to different road surface geometries, and provide key inputs for motion control, sensor fusion, and state estimation by continuously measuring three-axis acceleration, angular velocity, and attitude, thereby significantly improving the accuracy and integrity of crack detection.

[0031] The following will be described in detail with specific embodiments.

[0032] 1. Crack data set construction The RGB-D camera and wireless communication module on Tank are used to collect and transmit road surface images in real time. According to the visual feedback, the remote navigation robot goes to the potential crack area, adjusts the end angle and distance through the mechanical arm, and uses the 1080p high-definition camera to capture high-resolution images at close range. The crack images are transmitted back to the ground station in real time for storage, then annotated by humans, and finally a crack-specific dataset of 777 images with a resolution of 640x480 pixels is constructed.

[0033] 2 Model training and evaluation To improve model generalization and detection accuracy, the self-built dataset is combined with the public CrackForest dataset. A total of 716 images are used for training, 90 for validation, and 89 for testing.

[0034] Five mainstream object detection algorithms, Faster R-CNN, YOLOv5, YOLOv7, YOLOv9, and YOLOv11, are selected for performance comparison. All models are implemented based on the PyTorch framework, and training parameters are unified to ensure fair comparison. The experimental platform is Intel Xeon Gold 6226R 2.90GHz CPU and NVIDIA GeForce RTX 3090 GPU (24GB). The SGD optimizer is used for training with a learning rate of 5e-3, a batch size of 24, and 500 rounds of training.

[0035] Eight evaluation indicators are used to measure model performance: precision, recall, F1 score, mAP@0.5, mAP@0.5:0.95, model parameter size, FLOPs, and inference frame rate (FPS).

[0036]

[0037] Please refer to Figure 4 and Table 1, YOLOv11 outperforms other models in most evaluation indicators. Its precision is 0.8194, recall is 0.8579, F1 score is 0.8357, mAP@0.5 is 0.8853, mAP@0.5:0.95 is 0.6622, parameter size and FLOPs are the lowest, and it maintains a real-time detection speed of 65.8FPS, which is sufficient to meet the actual deployment requirements.

[0038] 3 Field test The trained YOLOv11 model is deployed on the industrial computer of the Tank for field testing. The test results show that high confidence multi-target detection can be achieved in various environments, including transverse, longitudinal, block and crack types, and the crack contour can be accurately delineated. Even in the presence of leaves, branches or complex textured surfaces, the model still has strong discrimination ability. In the face of strong light and weak light conditions, the detection performance remains stable, showing strong robustness to light changes.

[0039] In summary, the present application has the following advantages: First, through land-air separation type cooperative design, the unmanned aerial vehicle and the tracked vehicle each perform their own functions and complement each other, realizing full-process operation across space. This cooperative mode breaks through the limitations of traditional system coupling rigidity and function fragmentation, and realizes efficient and flexible task cooperation. Second, the on-site automatic repair mechanism is introduced, and the tracked vehicle is equipped with a pneumatic injection gun. After crack evaluation, the injection gun is immediately executed to plug the crack, forming a closed-loop process from detection to temporary repair, solving the problem that the existing system can only detect but cannot handle cracks on site. Third, the crack health analysis platform is integrated, and the image, detection parameters and maintenance recommendations are visualized, making detection, evaluation and decision unified on one platform, improving the information closed-loop capability, user interaction experience and engineering application value of the system.

[0040] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above examples, and the above examples and descriptions in the specification are only the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection claimed by the present application is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent identification and precise repair of road cracks based on land-air collaboration, characterized in that, Includes the following steps: The drone is controlled to fly autonomously over the target area, and an environmental map is built based on lidar and depth camera. During the flight, the drone can conduct a large-scale inspection of the target area. By using an aerial vision recognition algorithm deployed in the drone, the system can detect and identify areas that may have cracks in real time, and determine whether the area to be analyzed is located at a low position, which is a position that the tracked vehicle can reach and detect. If the location is low, the environmental map and the location of the area to be analyzed will be transmitted synchronously to the tracked vehicle and the crack health analysis platform via wireless communication. Based on the environmental map and the location of the area to be analyzed, the tracked vehicle is triggered to perform path planning and automatically navigate to the area to be analyzed in order to obtain a high-resolution image of the area and determine whether cracks exist. If cracks exist, the crack image is finely segmented, and the shape, length, and width of the cracks are evaluated and analyzed using the algorithm built into the tracked vehicle to determine whether repair is necessary. If repair is needed, the tracked vehicle will be triggered to temporarily repair the crack and transmit the repair information to the crack health analysis platform.

2. The intelligent identification and precise repair method for road cracks based on land-air collaboration as described in claim 1, characterized in that, This intelligent road crack identification and precise repair method based on land-air collaboration also includes: If the area to be analyzed is not located at a low position, a high-resolution image of the area to be analyzed is obtained by using a drone, and it is determined whether there are cracks. If cracks exist, the crack image is finely segmented, and the shape, length, and width of the cracks are evaluated and analyzed using the drone's built-in algorithm to determine whether repair is necessary. If repair is needed, the environmental map and the location of the area to be analyzed are simultaneously sent to the tracked vehicle and the crack health analysis platform via wireless communication. This triggers the tracked vehicle to perform path planning and automatically navigate to the area to be analyzed, where temporary repairs are made to the cracks. The repair information is then transmitted to the crack health analysis platform.

3. The intelligent identification and precise repair method for road cracks based on land-air collaboration as described in claim 1 or 2, characterized in that, Before temporarily repairing the cracks, this land-air collaborative intelligent identification and precise repair method for road cracks also includes: The volume of the crack is detected by laser scanning to determine the amount of adhesive needed to fill the crack.

4. The intelligent identification and precise repair method for road cracks based on land-air collaboration as described in claim 1 or 2, characterized in that, This intelligent road crack identification and precise repair method based on land-air collaboration also includes: After the tracked vehicle completes the crack repair task, it sends the repair information to the crack health analysis platform wirelessly. The crack health analysis platform processes and visualizes repair information, and provides reference suggestions based on the assessment model to assist users in the later maintenance of cracks.

5. A road crack intelligent identification and precise repair system based on land-air collaboration, characterized in that, This includes drones, tracked vehicles, and a crack health analysis platform that controls the drones and tracked vehicles via wireless communication. The tracked vehicle is equipped with a landing pad on its top, and the landing pad is equipped with a QR code for the UAV to identify and visually locate. The tracked vehicle is equipped with a glue-applying assembly on its top, a rotatable depth camera at the front, and searchlights and lidar on both side walls.

6. The intelligent road crack identification and precise repair system based on land-air collaboration according to claim 5, characterized in that, The gel filling assembly includes a robotic arm mounted on the tracked vehicle, a pneumatic pump, a reservoir for storing gel, and an injection gun and a camera mounted on the end effector of the robotic arm. The camera is used for close-range, high-precision image acquisition and morphological analysis of the crack, and the mounting axes of the camera and the injection gun are set parallel to each other; The injection gun, the air pump, and the liquid storage tank are connected by a hose.

7. The intelligent road crack identification and precise repair system based on land-air collaboration according to claim 6, characterized in that, The robotic arm is equipped with a laser scanning device at its end effector. After the laser scanning device detects the volume of the crack, it triggers the operation of the pneumatic pump to determine the amount of adhesive required to inject into the crack.

8. The intelligent road crack identification and precise repair system based on land-air collaboration according to claim 6, characterized in that, The robotic arm has a repeatability of ±0.1mm and a maximum working radius of 850mm.

Images