Road crack automatic repairing system and method with multi-stage perception function

Abstract

The invention discloses an automatic road crack repairing system and method with a multi-stage perception function, and belongs to the technical field of road maintenance equipment. The system is installed on a mobile carrier and comprises a multi-stage sensing module, a chassis cleaning module, an execution module and a central control unit. Relates to a modularized road crack automatic repairing system which integrates multi-stage visual inspection, laser ranging compensation and chassis pre-cleaning functions and can be adapted to various mobile carriers. The method comprises the steps of inspection guidance, dynamic pre-cleaning, precise positioning and parking, profiling repair and reset and circulation. According to the automatic road crack repairing system integrating graded sensing and chassis pre-sweeping, through cooperation of specific sensor layout and an execution mechanism, the problems that vehicle operation parking is difficult, the road surface profiling precision is low and the procedure connection efficiency is low are solved, and efficient cooperation of sweeping and repairing is achieved.

Description

Technical Field

[0001] This invention relates to a road crack repair system and method, specifically to an automated road crack repair system and method with multi-level sensing, belonging to the field of road maintenance equipment technology. Background Technology

[0002] Road crack repair is a core business of highway maintenance. Addressing the issues of low efficiency and poor quality associated with traditional manual repair, road crack repair operations are gradually transitioning towards automation. Existing vehicle-mounted automated repair equipment typically consists of a mobile vehicle, a vision camera, and a robotic arm. While achieving unmanned operation, the following structural defects still exist in practical engineering applications: First, there is a lack of effective positioning methods. Existing equipment mostly relies on a single camera to capture images of a localized section of the road surface, leaving significant blind spots for the driver or autonomous driving system. This makes it difficult to determine whether a crack has accurately entered the effective working radius of the robotic arm. Consequently, the vehicle's parking position deviates significantly, requiring frequent adjustments and resulting in low operational efficiency.

[0003] Second, there is a lack of high-dimensional perception capabilities. Conventional repair systems are mostly based on two-dimensional images for planar planning, ignoring height changes caused by road potholes, bulges, and vehicle suspension settlement. The lack of end-point ranging feedback means that the crack sealant gun cannot follow the contour, which can easily cause the nozzle to hit the road surface or result in inaccurate sealant application due to excessive height above the ground.

[0004] Third, the spatial layout of cleaning and repair processes is unreasonable. Existing equipment often suffers from severe interference between cleaning and repair devices, or requires robotic arms to be reused at different times. There is a lack of a compact spatial layout solution that can utilize vehicle movement or parking intervals for independent and rapid pre-cleaning.

[0005] Fourth, poor system integration and adaptability: Most existing equipment is an integrated customized design for specific vehicle models. The functional modules (such as robotic arms, material supply systems, and sensing units) lack standardized mechanical and electrical interfaces, making it difficult to quickly migrate and adapt to different types of engineering vehicles or trailer chassis.

[0006] Therefore, there is an urgent need for an automated repair system and repair method that can solve the positioning and contouring problems through multi-level sensors and achieve efficient coordination of cleaning and repair through optimized spatial layout. Summary of the Invention

[0007] To address the problems of difficult vehicle parking, low road surface contour accuracy, and low process connection efficiency, this invention proposes an automated road crack repair system and method with multi-level sensing. It integrates hierarchical sensing and chassis pre-cleaning, and achieves efficient coordination of cleaning and repair through a specific sensor layout and actuators.

[0008] The technical solution adopted by the present invention to solve the above problems is as follows: An automated road crack repair system with multi-level sensing capabilities is provided. The system is installed on a mobile vehicle and includes a multi-level sensing module, a chassis cleaning module, an execution module, and a central control unit. The multi-level perception module is used for hierarchical environmental perception and positioning; it includes a first-level guidance camera, a second-level positioning camera and a third-level ranging sensor. The first-level guidance camera is used to acquire images of the road surface in front of the mobile vehicle. The second-level positioning camera is used to locate the location of road cracks and acquire images of the working area. The third-level ranging sensor is used to detect the vertical height of the execution module end relative to the road surface. The chassis cleaning module is used to pre-clean the target crack area and eliminate repair obstacles; The execution module is used to perform spatial repair actions; it includes a robotic arm base and a multi-degree-of-freedom robotic arm. The robotic arm base is installed at the tail of the mobile vehicle, the multi-degree-of-freedom robotic arm is installed on the robotic arm base, and a crack filling actuator is installed at the end of the multi-degree-of-freedom robotic arm. The central control unit is electrically connected to the multi-level sensing module, the chassis cleaning module and the execution module, respectively, and is used for data processing, logical decision-making and multi-module collaborative control.

[0009] Furthermore, the primary guidance camera is mounted at the front end of the mobile vehicle, with its lens facing the direction in which the mobile vehicle is traveling.

[0010] Furthermore, the secondary positioning camera is installed at the rear of the mobile vehicle, with its lens tilted downwards.

[0011] Furthermore, the three-level ranging sensor is installed on the side of the end effector of the execution module.

[0012] Furthermore, the central control unit is equipped with a deep learning-based image processing module. The image processing module runs a target detection algorithm to process the image streams transmitted by the primary guide camera and the secondary positioning camera in real time, identify the crack distribution and determine the location of the target crack, and trigger the braking or parking command of the mobile vehicle and obtain the planned path of the multi-degree-of-freedom robotic arm based on the location of the target crack.

[0013] Furthermore, a transformation relationship between the camera pixel coordinate system and the robotic arm base is established through spatial calibration. This spatial calibration is achieved through the following steps: S1. Fix a calibration plate with a known geometric pattern to the end of the multi-degree-of-freedom robotic arm; S2. Control the multi-degree-of-freedom robotic arm to move the calibration plate to multiple different poses within the field of view of the secondary positioning camera; S3. In each pose, the image including the calibration plate is acquired by the secondary positioning camera, and the pose data of the end effector of the multi-degree-of-freedom robotic arm is recorded simultaneously. S4. Based on the collected multiple sets of images and the corresponding robotic arm pose data, calculate the spatial transformation matrix between the camera coordinate system of the secondary positioning camera and the coordinate system of the robotic arm base.

[0014] Furthermore, the third-level ranging sensor is a single-point lidar or a ToF ranging module, whose detection beam axis is parallel to the discharge axis of the crack filling actuator, and the projections of the two on the horizontal plane maintain a fixed geometric offset.

[0015] Furthermore, the chassis cleaning module includes an air knife nozzle, which is fixedly installed below the chassis of the mobile vehicle. Along the direction of travel of the mobile vehicle, the air knife nozzle is located between the primary guide camera and the execution module, and the nozzle orifice of the air knife nozzle faces the area of ​​the crack to be repaired.

[0016] Furthermore, the air knife nozzle includes several nozzles, which are arranged parallel to each other below the chassis of the mobile vehicle along the direction of travel of the mobile vehicle.

[0017] Furthermore, the multi-degree-of-freedom robotic arm is detachably connected to the robotic arm base, and the multi-degree-of-freedom robotic arm is a six-axis articulated robotic arm or a three-axis Cartesian coordinate robotic arm.

[0018] Furthermore, the system also includes a feeding module for providing crack filling material and conveying it to the crack filling actuator. The feeding module includes a hopper, the outlet of which is connected to a conveying pipe, and the other end of the conveying pipe is connected to the crack filling actuator.

[0019] Furthermore, the system also includes a power supply module for providing an independent and stable power supply to the entire system, including a generator connected to a voltage regulator module. The generator outputs electrical energy, which is processed by the voltage regulator module and then supplied to the multi-level sensing module, the chassis cleaning module, the execution module, the central control unit, and the feeding module through an electrical connection interface.

[0020] An automated road crack repair method with multi-level sensing, employing the aforementioned automated road crack repair system with multi-level sensing, includes the following steps: Step 1: Inspection and Guidance; As the mobile vehicle moves, the primary guidance camera captures images of the road surface ahead, and the central control unit identifies crack targets within them; Step 2, Dynamic pre-cleaning; When the mobile vehicle approaches the target crack area, the chassis cleaning module is activated, and the air knife nozzles are used to blow away the cracks that are about to enter the work area. Step 3: Precise Positioning and Parking; The work area is continuously monitored by a secondary positioning camera. When the central control unit determines that the target crack has entered the preset work area, the mobile vehicle is triggered to park. Step 4, contour repair: After the mobile vehicle comes to a stop, the multi-degree-of-freedom robotic arm of the control module drives the crack filling actuator to move along the crack. At the same time, the height of the crack filling actuator is adjusted in real time according to the feedback data of the three-level distance measuring sensor, and the filling operation is carried out.

[0021] Step 5, Reset and Cycle: After completing the filling operation of the current target crack, control the multi-degree-of-freedom robotic arm to drive the crack filling actuator to reset to the preset safe position; then, control the mobile vehicle to start, and repeat steps one to four to perform automated repair operation on the next target crack.

[0022] The beneficial effects of this invention are: 1. This invention, through the cooperation of primary guidance and secondary positioning, combined with the automatic detection of target detection algorithm based on deep learning and the working area determination logic, constructs a complete perception chain from macroscopic search to microscopic positioning, ensuring that the vehicle accurately stops at the optimal working position in one go, achieving precise parking and blind spot elimination.

[0023] 2. This invention introduces a three-level distance measuring sensor at the end, which gives the system the ability to follow road surface contours, effectively cope with road surface undulations, ensures that the crack sealing gun is always at the optimal working height, and improves the quality of operation.

[0024] 3. This invention places the cleaning module under the chassis and between the detection and execution stations. This spatial layout allows the system to complete pre-cleaning during the time it takes for the vehicle to decelerate from a driving state to a parked state, realizing a streamlined operation from positioning to cleaning to repair, without occupying the robotic arm's operating time.

[0025] 4. The subsystems of this invention adopt a modular design, are relatively independent, and have standardized interfaces. This not only facilitates maintenance but also allows for flexible adaptation to engineering vehicles of different sizes and types, lowering the system integration threshold. Through a universal flange base, flexible conveying pipe, and independent power supply module, the execution module and material supply module can be migrated as a whole to different vehicle models. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of one embodiment of the automated road crack repair system with multi-level sensing of the present invention; Figure 2 yes Figure 1 A schematic diagram of the structure viewed from below; Figure 3 This is a flowchart illustrating one implementation of the automated road crack repair method with multi-level sensing according to the present invention.

[0027] In the picture: 1-Mobile vehicle; 2-Central control unit; 10-Multi-level sensing module; 11-Level 1 guidance camera; 12-Level 2 positioning camera; 13-Level 3 ranging sensor; 20- Chassis cleaning module; 21- High-pressure blower; 22- Air knife nozzle; 30 - Execution module; 31 - Multi-degree-of-freedom robotic arm; 32 - Crack filling actuator; 33 - Robotic arm base; 40 - Feeding module; 41 - Hopper; 42 - Conveying pipe; 50 - Power supply module; 51 - Generator. Detailed Implementation

[0028] In the description of this invention, it should be noted that all directional indications (e.g., front, back, etc.) are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0029] Specific implementation method one: Combining Figure 1-3 This implementation method is described as follows: Figure 1 and Figure 2 As shown in the figure, this embodiment describes an automated road crack repair system with multi-level perception. It relates to a modular automated road crack repair system that integrates multi-level visual detection, laser ranging compensation, and chassis pre-cleaning functions, adaptable to various mobile vehicles. The system is installed on a mobile vehicle 1 and includes a multi-level perception module 10, a chassis cleaning module 20, an execution module 30, and a central control unit 2. Preferably, the system adopts a modular design, with each module detachably installed on the mobile vehicle 1 via a standard mounting interface. This facilitates maintenance and allows for flexible adaptation to engineering vehicles of different sizes and types, lowering the system integration threshold.

[0030] The multi-level sensing module 10 is used for hierarchical environmental perception and positioning, including a primary guidance camera 11, a secondary positioning camera 12, and a tertiary ranging sensor 13. Through the hierarchical architecture of primary macroscopic guidance, secondary microscopic positioning, and tertiary fine ranging, it realizes full-process perception from crack detection to accurate determination of the working space.

[0031] The primary guidance camera 11 is used to acquire images of the road surface ahead of the mobile vehicle 1. The primary guidance camera 11 is installed at the front end of the mobile vehicle 1, with its lens facing forward. The primary guidance camera 11 is installed at a high position at the front end of the mobile vehicle 1 (such as the top of the cab) to acquire images of the road surface ahead with a wide field of view (covering 5-15 meters of road surface in front of the vehicle), and transmits the acquired images to the central control unit 2.

[0032] The secondary positioning camera 12 is used to locate road cracks and collect images of the working area. The secondary positioning camera 12 is mounted at the rear of the mobile vehicle 1, with its lens tilted downwards. The tilt angle between the lens and the horizontal plane is preferably 25°-60°, most preferably 30°, and its lens faces downwards to ensure that the field of view covers the working space of the execution module 30 and the road surface area in front of the working space. The secondary positioning camera 12 is used to collect images of the working area in real time and transmit them to the central control unit 2.

[0033] The three-level ranging sensor 13 is installed on the side of the end effector of the execution module 30 and is used to detect the vertical height of the end effector of the execution module 30 relative to the road surface. Preferably, the three-level ranging sensor 13 is a single-point lidar or a ToF ranging module, and its detection beam axis is parallel to the discharge axis of the end effector of the execution module 30, with their projections on the horizontal plane maintaining a fixed geometric offset. The three-level ranging sensor 13 transmits the geometric offset to the central control unit 2.

[0034] The chassis cleaning module 20 includes an air knife nozzle 22, which is fixedly installed below the chassis of the mobile vehicle 1. Along the traveling direction of the mobile vehicle 1, the air knife nozzle 22 is located between the primary guide camera 11 and the execution module 30. The nozzle nozzle 22 faces the crack area to be repaired, and is used to pre-clean the target crack area to eliminate repair obstacles. Placing the cleaning module below the chassis and between the detection and execution stations allows the system to complete pre-cleaning during the time it takes for the vehicle to decelerate from a driving state to a parked state, achieving a streamlined operation from positioning to cleaning to repair, without occupying the robotic arm's operating time. Preferably, the air knife nozzle 22 includes several nozzles arranged parallel to each other below the chassis of the mobile vehicle 1 along a direction perpendicular to the traveling direction of the mobile vehicle 1. The air knife nozzle 22 is a horizontally arranged multi-nozzle array structure, and its effective blowing width is greater than the horizontal working coverage width of the execution module 30. Several nozzles are evenly spaced below the chassis of the mobile vehicle 1 along a direction perpendicular to the travel direction of the mobile vehicle 1 to ensure a reasonable cleaning range and improve the cleaning efficiency of the chassis cleaning module 20. The nozzle direction of the air knife nozzle 22 is at a preset angle (e.g., 30°-60°) to the road surface to blow debris forward. The air knife nozzle 22 is connected to a high-pressure blower 21 to further enhance cleaning efficiency.

[0035] The execution module 30 includes a robotic arm base 33 and a multi-degree-of-freedom robotic arm 31. The robotic arm base 33 is mounted at the tail of the mobile carrier 1, and the multi-degree-of-freedom robotic arm 31 is mounted on the robotic arm base 33. A crack filling actuator 32 (such as a crack filling gun) is installed at the end of the multi-degree-of-freedom robotic arm 31 for performing spatial repair actions. Combined with the sensing data from the multi-level sensing module 10, high-precision crack filling is achieved. The multi-degree-of-freedom robotic arm 31 is detachably connected to the robotic arm base 33. The multi-degree-of-freedom robotic arm 31 is a six-axis articulated robotic arm or a three-axis Cartesian coordinate robotic arm, and the robotic arm can be replaced according to operational requirements.

[0036] The automated road crack repair system with multi-level sensing also includes a material supply module 40. The material supply module 40 includes a hopper 41, the outlet of which is connected to a conveying pipe 42. The other end of the conveying pipe 42 is connected to the crack filling actuator 32, used to provide crack filling material and convey it to the crack filling actuator 32. Preferably, the material supply module 40 also includes a pumping unit, used to pressurize the material in the hopper 41. The hopper 41 has heating and insulation functions, such as an asphalt hopper 41. The conveying pipe 42 is a high-temperature resistant flexible conveying pipe. The hopper 41 stores crack filling material and maintains a suitable temperature. The pumping unit pressurizes the material in the hopper 41 and conveys it to the crack filling actuator 32 through the flexible conveying pipe 42, ensuring that the material properties are suitable for the repair requirements.

[0037] The system also includes a power supply module 50, which provides an independent and stable power supply for the entire system. The module includes a generator 51, which is connected to a voltage regulator module. The power output from the generator 51 is processed by the voltage regulator module and then supplied to the multi-level sensing module 10, the chassis cleaning module 20, the execution module 30, the central control unit 2, and the feeding module 40 through an electrical connection interface.

[0038] The central control unit 2 is electrically connected to the multi-level sensing module 10, the chassis cleaning module 20, the execution module 30, the feeding module 40, and the power supply module 50, respectively, and is used for data processing, logical decision-making, and multi-module collaborative control. Specifically, it is used to receive detection signals from the multi-level sensing module 10 and control the start and stop of the chassis cleaning module 20 and the operation of the execution module 30.

[0039] The central control unit 2 is equipped with a deep learning-based image processing module. This module runs a target detection algorithm (such as YOLO) to process the image streams transmitted from the primary guide camera 11 and the secondary positioning camera 12 in real time. The images acquired by the primary guide camera 11 are processed by the target detection algorithm in the central control unit 2 to identify the crack distribution trend and guide the mobile vehicle 1 to adjust its course so that the crack is located at the center of the travel path. The images acquired by the secondary guide camera are processed by the central control unit 2 to determine the location of the target crack and whether it has entered a preset working area. After determining that the target crack has entered the preset working area, the crack is simultaneously segmented using deep learning methods, the crack skeleton is extracted and mapped, and the planned path of the multi-degree-of-freedom robotic arm 31 is obtained. Specifically, it is determined whether the target crack has entered a preset optimal working rectangle (ROI). When the center coordinates of the detection box enter the pixel range of the working area calibrated by the secondary positioning camera 12, a braking or parking signal is triggered on the mobile vehicle 1. The braking or parking command of the mobile vehicle 1 is triggered based on the location of the target crack. Once it enters the preset working area, a parking command is immediately triggered, and the planned path of the multi-degree-of-freedom robotic arm 31 is obtained simultaneously. By combining primary guidance with secondary positioning, and integrating deep learning-based target detection algorithms for automatic detection and work area determination logic, a complete perception chain from macroscopic search to microscopic positioning is constructed, ensuring that the vehicle accurately stops at the optimal working position in one go, achieving precise parking and blind spot elimination.

[0040] The central control unit 2 receives the height data and geometric offset from the three-level ranging sensor 13, and calculates the actual road surface height at the nozzle of the crack sealing actuator 32 based on the posture of the multi-degree-of-freedom robotic arm 31 of the execution module 30. It also detects the vertical height of the end-effector crack sealing actuator 32 relative to the road surface in real time, providing data for Z-axis compensation of the robotic arm and controlling the Z-axis movement of the robotic arm for height compensation, thus achieving road surface contour compensation. The introduction of the end-effector three-level ranging sensor 13 endows the system with road surface contour capabilities, effectively handling road surface undulations and ensuring that the crack sealing gun is always at the optimal working height, thereby improving work quality.

[0041] When the vehicle decelerates, the central control unit 2 predicts the moment when the crack will reach below the air knife nozzle 22 based on the crack location identified by the primary guide camera 11, and starts the high-pressure blower 21 to scan and blow the crack area using the relative motion of the vehicle's forward movement, ensuring that the crack is free of debris when the vehicle is parked.

[0042] After the vehicle is parked, the crack sealing actuator 32 moves according to the path planned by the central control unit 2. At the same time, the Z-axis height is adjusted in real time (maintaining the optimal crack sealing height of 2-3cm above the ground) through inverse kinematics calculation combined with the height data of the three-level distance sensor 13, so as to realize the road surface contour operation and complete the crack sealing material filling.

[0043] The secondary positioning camera 12 and the robotic arm base 33 establish a transformation relationship between the camera pixel coordinate system and the robotic arm base 33 coordinate system through spatial calibration, making the position of the working area in space clear and solvable. The spatial calibration is achieved through the following steps: S1. Fix a calibration plate (checkerboard calibration plate, dot array calibration plate, or Charuco combination calibration plate) with a known geometric pattern to the end of the multi-degree-of-freedom robotic arm 31. S2. Control the multi-degree-of-freedom robotic arm 31 to move the calibration plate to multiple different poses within the field of view of the secondary positioning camera 12; S3. In each pose, the secondary positioning camera 12 acquires an image containing the calibration plate, and simultaneously records the pose data of the end of the multi-degree-of-freedom robotic arm 31. The pixel coordinates of the feature points of the calibration plate in the image are detected by the image processing algorithm. S4. Based on the acquired multiple sets of images and the corresponding robotic arm pose data, calculate the spatial transformation matrix between the camera coordinate system of the secondary positioning camera 12 and the coordinate system of the robotic arm base 33. The spatial transformation matrix is ​​used to convert the crack pixel coordinates identified in the image of the secondary positioning camera 12 into three-dimensional spatial coordinates in the coordinate system of the robotic arm base 33 in real time, so as to enable the robotic arm to plan its motion trajectory.

[0044] An automated road crack repair method with multi-level sensing, employing the aforementioned automated road crack repair system with multi-level sensing, includes the following steps: Step 1: Inspection and guidance; As the mobile vehicle 1 moves, it collects images of the road surface ahead through the primary guidance camera 11, and the central control unit 2 identifies crack targets in the images. Step 2, Dynamic pre-cleaning; When the mobile vehicle 1 approaches the target crack area, the chassis cleaning module 20 is activated, and the air knife nozzle 22 is used to blow away the cracks that are about to enter the work area. Step 3: Precise positioning and parking; The secondary positioning camera 12 continuously monitors the work area. When the central control unit 2 determines that the target crack has entered the preset work area, it triggers the parking of the mobile vehicle 1 to obtain the planned path of the multi-degree-of-freedom robotic arm 31. Step 4, contour repair: After the mobile carrier 1 comes to a stop, the multi-degree-of-freedom robotic arm 31 of the control execution module 30 drives the crack filling actuator 32 to move along the crack. At the same time, the height of the crack filling actuator 32 is adjusted in real time according to the feedback data of the three-level distance sensor 13, and the filling operation is carried out.

[0045] Step 5, Reset and Cycle: After completing the filling operation of the current target crack, control the multi-degree-of-freedom robotic arm 31 to drive the crack filling actuator 32 to reset to the preset safe position; then, control the mobile carrier 1 to start and repeat steps one to four to perform automated repair operation on the next target crack.

[0046] The present invention will now be described in detail with reference to specific embodiments.

[0047] Example 1 This embodiment describes the overall layout and modular design of the automated road crack repair system with multi-level sensing capabilities.

[0048] like Figure 1 and Figure 2 As shown, the present invention provides a modular automated road crack repair system. In this embodiment, the system is integrated and installed on a modified engineering truck (mobile vehicle 1).

[0049] The system adopts a modular design, including: Power supply module 50: Generator 51 and voltage stabilizing module are installed at the front end of the truck bed to provide independent power for the entire system.

[0050] Material supply module 40: includes an asphalt silo 41 with heating and heat preservation functions and a pumping component, which is connected to the end material gun through a high-temperature resistant flexible conveying pipe 42, and to the vehicle interface through a standardized lifting hole and a quick electrical plug.

[0051] Module 30: Installed at the rear of the truck bed. This embodiment uses a six-axis industrial robotic arm, whose base is fixed to the frame via a standard multi-hole flange (universal mounting base), facilitating subsequent maintenance or replacement with a gantry structure.

[0052] The multi-level sensing module 10 is fixed to the vehicle frame via a bracket with a pre-reserved standard hole spacing; the chassis cleaning module 20 is connected to the chassis via a flange seat to adapt to different chassis widths.

[0053] Example 2 In this embodiment, the multi-level sensing module 10 and the chassis cleaning module 20 of the automated road crack repair system with multi-level sensing are arranged.

[0054] like Figure 1 and Figure 2 As shown, this embodiment optimizes the spatial layout of the sensing and cleaning modules: Level 1 Guide Camera 11: Installed on the top of the cab, its field of view covers the road surface 5-15 meters in front of the vehicle, and is used to capture the distribution trend of long-distance cracks.

[0055] Secondary positioning camera 12: Installed at a high position on the rear support of the vehicle, with the lens shooting at an angle downwards, covering the rectangular area in front of the robotic arm base 33 (i.e. the optimal working space of the robotic arm).

[0056] Air knife nozzle 22: Installed under the chassis, located in the area between the fields of view of the primary and secondary cameras. The air knife nozzle 22 consists of multiple horizontally arranged flat nozzles, with an effective cleaning width slightly larger than the lateral working range of the robotic arm, ensuring complete coverage of the crack area. Coordination with chassis cleaning module 20: As the vehicle moves forward, the primary guidance camera 11 at the front of the vehicle detects the crack entering the area below the air knife nozzle 22. At this time, the central control unit 2 activates the high-pressure blower 21 to perform predictive cleaning of the crack (the crack position in the field of view of the primary camera is converted into vehicle coordinates through the calibration relationship with the vehicle coordinates, and then the time when the crack passes below the air knife nozzle 22 is calculated, thereby controlling the start and stop of the blower). When the vehicle continues to move forward and stops at the crack in the robotic arm's working area, the crack is not only precisely located but has also been pre-cleaned.

[0057] Example 3 In this embodiment, the terminal structure and control of the execution module 30 are provided for the automated road crack repair system with multi-level perception.

[0058] The crack filling actuator 32 is a crack filling gun. A three-level ranging sensor 13 (single-point ToF lidar) is fixed parallel to the side of the crack filling gun by a clamp. The optical axis of the sensor is parallel to the discharge axis of the filling gun, and the two maintain a fixed horizontal offset (e.g., 3-5cm).

[0059] During operation, the central control unit 2 reads the height data fed back by the three-level distance measuring sensor 13 in real time and calculates the actual vertical distance between the nozzle and the road surface. When potholes or bumps appear on the road surface, the system adjusts the Z-axis height in real time through the inverse kinematics calculation of the robotic arm, so that the nozzle is always kept at the optimal grouting height of 2-3cm above the ground, realizing road surface contour operation (sampling is performed once on a flat road surface before operation as the initial height reference).

[0060] Example 4 In this embodiment, the method for automated repair of road cracks with multi-level sensing is described.

[0061] like Figure 3 As shown, the steps of the automated road crack repair method with multi-level sensing (the system's automated operation logic) are as follows: Step 1, Inspection and Guidance: The vehicle maintains a constant speed, and the primary guidance camera 11 acquires images in real time. The central control unit 2 runs a deep learning-based target detection algorithm (such as YOLO). When a crack is detected on the road surface ahead, it prompts the driver or the autonomous driving system to slow down and adjust the course so that the crack is located in the center of the vehicle's travel path.

[0062] Step 2, Dynamic Pre-cleaning: As the vehicle decelerates and approaches the target area, the chassis cleaning module 20 automatically starts. High-pressure airflow passes through the air knife nozzles 22, utilizing the relative motion of the vehicle's forward movement to scan and clean the cracks that are about to enter the work area.

[0063] Step 3, Precise Positioning and Parking: The secondary positioning camera 12 continuously monitors the work area. The central control unit 2 determines whether the center coordinates of the detection box of the target detection algorithm based on deep learning (e.g., YOLO) enter the preset optimal working rectangle (ROI). After determining that the target crack has entered the preset work area, the crack is simultaneously segmented using deep learning methods, the crack skeleton is extracted and mapped, and the planned path of the multi-degree-of-freedom robotic arm 31 is obtained. Once it enters the area, a parking command is immediately sent, and the vehicle stops. At this time, the crack is within the reach of the robotic arm and the cleaning has been completed.

[0064] Step 4, Contouring Repair: After the vehicle comes to a complete stop, the robotic arm moves the crack filling gun according to the planned path. During the movement, real-time Z-axis height compensation is performed based on data from the three-level distance sensor 13, while simultaneously controlling the opening of the solenoid valve of the filling gun to complete the crack filling operation.

[0065] Step 5, Cyclic Operation: After completing the repair of the current area, the robotic arm resets, the vehicle starts moving to the next crack point, and the above process is repeated.

[0066] Example 5 In this embodiment, the electrical connection and control logic of the automated road crack repair system with multi-level sensing is described.

[0067] The central control unit 2 serves as the core, integrating an image processing module (a computing unit with a GPU accelerator card in hardware) and a collaborative control module (an IO control board or VCU in hardware).

[0068] The central control unit 2 is connected to the first-level guide camera 11 and the second-level positioning camera 12 via GigE or USB 3.0 interfaces respectively; it is connected to the third-level ranging sensor 13 via RS485 or analog interfaces; it is connected to the controller of the multi-degree-of-freedom robotic arm 31 via Ethernet port; and it is electrically connected to the solenoid valves of the high-pressure blower 21 and the crack filling gun via relay groups.

[0069] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. An automated road crack repair system with multi-level sensing, the system being mounted on a mobile vehicle (1), characterized in that... It includes a multi-level sensing module (10), a chassis cleaning module (20), an execution module (30), and a central control unit (2). The multi-level perception module (10) is used for hierarchical environmental perception and positioning; it includes a first-level guidance camera (11), a second-level positioning camera (12) and a third-level ranging sensor (13). The first-level guidance camera (11) is used to collect images of the road surface in front of the mobile vehicle (1). The second-level positioning camera (12) is used to locate the location of road cracks and collect images of the working area. The third-level ranging sensor (13) is used to detect the vertical height of the end of the execution module (30) relative to the road surface. The chassis cleaning module (20) is used to pre-clean the target crack area and eliminate repair obstacles; The execution module (30) is used to perform space repair actions; it includes a robotic arm base (33) and a multi-degree-of-freedom robotic arm (31). The robotic arm base (33) is installed at the tail of the mobile vehicle (1), and the multi-degree-of-freedom robotic arm (31) is installed on the robotic arm base (33). A crack filling actuator (32) is installed at the end of the multi-degree-of-freedom robotic arm (31). The central control unit (2) is electrically connected to the multi-level sensing module (10), the chassis cleaning module (20) and the execution module (30) respectively, and is used for data processing, logical decision-making and multi-module collaborative control.

2. The automated road crack repair system with multi-level sensing as described in claim 1, characterized in that... The central control unit (2) is equipped with a deep learning-based image processing module. The image processing module runs a target detection algorithm to process the image streams transmitted by the primary guide camera (11) and the secondary positioning camera (12) in real time, identify the crack distribution and determine the target crack position, trigger the braking or parking command of the mobile vehicle (1) according to the target crack position and obtain the planned path of the multi-degree-of-freedom robotic arm (31).

3. The automated road crack repair system with multi-level sensing as described in claim 2, characterized in that... The transformation relationship between the camera pixel coordinate system and the robot arm base (33) is established through spatial calibration between the secondary positioning camera (12) and the robot arm base (33). The spatial calibration is achieved through the following steps: S1. Fix a calibration plate with a known geometric pattern to the end of the multi-degree-of-freedom robotic arm (31); S2. Control the multi-degree-of-freedom robotic arm (31) to move the calibration plate to multiple different poses within the field of view of the secondary positioning camera (12); S3. In each pose, the image containing the calibration plate is acquired by the secondary positioning camera (12), and the pose data of the end of the multi-degree-of-freedom robotic arm (31) is recorded simultaneously. S4. Based on the collected multiple sets of images and the corresponding robotic arm pose data, calculate the spatial transformation matrix between the camera coordinate system of the secondary positioning camera (12) and the coordinate system of the robotic arm base (33).

4. The automated road crack repair system with multi-level sensing as described in claim 2, characterized in that... The three-level ranging sensor (13) is a single-point lidar or ToF ranging module. Its detection beam axis is parallel to the discharge axis of the crack filling actuator (32), and the projections of the two on the horizontal plane maintain a fixed geometric offset.

5. The automated road crack repair system with multi-level sensing as described in claim 1, characterized in that... The chassis cleaning module (20) includes an air knife nozzle (22), which is fixedly installed below the chassis of the mobile vehicle (1). Along the travel direction of the mobile vehicle (1), the air knife nozzle (22) is located between the primary guide camera (11) and the execution module (30), and the nozzle direction of the air knife nozzle (22) is towards the crack area to be repaired.

6. The automated road crack repair system with multi-level sensing as described in claim 5, characterized in that... The air knife nozzle (22) includes several nozzles, which are arranged parallel to each other below the chassis of the mobile vehicle (1) along the direction of travel of the mobile vehicle (1).

7. The automated road crack repair system with multi-level sensing as described in claim 1, characterized in that... The multi-degree-of-freedom robotic arm (31) is detachably connected to the robotic arm base (33), and the multi-degree-of-freedom robotic arm (31) is a six-axis articulated robotic arm or a three-axis rectangular coordinate robotic arm.

8. The automated road crack repair system with multi-level sensing as described in claim 1, characterized in that... The system also includes a feeding module (40) for providing crack filling material and conveying it to the crack filling actuator (32). The feeding module (40) includes a hopper (41), the outlet of which is connected to a conveying pipe (42), and the other end of the conveying pipe (42) is connected to the crack filling actuator (32).

9. The automated road crack repair system with multi-level sensing as described in claim 8, characterized in that... The system also includes a power supply module (50) for providing an independent and stable power supply for the entire system, including a generator (51). The generator (51) is connected to a voltage regulator module. The electrical energy output by the generator (51) is processed by the voltage regulator module and then supplied to the multi-level sensing module (10), the chassis cleaning module (20), the execution module (30), the central control unit (2), and the feeding module (40) through the electrical connection interface.

10. An automated road crack repair method with multi-level sensing, characterized in that... The automated road crack repair system with multi-level sensing capability as described in any one of claims 1-9 includes the following steps: Step 1, Inspection and Guidance; While the mobile vehicle (1) is in motion, the first-level guidance camera (11) collects images of the road surface ahead, and the central control unit (2) identifies the crack targets in them; Step 2, Dynamic pre-cleaning; When the mobile vehicle (1) approaches the target crack area, the chassis cleaning module (20) is activated, and the air knife nozzle (22) is used to blow the crack that is about to enter the work area. Step 3, Precise positioning and parking; The working area is continuously monitored by the secondary positioning camera (12). When the central control unit (2) determines that the target crack has entered the preset working area, the mobile vehicle (1) is triggered to park, and the planned path of the multi-degree-of-freedom robotic arm (31) is obtained. Step 4, contour repair: After the mobile vehicle (1) stops, the multi-degree-of-freedom robotic arm (31) of the control execution module (30) drives the crack filling actuator (32) to move along the crack. At the same time, the height of the crack filling actuator (32) is adjusted in real time according to the feedback data of the three-level distance sensor (13), and the filling operation is carried out. Step 5, Reset and Cycle: After completing the filling operation of the current target crack, control the multi-degree-of-freedom robotic arm (31) to drive the crack filling actuator (32) to reset to the preset safe position; then, control the mobile carrier (1) to start, and repeat steps one to four to perform automated repair operation on the next crack target.

Images