An autonomous crack-patching method and system for road cracks

Abstract

The application discloses a kind of road crack autonomous crack pouring repair methods and systems, method includes the following steps: S1: the multidimensional state feature point cloud of road surface crack is obtained, the multidimensional state feature point cloud includes the three-dimensional space coordinate information of crack skeleton line and crack physical width information;S2: based on the multidimensional state feature point cloud, construct with crack section as node, with the space rotation of crack section between as edge full connectivity topological graph network, and utilize ant colony algorithm on the full connectivity topological graph network carries out path optimization, output optimal repair path traversing all crack sections;S3: according to the optimal repair path and the multidimensional state feature point cloud, generate and include space pose sequence, glue output amount instruction and real-time compensation crack pouring spray gun in the axial direction compensation amount collaborative operation script.The application realizes to the efficient, low energy consumption autonomous crack pouring repair of complex crack on three-dimensional undulating pavement.

Description

Technical Field

[0001] This invention relates to the field of road maintenance engineering technology, specifically to a method and system for autonomous crack filling and repair of road cracks. Background Technology

[0002] Road surface cracks are the earliest and most common form of road damage. Currently, most crack sealing operations in China are still done manually, resulting in low efficiency, reliance on worker experience for quality, and high safety risks. To address this, the industry has proposed several automated alternatives. For example, automated crack sealing vehicles based on XY gantry frames use guide rails and nozzles mounted on the gantry, with a top camera identifying the two-dimensional coordinates of the cracks and controlling the nozzles to move along the XY direction to inject sealant. Another example is crack sealing trolleys based on remote control or preset trajectories, where the chassis moves to drive the spray gun along the crack for repair. However, the above-mentioned existing solutions still have the following shortcomings: On the one hand, the existing spray gun movement is limited to a two-dimensional plane, which cannot adapt to road surface environments with three-dimensional height changes such as road camber and slope. When the nozzle changes with the road surface height, it is easy to cause the glue to deviate or the sealant to splash. On the other hand, the path planning of the existing solution is only for simple linear cracks. When facing network cracks or multi-branch cracks, it is difficult to plan a repair sequence that takes into account both traversal integrity and execution energy consumption, which can easily lead to problems such as frequent gun lifting and repeated paths.

[0003] In summary, there is an urgent need for an autonomous crack filling and repair method and system for roads to solve or at least partially solve the problems existing in the current technology. Summary of the Invention

[0004] The purpose of this invention is to provide an autonomous crack filling and repair method for roads, so as to solve the problem of efficient and low-energy autonomous crack filling for repairing complex network cracks in a three-dimensional pavement environment. The specific technical solution is as follows: A method for self-sealing and repairing road cracks includes the following steps: S1. Obtain a multi-dimensional state feature point cloud of road surface cracks, wherein the multi-dimensional state feature point cloud includes the three-dimensional spatial coordinate information of the crack skeleton line and the physical width information of the crack. S2: Based on the multi-dimensional state feature point cloud, construct a fully connected topological graph network with crack segments as nodes and spatial transitions between crack segments as edges, and use the ant colony algorithm to perform path optimization on the fully connected topological graph network to output the optimal repair path that traverses all crack segments. S3: Based on the optimal repair path and the multi-dimensional state feature point cloud, generate a grouting spray gun containing a spatial pose sequence, glue dispensing command, and real-time compensation. Z Axial compensation Collaborative work scripts.

[0005] Preferably, the specific method for obtaining the three-dimensional spatial coordinate information of the crack skeleton line and the crack physical width information in step S1 includes: S11: Acquire RGB and depth images of the road surface; S12: Perform instance segmentation on the RGB image to obtain a binary mask of the crack region; S13: Perform Euclidean distance transform on the binarized mask to obtain the pixels within the mask region. Distance to the edge of the mask ; S14: Perform morphological thinning on the binarized mask to extract the crack skeleton line composed of single pixels, and the pixels on the crack skeleton line... Distance to the edge of the mask Double the value as a pixel The corresponding physical width characterization value of the crack; S15: Map the pixel coordinates of the crack skeleton line to the depth image, and perform a three-dimensional inverse mapping in combination with the camera intrinsic parameters to generate the three-dimensional spatial coordinates of the crack skeleton line. The physical width characterization value of the crack is mapped to the depth image, and three-dimensional inverse mapping is performed in combination with camera intrinsic parameters to generate the true crack width value of the crack skeleton line. S16: Associate the physical width value of the crack corresponding to each pixel of the crack skeleton line with the three-dimensional spatial coordinates to generate the multi-dimensional state feature point cloud.

[0006] Preferably, the construction of the fully connected topology graph network in step S2 specifically includes: Each refined crack segment is defined as a node in the topological graph network; The spatial transfer action of moving the crack sealing spray gun from one crack segment to another is defined as the edge of the topological graph network, thereby constructing a fully connected topological graph network in which there is a connected path between any two nodes.

[0007] Preferably, in step S2, the ant colony algorithm is used to generate the optimal repair path with the objective of minimizing the overall energy consumption cost; wherein the overall energy consumption cost is expressed as: in, To the crack segment Transfer to the fracture segment The comprehensive energy consumption value; A Boolean variable representing a crack segment With crack segment Whether they are physically directly connected; when Time indicates physical connectivity, when This indicates that the connection is not physical; Not equal to , Indicates except Any crack that has never been there before; For crack segment of actual Physical length; Indicates crack segment With crack segment Between The spatial linear distance of the transfer strategy; The transfer strategy includes crack segments. The end point to the crack segment The starting point or crack segment The end point to the crack segment There are two kinds of endpoints. This represents any one of the transition strategies; The penalty for turning , For crack segment With crack segment The angle between the local tangent vectors at the intersection point; To increase the cost of punishment, when the crack segment With crack segment When physically disconnected, When physically connected ; The fixed lifting work is performed for the robotic arm to perform one Z-axis traverse motion; To characterize the energy consumption per unit distance during the translation of the robotic arm and chassis, This is a conversion coefficient characterizing the energy consumption per unit deflection angle during robotic arm posture adjustment.

[0008] Preferably, in step S2, when using the ant colony algorithm for path optimization, the first... Only virtual ants are generated from the current crack segment. Select to move to the candidate crack segment transition probability Represented as: in, For the first In the next iteration, from the crack segment To the crack section The concentration of pheromones along the path, As a pheromone-inspired factor, As the expected heuristic factor, For the first The set of crack nodes that the virtual ants have not yet visited; Comprehensive energy consumption value The reciprocal of; The summation traversal variable is used to iterate over the set of unvisited crack nodes. Any crack node in the array.

[0009] Preferably, after each iteration, the path with the lowest overall energy consumption is selected for positive feedback updating of pheromone concentration, and the update formula is: in, The pheromone evaporation coefficient, For pheromone increment, Where is the pheromone intensity constant. This represents the comprehensive energy consumption value of the optimal path in this iteration.

[0010] Preferably, the specific method for generating the collaborative task script in step S3 includes: Downsampling is performed on the multidimensional state feature point cloud to extract the spatial position control points of the crack skeleton line; The three-dimensional coordinates of the spatial location control points and their corresponding crack physical width information are used as four-dimensional input data. B-spline curve fitting is performed to obtain a smooth repair trajectory line. At the same time, the crack physical width information of each point on the repair trajectory line is compiled into glue output control parameters. Based on the spatial distance and curvature of the repair trajectory line, the smooth repair trajectory line is segmented. Combined with the optimal repair path, a system is generated that includes a spatial pose sequence, adhesive dispensing instructions, and a real-time compensation grouting spray gun. Z Compensation amount in the axial direction Collaborative work scripts.

[0011] Preferably, in step S3, the real-time compensation grouting spray gun is in Z The position in the axial direction adopts an impedance control strategy, based on the contact force deviation between the crack sealing spray gun and the road surface. An impedance control strategy is used to calculate ,That Z Shaft compensation The following relationship must be satisfied: in, For inertial mass, For damping parameters, For stiffness parameters, This is the deviation between the actual contact force and the expected contact force of the crack sealing spray gun. To compensate for the crack sealing spray gun in Z Acceleration in the axial direction; To compensate for the crack sealing spray gun in Z Velocity in the axial direction; To compensate for the crack sealing spray gun in Z Displacement in the axial direction.

[0012] Preferably, after performing step S3, step S4 is further included: S4: Evaluation step. Calculate the repair coverage rate of the repaired cracks. If the repair coverage rate is greater than or equal to a preset threshold, the repair is deemed qualified; otherwise, a supplementary repair process is triggered.

[0013] On the other hand, this application also provides a self-sealing and repair system for road cracks, comprising: Mobile chassis; A multi-degree-of-freedom robotic arm is mounted on the mobile chassis, and a crack-filling spray gun is installed at its end. The perception module includes a depth camera for acquiring RGB and depth images of the road surface; A control module is signal-connected to the sensing module, the multi-degree-of-freedom robotic arm, and the crack filling spray gun. The control module includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the steps of the above-mentioned autonomous crack filling and repair method for road cracks. The grouting spray gun includes a spray gun body and an extrusion motor. The rotation speed of the extrusion motor is controlled by the glue dispensing command output by the control module.

[0014] The application of the technical solution of the present invention has the following beneficial effects: This invention uses a two-dimensional semantic mask of the crack as the data source, obtains the physical crack width corresponding to each pixel through Euclidean distance transformation, and then inversely maps the skeleton pixel coordinates with the depth map and camera intrinsic parameters to three-dimensional space, constructing a multi-dimensional state feature point cloud composed of three-dimensional spatial coordinates and physical crack width. This technique can accurately obtain the three-dimensional pose and true physical width of the crack with extremely low computational overhead without reconstructing massive global environment point clouds, significantly reducing the computational burden on industrial control computers, and providing complete and accurate process data for subsequent path planning and variable dispensing.

[0015] This invention maps the physical topology graph into a fully connected topology graph network with crack segments as nodes and spatial movement as edges, allowing virtual ants to move freely between any crack nodes. This fundamentally avoids the problem of local deadlock that traditional ant colony algorithms easily get stuck in non-connected crack networks, achieving complete traversal of all crack branches. At the same time, this invention designs an energy consumption cost function that integrates movement distance, turning angle, and Z-axis lifting work, so that the planning results reflect the actual operating energy consumption of the robotic arm rather than just optimizing the path geometry length. This significantly reduces frequent gun lifting and repeated paths, lowers operating energy consumption, and improves operating efficiency.

[0016] This invention uses the physical width of the crack as the fourth dimension coordinate, and inputs it together with the three-dimensional spatial coordinates into a B-spline kernel function for joint smoothing and fitting, generating a collaborative operation script that synchronizes spatial displacement and adhesive output in time. This script simultaneously drives the robotic arm to sweep along the trajectory and the extrusion motor to inject adhesive according to the crack width variable, achieving adaptive repair with large output for wide cracks and small output for narrow cracks, ensuring the fullness of crack filling and consistency of project quality under all road conditions.

[0017] This invention introduces a force-position hybrid servo mechanism with Z-axis impedance control at the execution layer, constructing a control architecture with rigid XY plane position guidance and flexible Z-axis impedance compliance. This enables heavy-duty robotic arms to achieve high-frequency, collision-free dynamic contact on bumpy roads, eliminating servo system collision damage caused by rigid overload and significantly extending the service life of the end effector.

[0018] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of an embodiment 1 of this application, which describes a method for autonomously filling and repairing road cracks. Figure 2 This is a schematic diagram of crack mask identification and segmentation in an autonomous crack filling and repair method for road cracks according to Embodiment 1 of this application; Figure 3 This is a schematic diagram of crack skeleton line extraction in an autonomous crack filling and repair method for road cracks according to Embodiment 1 of this application; Figure 4 This is a schematic diagram of the crack connectivity analysis encoding in an autonomous crack filling and repair method for road cracks according to Embodiment 1 of this application; Figure 5 This is a physical topology diagram of the cracks in an autonomous crack filling and repair method for roads according to Embodiment 1 of this application, where the edges represent cracks; Figure 6 This is a fully connected topology network in an autonomous crack filling and repair method for road cracks according to Embodiment 1 of this application, where nodes represent cracks, solid lines represent physical connection transitions, and dashed lines represent physical breaks that require Z-axis lifting. Figure 7 This is a schematic diagram of four-dimensional smooth fitting in an autonomous crack filling and repair method for road cracks according to Embodiment 1 of this application. Detailed Implementation

[0020] To facilitate understanding of the present invention, a more comprehensive description is provided below, along with preferred embodiments. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the present invention.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0022] Example 1: See Figures 1-7 This embodiment provides a method for self-sealing and repairing road cracks, including the following steps: S1. Obtain a multi-dimensional state feature point cloud of road surface cracks. The multi-dimensional state feature point cloud includes at least the three-dimensional spatial coordinate information of the crack skeleton line and the crack physical width information corresponding to the location of the crack skeleton line. In step S1, RGB and depth images of the road surface are simultaneously acquired using a depth camera. Instance segmentation, Euclidean distance transformation, morphological thinning, and 3D inverse mapping are then applied to obtain a multidimensional state feature point cloud composed of 3D spatial coordinates and physical seam widths. This process includes the following sub-steps: S11: Acquire RGB images of the road surface and corresponding single-channel depth images using a depth camera. The RGB image is used to identify the planar morphology of cracks, and the depth image is used to provide distance information of each pixel from the camera. To ensure subsequent mapping accuracy, the RGB image and depth image need to be aligned (registered) with pixel coordinates before acquisition.

[0023] In this embodiment, the depth camera can be an RGB-D structured light camera or a ToF camera (such as the Kinect series), which is mounted on the mobile chassis or robotic arm of the crack filling and repair system. During data acquisition, the camera's optical axis should be as perpendicular to the road surface as possible to reduce mapping errors caused by tilting. It is understood that the camera's installation position and orientation can be adjusted according to the operational requirements.

[0024] S12: Input the RGB image into the pre-trained instance segmentation model. After processing, the instance segmentation model outputs a binary mask of the crack region. In the binary mask, the pixel value of the crack region is 1, and the pixel value of the non-crack region is 0.

[0025] In this embodiment, the instance segmentation model can use the YOLOv8 model, and the training samples are manually annotated road surface crack images, covering various forms such as transverse cracks, longitudinal cracks, and crisscross cracks. The segmentation results are as follows: Figure 2 As shown.

[0026] S13: Perform Euclidean distance transform on the binarized mask. For each pixel within the mask region... Calculate the Euclidean distance from it to the nearest mask edge pixel. This distance value represents how close the pixel is to the crack boundary.

[0027] S14: Morphological thinning is performed on the binarized mask to extract the crack skeleton line composed of single pixels. This embodiment uses the Zhang-Suen parallel thinning algorithm, which iterates multiple times to peel away the edge pixels of the mask layer by layer while maintaining the topological connectivity of the object, until only the center line of a single pixel width remains. The crack skeleton line after thinning retains the complete topological structure of the crack, including branch points, endpoints, and each path segment. The crack skeleton line extraction result in this embodiment is as follows: Figure 3 As shown.

[0028] At the location of the crack skeleton, at a distance The value is approximately equal to half the crack width, therefore twice the pixel distance is taken as the pixel value. Corresponding crack physical width characterization value After Euclidean distance transformation, the mask points are transformed from two-dimensional coordinates. Upgrade to three-dimensional attributes ,in Represents pixels The physical width characterization value of the crack, Represents pixels The corresponding horizontal pixel coordinates are pointing to the right. Represents pixels The corresponding vertical pixel coordinate is downwards.

[0029] After extracting the single-pixel crack skeleton line, the crack branches are divided into blocks for encoding by breaking intersections and combining connected component analysis. Specifically, firstly, intersections (i.e., the points where three or more crack skeleton lines meet) are identified in the crack skeleton line, and the crack skeleton line is broken at these intersections. Then, connected component labeling is performed on each broken crack skeleton line segment, and a unique crack segment number is assigned to each connected component. ,like Figure 4 As shown, This is the first crack segment. This is the second crack segment. This is the third crack segment. Simultaneously, the start point, end point, and path point sequences of each crack segment are recorded, and a graph-structured dictionary database, Crack_Database_2D, is established.

[0030] Each crack segment entry records the two-dimensional pixel coordinates and width attributes of the crack segment's start point, end point, and all path points.

[0031] S15: Map the pixel coordinates of the crack skeleton line to the depth image and obtain the depth value corresponding to each pixel coordinate on the crack skeleton line. Then, combined with the intrinsic parameter matrix of the depth camera. Rotation matrix Translation vector The two-dimensional pixel coordinates are converted through a three-dimensional inverse mapping algorithm. Convert to 3D spatial coordinates in camera coordinate system The calculation formula is: in, In the world coordinate system Axis coordinates In the world coordinate system Axis coordinates In the world coordinate system Axis coordinates The camera intrinsic parameter matrix can be obtained in advance using Zhang Zhengyou's calibration method; Let be the rotation matrix from the world coordinate system to the camera coordinate system. This is the translation vector from the world coordinate system to the camera coordinate system, which can be obtained through hand-eye calibration; pixels in a depth image The depth value corresponding to the location; Rotation matrix The transpose of the value obtained in step S13 is used simultaneously to represent the pixel-level seam width. Convert to actual physical width The conversion formula is: in, It's a crack. The actual width in the world coordinate system at this point. The reference point (0,0,0) of the world coordinate system is defined by the engineer, and is generally defined as a point on the robot arm chassis; This is the focal length of the depth camera in the horizontal direction.

[0032] S16: Transfer three-dimensional spatial coordinates With the corresponding actual physical width Correlate them to generate a four-dimensional feature vector for each skeleton point. This forms a multidimensional state feature point cloud, and simultaneously updates the graph structure dictionary database to Crack_Database_3D.

[0033] Thus, with extremely low computational overhead and without the need to reconstruct a massive global environment point cloud, a multidimensional state feature point cloud composed of the three-dimensional spatial coordinates of the crack skeleton line and the actual physical crack width is constructed, providing a complete and accurate data foundation for subsequent path optimization and adaptive process execution.

[0034] S2: Based on multi-dimensional state feature point cloud, construct a fully connected topological graph network with crack segments as nodes and spatial transitions between crack segments as edges, and use ant colony algorithm to perform path optimization on the fully connected topological graph network to output the optimal repair path that traverses all crack segments.

[0035] In step S2, each refined crack segment is defined as a node in the topological graph network; the spatial transfer action of the crack filling spray gun from one crack segment to another is defined as an edge in the topological graph network, thereby constructing a fully connected topological graph network in which there is a connected path between any two nodes. More specifically, step S2 includes the following sub-steps: Step 21: Based on the two-dimensional crack skeleton lines extracted in step S14, construct a physical topology network reflecting the physical connection relationships of the cracks. ,like Figure 5 As shown, the blue lines represent crack segments, and the labels C1, C2, C3, ..., C7 on the blue lines represent the crack segment numbers; the black dots A, B, C, D, ..., I represent the endpoints of crack segments or the intersections of two adjacent crack segments. The vertex set... Includes the endpoints and intersections of the crack, and the set of edges. For each crack segment.

[0036] Step 22: Connect the physical topology graph network Each edge (i.e., each crack segment) in the graph is mapped to a fully connected topological graph network. A node in the set of nodes, that is, a node set. = The spatial transfer action of moving the crack sealing spray gun from one crack segment to another is defined as... edge set in Since spatial transfer occurs between any two crack segments, movement occurs along the crack when they are physically connected, and when they are physically disconnected, movement is achieved by raising the weapon and crossing the crack. There is a connected path between any two nodes in the graph, forming a fully connected topological network, such as... Figure 6 As described, the red dots C1, C2, C3, ..., C7 represent crack segments, the lines represent transfer costs, the solid green lines represent the physical connection between two crack segments, and the dashed gray lines represent the physical break between two crack segments. During repair, the crack filling spray gun needs to be lifted along the Z-axis. This fully connected topology network allows the virtual ants of the ant colony algorithm to freely roam between any crack nodes, fundamentally avoiding the problem of local deadlock in non-connected crack networks (such as mesh cracks and discontinuous cracks) encountered by traditional algorithms.

[0037] To support subsequent cost calculations, the following three feature matrices are constructed based on the multidimensional state feature point cloud: (a) Crack geometric feature matrix: diagonal elements Indicates crack segment The actual physical length (obtained by 3D point cloud computing), off-diagonal elements Indicates crack segment With crack segment The angle between the local tangent vectors at the intersection point; (b) Crack connectivity state matrix: elements It is a Boolean variable. Indicates crack segment With crack segment Physically directly connected This indicates that the devices are physically disconnected. (c) Crack distance matrix: elements Indicates crack segment With crack segment The straight-line distance between them; when physically connected. Take the crack segment The end to the crack section The distance between the starting and ending points (usually 0 or a very small value, negligible); when physically disconnected. Take the straight-line spatial distance between the two crack segments as determined by the transfer strategy.

[0038] The transfer strategy includes two types: crack segment The end point to the crack segment The starting point and crack segment The end point to the crack segment The finish line. The spatial straight-line distance corresponding to each transfer strategy is denoted as . In actual calculations, the comprehensive energy consumption cost of the two transfer strategies is calculated based on the current pose and motion continuity of the robotic arm. The final selection of the transfer strategy is based on minimizing the total comprehensive energy consumption cost, i.e., determining which transfer strategy to use for the crack segment. Is the starting point or the ending point used as the entry point?

[0039] Step 23: Calculate the comprehensive dynamic energy consumption cost function.

[0040] Design a cost function with the goal of minimizing the overall energy consumption cost. Specifically, it means that after the crack section is repaired... Later transferred to the fracture segment The overall energy cost of this action is calculated using the following formula: in, Boolean variables (i.e., those in the aforementioned connected state matrix) Substituting the value into the input (which can be 0 or 1), represents the crack segment. With crack segment Whether they are physically directly connected: when Time indicates physical connectivity, when This indicates that the physical connection is not established. For crack segment The actual physical length is directly calculated from the three-dimensional point cloud obtained in step S16. Indicates crack segment With crack segment Between The spatial linear distance of the transfer strategy, including the crack segment. The end point to the crack segment The starting point and crack segment The end point to the crack segment There are two kinds of endpoints. This represents any one of the transition strategies. The penalty for turning is calculated using the following formula: ,in For crack segment With crack segment The angle between the local tangent vectors at the intersection point. The unit is degrees (°); its physical meaning is: the robotic arm moves from the crack segment With crack segment It is necessary to adjust the end attitude to match the crack segment. The greater the deflection angle, the higher the energy consumption for attitude adjustment. To increase the penalty: when the crack segment With crack segment When the physical connection is broken, the robotic arm must lift the grouting spray gun to perform spatial obstacle crossing, which triggers Z-axis lifting, causing... , The work done by the robotic arm during a fixed lifting motion across the Z-axis is pre-calibrated using the robotic arm's dynamic parameters (mass of each joint, arm length, maximum lifting speed, etc.); when physically connected... . The conversion coefficient is used to characterize the energy consumption per unit distance during the translation of the robotic arm and chassis. The dimension of is joules per meter; The conversion coefficient characterizes the energy consumption per unit deflection angle during robotic arm posture adjustment. The units of measurement are joules per degree. Both are preset constants, and their specific values ​​can be obtained through calibration experiments.

[0041] As can be seen from the above formula, when the crack segment With crack segment Physical connectivity ( When ), the cost is along the crack segment. The energy consumption for translational motion plus the energy consumption for turning attitude adjustment; when physically disconnected ( When this occurs, the cost is the cross-space transfer distance and the crack segment. The translational energy consumption of length plus the lifting work makes the cost function fully cover the three types of dynamic energy consumption factors of distance, rotation angle and gun lifting in the robotic arm repair process.

[0042] Step 24: Solve for the state transition probability.

[0043] In the In the nth iteration, the 1st Only virtual ants are generated from the current crack segment. The endpoint selection is shifted to the candidate fracture segment. transition probability The calculation formula is: in, For the first In the next iteration, from the crack segment To the crack section The concentration of pheromones along the path; The pheromone heuristic factor (typically ranging from [0.5, 5]) is used to control the degree to which the algorithm depends on the historical optimal path; This is the expected heuristic factor (typically ranging from [0.5, 5]), used to control the algorithm's sensitivity to local energy consumption costs; For the first The set of crack nodes that the virtual ant has not yet visited (dynamic tabu list). The summation traversal variable is used to iterate over the set of unvisited crack nodes. Any crack node in the array; Comprehensive energy consumption value The reciprocal of , i.e. the expected heuristic function.

[0044] After calculating the transition probabilities of each candidate node, a roulette wheel algorithm is used to select the next node: the transition probability of each candidate crack node is mapped to the sector area on a virtual roulette wheel; the higher the probability, the larger the sector area and the higher the probability of being selected. This mechanism ensures the algorithm's greediness while retaining a certain degree of random heuristics, preventing the path solving from getting trapped in local optima too quickly.

[0045] Step 25: Global pheromone update.

[0046] To accelerate algorithm convergence and filter suboptimal solutions, after each iteration, the path with the lowest total energy consumption (i.e., the minimum total energy cost) is selected for positive feedback pheromone updates. The update formula is as follows: in, The pheromone evaporation coefficient has a value in the range of (0, 1). For pheromone increment; This is the pheromone intensity constant (a preset constant, typically ranging from [10, 1000]). This is the total energy consumption of the optimal path in this iteration (i.e., the lowest energy consumption value in this iteration).

[0047] As the number of iterations increases, the pheromone concentration on the optimal repair path accumulates until convergence. After the algorithm reaches the maximum number of iterations (a preset value, usually 100 to 500), the optimal path is extracted based on the finally converged global pheromone matrix τ: a deterministic greedy search strategy is adopted, starting from the initial crack node, and at each state transition, the pheromone concentration of a single-step edge is directly selected from the unvisited crack nodes. The largest node is used as the transfer target until all crack nodes are traversed, resulting in the final globally optimal repair operation path. The order in which the crack segments in this path sequence are arranged is the optimal repair path (the repair trajectory of each crack segment is organized in this order in step S3).

[0048] S3: Based on the optimal repair path and multi-dimensional state feature point cloud, generate a collaborative operation script that includes spatial pose sequence, glue dispensing command, and real-time compensation of the crack filling spray gun in the Z-axis direction ∆Z.

[0049] In step S3, the multidimensional state feature point cloud is downsampled and curve-fitted to generate a smooth repair trajectory. The spatial pose of the trajectory, the amount of adhesive applied, and the control parameters required for real-time Z-axis compensation are then compiled into a collaborative operation script. Specifically, this includes the following sub-steps: Step 31: Control point acquisition.

[0050] The crack skeleton line four-dimensional point cloud data Crack_Database_3D obtained in step S1 is downsampled to obtain the front three-dimensional coordinates of the crack skeleton line point cloud. As input, spatial control points of the crack skeleton are extracted using the Douglas-Puk algorithm. The specific operation of this algorithm involves setting a spatial tolerance threshold. (For example, 2mm), connect the first and last points of the crack with a straight line, and calculate the perpendicular distance from all intermediate skeleton points to this straight line; if the maximum perpendicular distance is less than Then delete all intermediate points if the maximum vertical distance is greater than or equal to 1. Then retain the point with the maximum distance, and divide the curve into two segments using this point as the boundary. Recursively perform the above operation on each segment until the perpendicular distance from all remaining points to their corresponding lines is less than 1. This algorithm, while preserving the macroscopic geometry of the crack, significantly eliminates redundant nodes on straight or slightly jagged road sections, retaining only the key extreme points that determine the crack's orientation as control points, such as... Figure 7 As stated above.

[0051] Step 32: Smooth curve fitting and trajectory point sequence generation.

[0052] The four-dimensional coordinates of the spatial control points obtained after downsampling The B-spline kernel function is used for fitting to obtain a smooth, frizz-free repair trajectory. The first three dimensions are... Controlling the spatial trajectory of the crack sealing spray gun, fourth-dimensional coordinates This provides information on the physical width of the crack. B-spline fitting offers local control (modifying a control point only affects the shape of the local curve) and high-order continuous smoothness, ensuring continuous acceleration of the robotic arm and avoiding sudden stop impacts.

[0053] By using the physical width of the crack as a generalized fourth-dimensional coordinate and inputting it into the B-spline kernel function for joint smoothing, the glue output curve is simultaneously smoothed while generating a smooth spatial trajectory. Since both share the same parameter domain (arc length parameter), the spatial displacement command and the glue output command are naturally synchronized in time, fundamentally avoiding the glue accumulation or glue breakage problems caused by the delayed adjustment of flow rate after spatial movement in traditional solutions. At the same time, the physical width information of the crack at each point on the repair trajectory line is compiled into the corresponding glue output control parameter (e.g., compiled into an analog quantity of extrusion motor speed based on the positive mapping relationship between crack width and extrusion motor speed).

[0054] Step 33: Trajectory segmentation and script compilation.

[0055] Adaptive segmentation of the smooth trajectory line is performed based on spatial distance and curvature: a smaller segmentation step size is used in areas with high curvature (sharp bends in the crack) to ensure the tracking accuracy of the robotic arm, while a larger segmentation step size is used in areas with low curvature (straight sections of the crack) to reduce data volume and improve execution efficiency. Combining the optimal repair path determined in step S2, the segmented trajectory points of each crack segment are sequentially concatenated to generate a complete repair trajectory sequence. The spatial distance is the distance between two points in space. In the smooth trajectory line fitted by the B-spline, a smaller segmentation step size is selected in areas with high curvature, i.e., a shorter spatial interval is used to select the segmented trajectory points; in areas with low curvature, a larger segmentation step size is selected, i.e., a longer spatial interval is used to select the segmented trajectory points. For example, in areas with high curvature, a segmented trajectory point is selected every 3 mm; in areas with low curvature, a segmented trajectory point is selected every 20 mm.

[0056] The processor of the control module will process the spatial location information in the trajectory sequence. The physical width information (already compiled into the corresponding dispensing control parameters) and the gun lifting requirements (whether the Z-axis needs to cross between the slit segments) are packaged, solved, compiled into a pose-flow strongly coupled offline operation script, and sent to the underlying controller. This operation script is a sequence of timing instructions, each instruction containing: the target spatial pose (X, Y, Z coordinates and the angles of each joint of the robotic arm), and the analog value of the extrusion motor speed (derived from the slit width at that point). The script includes a mapping result and a status flag (indicating whether to perform Z-axis lifting or lowering). It also includes a real-time Z-axis compensation command segment, carrying the desired contact force and impedance control parameters. This allows the underlying controller to calculate the compensation amount ΔZ of the grouting gun in the Z-axis direction in real time according to the impedance control strategy during execution and superimpose it onto the current Z-axis coordinate. The script is sent to the underlying controller via a fieldbus (e.g., CAN bus, EtherCAT bus).

[0057] In a preferred embodiment of this invention, the underlying controller parses the collaborative work script line by line, drives the multi-degree-of-freedom robotic arm to sweep and repair the cracks along the crack trajectory with the crack-sealing spray gun, and compensates for the position of the spray gun in the Z-axis direction in real time according to the contact force between the crack-sealing spray gun and the road surface during the repair process, thereby achieving flexible compliant repair with constant force in the Z-axis direction. Specifically: The system uses a force sensor installed at the end of the robotic arm to detect the actual contact force between the crack sealing spray gun and the road surface in real time. and the preset desired contact force Compare and calculate the contact force deviation. Based on the contact force deviation between the crack sealing spray gun and the road surface An impedance control strategy is used to calculate The impedance control model adopts a second-order dynamic system of mass-spring-damped system, and its differential equation is: in, The desired inertial mass parameter of the system. The desired damping parameters of the system, These are the system's desired stiffness parameters, and all three are preset control parameters, pre-tuned based on the end-effector mass, desired response speed, etc. This represents the displacement compensation amount of the spray gun in the Z-axis direction. The first derivative of ΔZ with respect to time (the velocity of the grouting gun in the Z-axis direction). The second derivative of ΔZ with respect to time (the acceleration of the grouting gun in the Z-axis direction). This is the deviation between the actual contact force and the expected contact force of the crack sealing spray gun.

[0058] Its physical meaning is: when a road surface protrusion causes an increase in contact force ( )hour, For positive, the spray gun moves upwards and backs away; when the road surface is concave, causing a decrease in contact force ( )hour, If the value is negative, the spray gun moves downwards to maintain a constant desired contact force with the road surface. The calculated value will then be used to... Superimposed on the Z-axis coordinate of the current spatial pose command in the collaborative operation script, the robotic arm is driven to perform flexible compliant motion in the Z-axis direction, constructing a force-position hybrid control architecture of rigid position guidance in the XY plane and flexible impedance compliance in the Z-axis: in the horizontal direction, it strictly follows the script trajectory to ensure glue injection accuracy, and in the vertical direction, it flexibly adapts to road undulations to avoid hard collisions and glue splashing, protecting the end of the robotic arm from damage due to collisions.

[0059] During the sweeping repair, the extrusion motor speed is controlled by the glue output command in the script: for larger crack widths ( A trajectory point with a large value corresponds to a higher extrusion speed, achieving a wide slit and large displacement; a smaller slit width ( A smaller value corresponds to a lower extrusion speed, enabling small-volume, narrow-slit extrusion. The dispensing quantity command and spatial displacement command are arranged in the same sequence within the same script, achieving synchronized execution of the robotic arm's sweeping motion and variable dispensing.

[0060] As a further preferred embodiment of this example, after step S3 is completed, step S4 is also included: an evaluation step, which calculates the repair coverage rate of the repaired crack. When the repair coverage rate is greater than or equal to a preset threshold, the repair is deemed qualified; otherwise, a supplementary repair process is triggered.

[0061] After completing the crack sealing and repair of all crack segments, RGB and depth images of the repaired pavement are acquired again using a depth camera. The repaired area is segmented to extract the actual coverage area of ​​the repair adhesive. The ratio of the overlap area between the actual coverage area and the original crack mask area extracted in step S1 is calculated; this ratio is the repair coverage rate. When the repair coverage rate is greater than or equal to a preset threshold (preferably 80%), the system determines that the repair is qualified and ends the operation. When the repair coverage rate is less than the preset threshold, the system determines that the repair is unqualified and automatically triggers a supplementary repair process, i.e., re-executes steps S1 to S3 for the uncovered areas until the repair coverage rate meets the requirements or the maximum number of retries is reached. This evaluation step forms a complete closed-loop control of perception, decision-making, execution, and evaluation, ensuring the quality controllability and traceability of the crack sealing operation.

[0062] Example 2: This embodiment provides an autonomous road crack filling and repair system, which includes a mobile chassis, a multi-degree-of-freedom robotic arm, a sensing module, and a control module.

[0063] The mobile chassis supports the system components and enables movement on the road surface. It preferably employs an Ackermann steering structure, providing excellent straight-line stability and steering agility. A power supply unit is mounted on the mobile chassis to provide power to the entire system.

[0064] A multi-degree-of-freedom robotic arm is mounted on a mobile chassis, with a crack-filling spray gun installed at its end. The robotic arm is preferably a six-degree-of-freedom articulated industrial robotic arm, possessing omnidirectional spatial attitude adjustment capabilities, enabling it to move along any trajectory in three-dimensional space. The crack-filling spray gun includes a spray gun body and an extrusion motor. The extrusion motor is used to extrude sealant from the spray gun body and inject it into the cracks. The speed of the extrusion motor is controlled by the sealant dispensing command output by the control module to achieve variable sealant injection with a large dispensing volume for wide cracks and a small dispensing volume for narrow cracks.

[0065] The perception module includes a depth camera, mounted on a mobile chassis or robotic arm, used to simultaneously acquire RGB and depth images of the road surface. The RGB and depth images correspond one-to-one in pixel coordinates. The RGB images are used in the instance segmentation model to identify the planar morphology of cracks, while the depth images provide physical distance information of each pixel from the camera, providing depth data for subsequent 3D inverse mapping.

[0066] The control module is signal-connected to the sensing module, the multi-degree-of-freedom robotic arm, and the crack filling spray gun. The control module includes a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the steps of the autonomous road crack filling repair method described in Example 1. Specifically, when the processor executes the computer program, it performs the following functions: acquiring a multi-dimensional state feature point cloud of the road surface cracks, which at least includes the three-dimensional spatial coordinate information of the crack skeleton line and the crack physical width information corresponding to the position of the crack skeleton line; based on the multi-dimensional state feature point cloud, constructing a fully connected topological graph network with crack segments as nodes and spatial transitions between crack segments as edges, and using an ant colony algorithm to perform path optimization on the fully connected topological graph network, outputting the optimal repair path traversing all crack segments; and generating a collaborative operation script containing spatial pose commands and glue dispensing commands based on the optimal repair path and the multi-dimensional state feature point cloud, and sending it to the underlying controllers of the multi-degree-of-freedom robotic arm and the crack filling spray gun.

[0067] More specifically, when the processor of the control module executes the computer program, it acquires RGB and depth images of the road surface through the perception module, performs instance segmentation on the RGB image to obtain a binary mask of the crack region, performs Euclidean distance transformation on the binary mask to obtain the distance from each pixel in the mask region to the edge of the mask as the physical width representation value of the crack, performs morphological thinning processing on the binary mask to extract the crack skeleton line composed of single pixels, maps the pixel coordinates of the crack skeleton line to the depth image and performs 3D inverse mapping in combination with camera intrinsic parameters to generate the 3D spatial coordinates of the crack skeleton line, and associates the crack physical width representation value corresponding to each pixel of the crack skeleton line with the 3D spatial coordinates to generate a multi-dimensional state feature point cloud.

[0068] The processor of the control module also executes a computer program to: define each refined crack segment as a node in a topological graph network; define the spatial transfer action of the crack filling gun from one crack segment to another as an edge in the topological graph network; construct a fully connected topological graph network in which there is a path between any two nodes; and calculate the distance from the crack segment... Transfer to the fracture segment The comprehensive energy consumption cost is calculated, and the ant colony algorithm is used for path optimization to output the optimal repair path that traverses all crack segments. The calculation formula for the comprehensive energy consumption cost, the calculation method for the state transition probability, and the pheromone update method are the same as in Example 1, and will not be repeated here.

[0069] The processor of the control module also executes computer programs to: downsample the multidimensional state feature point cloud, extract the spatial position control points of the crack skeleton line using the Douglas-Puk algorithm; use the three-dimensional coordinates of the spatial position control points and the corresponding crack physical width information as four-dimensional input data, perform B-spline curve fitting to obtain a smooth repair trajectory line, and compile the crack physical width information into the corresponding glue dispensing control parameters; segment the smooth repair trajectory line based on spatial distance and curvature, and combine the optimal repair path to generate a collaborative operation script containing a spatial pose sequence and corresponding glue dispensing instructions.

[0070] The underlying controller of the control module receives and parses the collaborative operation script, driving the multi-degree-of-freedom robotic arm to sweep and repair the cracks along the crack trajectory with the crack-sealing spray gun. During execution, the control module acquires the contact force between the crack-sealing spray gun and the road surface in real time through a force sensor installed at the end of the robotic arm, and compensates for the position of the spray gun in the Z-axis direction in real time based on an impedance control strategy. The calculation method of the Z-axis compensation amount ∆Z is the same as in Example 1. At the same time, the control module adjusts the speed of the extrusion motor according to the glue dispensing command to achieve variable glue injection with a large dispensing volume for wide cracks and a small dispensing volume for narrow cracks.

[0071] As another preferred embodiment of the present invention, when the processor of the control module executes the computer program, it also performs an evaluation step after the crack filling and repair is completed: the repair coverage rate of the crack after the repair is completed is calculated, and the repair is deemed qualified when the repair coverage rate is greater than or equal to a preset threshold (preferably 80%); otherwise, a supplementary repair process is triggered, forming a complete closed loop of perception, decision-making, execution and evaluation.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for self-sealing and repairing road cracks, characterized in that, Includes the following steps: S1: Obtain a multi-dimensional state feature point cloud of road surface cracks, the multi-dimensional state feature point cloud including the three-dimensional spatial coordinate information of the crack skeleton line and the physical width information of the crack. S2: Based on the multi-dimensional state feature point cloud, construct a fully connected topological graph network with crack segments as nodes and spatial transitions between crack segments as edges, and use the ant colony algorithm to perform path optimization on the fully connected topological graph network to output the optimal repair path that traverses all crack segments. S3: Based on the optimal repair path and the multi-dimensional state feature point cloud, generate a grouting spray gun containing a spatial pose sequence, glue dispensing command, and real-time compensation. Axial compensation Collaborative work scripts.

2. The self-sealing and repair method for road cracks according to claim 1, characterized in that, The specific methods for obtaining the three-dimensional spatial coordinate information of the crack skeleton line and the physical width information of the crack in step S1 include: S11: Acquire RGB and depth images of the road surface; S12: Perform instance segmentation on the RGB image to obtain a binary mask of the crack region; S13: Perform Euclidean distance transform on the binarized mask to obtain the pixels within the mask region. Distance to the edge of the mask ; S14: Perform morphological thinning on the binarized mask, extract the crack skeleton line composed of single pixels, and take the pixels on the crack skeleton line. Distance to the edge of the mask Double the value as a pixel The corresponding physical width characterization value of the crack; S15: Map the pixel coordinates of the crack skeleton line to the depth image, and perform a three-dimensional inverse mapping in combination with the camera intrinsic parameters to generate the three-dimensional spatial coordinates of the crack skeleton line. The physical width characterization value of the crack is mapped to the depth image, and three-dimensional inverse mapping is performed in combination with camera intrinsic parameters to generate the true crack width value of the crack skeleton line. S16: Associate the physical width value of the crack corresponding to each pixel of the crack skeleton line with the three-dimensional spatial coordinates to generate the multi-dimensional state feature point cloud.

3. The self-sealing and repair method for road cracks according to claim 2, characterized in that, The construction of the fully connected topology graph network in step S2 specifically includes: Each refined crack segment is defined as a node in the topological graph network; The spatial transfer action of moving the crack sealing spray gun from one crack segment to another is defined as the edge of the topological graph network, thereby constructing a fully connected topological graph network in which there is a connected path between any two nodes.

4. The self-sealing and repair method for road cracks according to claim 3, characterized in that, In step S2, the ant colony algorithm is used to generate the optimal repair path with the objective of minimizing the overall energy consumption cost; where the overall energy consumption cost is expressed as: in, To the crack segment Transfer to the fracture segment The comprehensive energy consumption value; A Boolean variable representing a crack segment With crack segment Whether they are physically directly connected; when Time indicates physical connectivity, when This indicates that the connection is not physical; Not equal to , Indicates except Any crack that has never been there before; For crack segment The actual physical length; Indicates crack segment With crack segment Between The spatial linear distance of a transfer strategy, the transfer strategy including crack segments The end point to the crack segment The starting point and crack segment The end point to the crack segment There are two kinds of endpoints. This represents any one of the transition strategies; The penalty for turning , For crack segment With crack segment The angle between the local tangent vectors at the intersection point; To increase the cost of punishment, when the crack segment With crack segment When physically disconnected, When physically connected ; For the robotic arm to perform one Fixed lifting and lowering work performed by the axis crossing motion; To characterize the energy consumption per unit distance during the translation of the robotic arm and chassis, This is a conversion coefficient characterizing the energy consumption per unit deflection angle during robotic arm posture adjustment.

5. The self-sealing and repair method for road cracks according to claim 4, characterized in that, In step S2, when using the ant colony algorithm for path optimization, the first... Only virtual ants are generated from the current crack segment. The endpoint selection is shifted to the candidate fracture segment. transition probability Represented as: in, For the first In the next iteration, from the crack segment To the crack section The concentration of pheromones along the path, As a pheromone-inspired factor, As the expected heuristic factor, For the first The set of crack nodes that the virtual ants have not yet visited; Comprehensive energy consumption value The reciprocal of; The summation traversal variable is used to iterate over the set of unvisited crack nodes. Any crack node in the array.

6. The self-sealing and repair method for road cracks according to claim 5, characterized in that, After each iteration, the path with the lowest overall energy consumption is selected for positive feedback updating of pheromone concentration. The update formula is: in, The pheromone evaporation coefficient, For pheromone increment, Where is the pheromone intensity constant. This represents the comprehensive energy consumption value of the optimal path in this iteration.

7. The self-sealing and repair method for road cracks according to claim 1, characterized in that, The specific method for generating the collaborative task script in step S3 includes: Downsampling is performed on the multidimensional state feature point cloud to extract the spatial position control points of the crack skeleton line; The three-dimensional coordinates of the spatial location control points and their corresponding crack physical width information are used as four-dimensional input data. B-spline curve fitting is performed to obtain a smooth repair trajectory line. At the same time, the crack physical width information of each point on the repair trajectory line is compiled into glue output control parameters. Based on the spatial distance and curvature of the repair trajectory line, the smooth repair trajectory line is segmented. Combined with the optimal repair path, a system is generated that includes a spatial pose sequence, adhesive dispensing instructions, and a real-time compensation grouting spray gun. Compensation amount in the axial direction Collaborative work scripts.

8. The self-sealing and repair method for road cracks according to claim 1, characterized in that, In step S3, the real-time compensation grouting spray gun is in The position in the axial direction adopts an impedance control strategy, based on the contact force deviation between the crack sealing spray gun and the road surface. An impedance control strategy is used to calculate ,That Shaft compensation The following relationship must be satisfied: in, For inertial mass, For damping parameters, For stiffness parameters, This is the deviation between the actual contact force and the expected contact force of the crack sealing spray gun. To compensate for the crack sealing spray gun in Acceleration in the axial direction; To compensate for the crack sealing spray gun in Velocity in the axial direction; To compensate for the crack sealing spray gun in Displacement in the axial direction.

9. The self-sealing and repair method for road cracks according to claim 1, characterized in that, After performing step S3, step S4 is also included: S4: Evaluation step. Calculate the repair coverage rate of the repaired cracks. If the repair coverage rate is greater than or equal to a preset threshold, the repair is deemed qualified; otherwise, a supplementary repair process is triggered.

10. A self-contained crack sealing and repair system for roads, characterized in that, include: Mobile chassis; A multi-degree-of-freedom robotic arm is mounted on the mobile chassis, and a crack-filling spray gun is installed at its end. The perception module includes a depth camera for acquiring RGB and depth images of the road surface; A control module is signal-connected to the sensing module, the multi-degree-of-freedom robotic arm, and the crack filling spray gun. The control module includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the steps of the road crack autonomous crack filling and repair method as described in any one of claims 1 to 9. The grouting spray gun includes a spray gun body and an extrusion motor. The rotation speed of the extrusion motor is controlled by the glue dispensing command output by the control module.

Images