Road crack repair trajectory end-to-end generation method and system based on two-dimensional vision

Abstract

The application discloses a kind of based on two-dimensional vision's road crack repair trajectory end-to-end generation method and system, belong to road intelligent maintenance technical field.The based on two-dimensional vision's road crack repair trajectory end-to-end generation method includes anti-interference crack segmentation based on frequency domain analysis, topology extraction and deconstruction of crack skeleton, global path connection based on graph optimization and space mapping and end-to-end instruction generation.Skeleton extraction, topology breaking and graph optimization connection strategy are adopted, effectively solve the path ambiguity and self-intersection problem faced by traditional method when processing cross or net crack.The application solves the problems existing in the prior art, such as low segmentation accuracy of visual perception under complex lighting conditions such as tree shadow and water stain;lack of effective constraints for complex cracks with cross and branch, leading to chaotic path planning;and lack of timing coordination in visual recognition and execution control link.

Description

Technical Field

[0001] This invention relates to a method and system for repairing road cracks, specifically to an end-to-end method for generating road crack repair trajectories based on two-dimensional vision and a road crack repair system, belonging to the field of intelligent road maintenance technology. Background Technology

[0002] Road crack repair is a core business of highway maintenance. Addressing the issues of low efficiency and poor quality associated with traditional manual repair, automated repair technology based on machine vision has gradually become a research hotspot.

[0003] At the visual perception level, existing deep learning segmentation algorithms can identify crack regions quite well. However, how to convert pixel-level segmentation results into motion commands that can be executed by the robot remains a key bottleneck that has not yet been solved. Existing path generation methods based on convolutional neural networks and region coverage, as well as rule-based path generation methods, mainly suffer from the following drawbacks: First, they lack topological constraints, and when faced with complex cracks such as bifurcations and meshes, the generated paths are prone to jumping, self-intersection, or omissions; second, they lack geometric smoothness, and the generated trajectory points cause jitter in the robotic arm's motion, making it difficult to meet the requirements of the potting process; third, perception and control are decoupled, with the vision system only outputting image coordinates and lacking a linkage design with world coordinate mapping and end-effector material discharge timing.

[0004] Therefore, the existing technology lacks an end-to-end method that can directly use two-dimensional images as input to continuously complete crack topology deconstruction, global path optimization, spatial mapping and control parameter generation in a single process, which to some extent restricts the engineering application of automated repair systems. Summary of the Invention

[0005] This invention addresses the problems in existing automatic road crack repair technologies, such as low segmentation accuracy of visual perception under complex lighting conditions including tree shadows and water stains; lack of effective constraints for complex intersecting and branching cracks, leading to chaotic path planning; and lack of temporal coordination between visual recognition and execution control. Therefore, it proposes an end-to-end method and system for generating road crack repair trajectories based on two-dimensional vision.

[0006] The technical solution adopted by the present invention to solve the above problems is as follows: An end-to-end method for generating road crack repair trajectories based on two-dimensional vision includes the following steps: Step 1: Anti-interference crack segmentation based on frequency domain analysis; acquire images of the road surface to be repaired, and denoise the acquired images of the road surface to be repaired through a pre-set deep learning network to obtain a high-precision crack binarization mask image; Step 2: Topological extraction and deconstruction of the crack skeleton; First, the morphological thinning process is performed on the crack binarized mask image obtained in Step 1 to obtain a crack skeleton with a single pixel width. Then, cross ambiguity is eliminated through topological analysis to obtain a discrete set of simple line segments. Step 3: Global path connection based on graph optimization; Construct a fully connected graph model, and plan the globally optimal traversal path among the discrete simple line segment sets obtained in Step 2 using a composite cost function to form an ordered pixel-level trajectory sequence; Step 4: Spatial mapping and end-to-end command generation; The ordered pixel-level trajectory sequence formed in Step 3 is mapped to a spatial motion trajectory in the physical world coordinate system. The spatial motion trajectory is subjected to velocity planning and smoothing. Based on the planned velocity distribution and the response delay characteristics of the grouting actuator, a time-domain compensated control signal sequence is generated. According to the spatial motion trajectory, the planned velocity, and the control signal sequence, an end-to-end repair operation control command flow is generated.

[0007] Furthermore, the pre-set deep learning network mentioned in step 1 is a pre-set deep learning segmentation network based on the ResUNet architecture, wherein the segmentation network integrates an improved Discrete Cosine Transform (DCT) spectral analysis module in the feature extraction layer of the encoder.

[0008] Furthermore, the improved Discrete Cosine Transform (DCT) spectral analysis module operates as follows: First, the input intermediate layer feature map is divided into local blocks, and a two-dimensional DCT transformation is performed on each local block to obtain frequency domain coefficients; then, a learnable frequency domain gating mask is introduced and multiplied element-wise with the frequency domain coefficients to attenuate low-frequency coefficients and retain high-frequency coefficients; finally, the spatial domain features are recovered through Inverse Discrete Cosine Transform (IDCT) and residually connected with the original input features.

[0009] Furthermore, the topology analysis described in step 2 specifically involves: first, traversing the neighborhood connectivity of the skeleton pixels, identifying the endpoints and multi-way intersecting branch points in the skeleton, then implementing a topology breaking strategy to remove all identified branch points and their neighboring pixels from the skeleton, and finally forcibly decomposing the complex mesh skeleton into several independent sets of simple connected domain line segments without branches.

[0010] Furthermore, the topology breaking strategy specifically involves: calculating the number of 8-neighbor connected pixels for each pixel on the skeleton; defining a point with 1 8-neighbor connected pixel as an endpoint and a point with more than 2 8-neighbor connected pixels as a branch point; deleting the branch point and the pixels within a preset range around it, so that the intersection structure is broken into unconnected line segments.

[0011] Furthermore, the fully connected graph model described in step 3 is a weighted undirected graph model constructed between line segment endpoints based on the distance and direction constraints between endpoints, with the line segment endpoints as nodes and the connection relationships between line segment endpoints as edges.

[0012] Furthermore, the formula for calculating the composite cost function is as follows:

[0013] in, This is the Euclidean distance between the end of the current path and the endpoint of the line segment to be connected. Let be the angle between the direction of the connecting vector and the tangent direction of the endpoint of the line segment to be connected. and These are the weighting coefficients.

[0014] Furthermore, the time-domain compensation in step 4 specifically involves: obtaining the response delay parameter of the crack sealing actuator; based on the response delay parameter, generating an opening command at the moment before the planned spatial motion trajectory reaches the crack operation start point, and generating a closing command at the moment after leaving the crack operation end point.

[0015] Furthermore, the end-to-end repair operation control command flow includes spatial pose, timestamp, IO status, and flow control parameters.

[0016] A road crack repair system, comprising: A monocular vision acquisition unit is used to acquire two-dimensional digital images of the road surface to be repaired; The central processing unit includes an anti-interference segmentation module, a topology planning module, and a control generation module, which are used to apply the two-dimensional vision-based end-to-end road crack repair trajectory generation method to generate an end-to-end control command flow. The execution unit includes a robotic arm base and a multi-degree-of-freedom robotic arm. The multi-degree-of-freedom robotic arm is mounted on the robotic arm base, and a crack filling actuator is installed at the end of the multi-degree-of-freedom robotic arm for receiving control command streams from the central processing unit to perform road crack repair operations.

[0017] The beneficial effects of this invention are: 1. This invention integrates an improved DCT spectral analysis module into the ResUNet network, and utilizes the frequency domain distribution differences between cracks and shadows to effectively filter out low-frequency environmental noise such as shadows and water stains from the algorithm's underlying layer, significantly improving the robustness and accuracy of crack identification under complex lighting conditions.

[0018] 2. This invention employs a strategy of skeleton extraction, topology breaking, and graph optimization connection, which effectively solves the path ambiguity and self-intersection problems faced by traditional methods when dealing with intersections or mesh cracks, ensuring that the generated robot motion trajectory is geometrically smooth and continuous and logically uniquely traversed.

[0019] 3. This invention abandons the expensive multi-sensor fusion scheme and realizes end-to-end generation of control commands containing time-series parameters from image perception based on monocular vision. It eliminates the technical gap between visual recognition and execution control, and significantly improves the engineering availability and operation efficiency of the automated road crack repair system while reducing hardware costs and computing power requirements. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating one implementation of the road crack repair trajectory end-to-end generation method based on two-dimensional vision according to the present invention; Figure 2 This is a schematic diagram of the ResUNet network structure for introducing the DCT spectral analysis module in this invention; Figure 3 This is a schematic diagram of the crack skeleton extraction and topology breaking process of the present invention (including a comparison before and after branch point removal). Figure 4 This is a schematic diagram of line segment connection and path generation based on graph optimization according to the present invention. Detailed Implementation

[0021] In the description of this invention, it should be noted that all directional indications (e.g., front, back, etc.) are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0022] Specific implementation method one: Combining Figure 1-4 This implementation method is described as follows: Figure 1 As shown in the figure, the end-to-end generation method for road crack repair trajectory based on two-dimensional vision described in this embodiment relates to a method for generating end-to-end road crack repair trajectories based on two-dimensional digital image processing, capable of directly generating robot motion commands containing spatial pose and temporal parameters. The method includes the following steps: Step 1: Anti-interference crack segmentation based on frequency domain analysis; acquire images of the road surface to be repaired. The acquisition method can be to use an on-board industrial control computer to receive the road surface images acquired by a monocular camera to obtain a two-dimensional digital image of the road surface to be repaired.

[0023] The acquired images of the road surface to be repaired are subjected to DCT frequency domain transformation and low-frequency suppression through a pre-set deep learning network. After noise reduction (filtering out interference such as shadows and water stains), a high-precision crack binarization mask image is obtained.

[0024] The pre-built deep learning network is a pre-built deep learning segmentation network based on the ResUNet architecture. This segmentation network integrates an improved Discrete Cosine Transform (DCT) spectral analysis module in the encoder's feature extraction layer. This module transforms the feature map to the frequency domain and uses a gating mechanism to suppress low-frequency components representing illumination changes and enhance high-frequency components representing crack texture. The decoder then outputs a denoised crack binarized mask image. The improved DCT spectral analysis module works as follows: first, the input intermediate layer feature map is divided into local blocks; a two-dimensional DCT transformation is performed on each local block to obtain frequency domain coefficients; then, a learnable frequency domain gating mask is introduced and multiplied element-wise with the frequency domain coefficients to attenuate low-frequency coefficients and retain high-frequency coefficients; finally, the spatial domain features are recovered through Inverse Discrete Cosine Transform (IDCT) and residually connected to the original input features. By integrating an improved DCT spectral analysis module into the ResUNet network, the algorithm effectively filters out low-frequency environmental noise such as shadows and water stains by utilizing the frequency domain distribution differences between cracks and shadows, significantly improving the robustness and accuracy of crack identification under complex lighting conditions.

[0025] The frequency domain analysis-based anti-interference crack segmentation specifically refers to: Step 1.1, Image Acquisition and Preprocessing: Control the monocular industrial camera (mounted at the rear of the mobile platform) to vertically photograph the road surface, acquiring an image with a resolution of [resolution missing]. The two-dimensional RGB digital image is obtained and normalized.

[0026] Step 1.2, Frequency Domain Feature Transformation: The preprocessed image is input into a segmentation network based on the ResUNet architecture. During the deep feature extraction stage of the encoder, the network divides the feature map into predefined sizes. (For example The spatial domain features are divided into local blocks, and a two-dimensional discrete cosine transform (DCT) is performed on each local block to transform the spatial domain features. Convert to frequency domain coefficients .

[0027] Step 1.3, Frequency Domain Gated Filtering: A learnable frequency domain gating mechanism is introduced to weight the frequency domain coefficients, suppressing low-frequency components representing illumination changes and enhancing high-frequency components representing crack texture. The filtering process is shown in the following equation:

[0028] in, For frequency domain coordinates, For the enhanced frequency domain features; The gating mask weights are learned through backpropagation, and their values ​​approach 0 in the low-frequency region and approach 1 in the high-frequency region.

[0029] Step 1.4, Mask Generation: The processed frequency domain features are restored to spatial domain features using Inverse Discrete Cosine Transform (IDCT), and then reconstructed by a decoder and classified using Softmax to output a denoised binary mask image of the cracks. .

[0030] Step 2: Topological extraction and deconstruction of the crack skeleton; First, the morphological thinning process is performed on the crack binarized mask image obtained in Step 1 to transform the pixel-level planar mask into a single-pixel-width skeleton, thus obtaining a single-pixel-width crack skeleton; Then, cross-ambiguity is eliminated through topological analysis to obtain a discrete set of simple line segments.

[0031] The topology analysis specifically involves: first, traversing the neighborhood connectivity of the skeleton pixels to identify the endpoints and multi-way intersections in the skeleton; then, implementing a topology breaking strategy to remove all identified branch points and their neighboring pixels from the skeleton; and finally, forcibly decomposing the complex mesh skeleton into several independent sets of simple connected domain line segments without branches.

[0032] The topology disruption strategy is as follows: calculate the number of 8-neighbor connected pixels (8 - number of neighboring pixels) of each pixel on the skeleton, define the point with 1 8-neighbor connected pixel as the endpoint, and the point with more than 2 8-neighbor connected pixels as the branch point; delete the branch point and the pixels within a preset range around it, so that the originally connected intersection structure is broken into unconnected line segments, eliminating the ambiguity of the path direction.

[0033] The topological extraction and deconstruction of the crack skeleton are specifically as follows: Step 2.1, Skeleton Refinement: A fast parallel refinement algorithm is used to refine the mask image. Perform iterative erosion until all crack areas shrink to a centerline skeleton of a single pixel width. .

[0034] Step 2.2, Key Point Recognition: Traversing the Skeleton Each pixel on Calculate the number of connected pixels in its 8-neighborhood. :

[0035] in, The value is the neighboring pixel value (0 or 1). Based on... The value is used to classify pixels: if Marked as an endpoint; if Mark them as waypoints; if Mark them as branch points.

[0036] Step 2.3: Implement the topology breaking strategy to physically remove all identified branch points and their surrounding pixels within a preset range from the skeleton.

[0037] Step 2.4, Generation of line segment sets: After the breaking operation, the originally complex mesh skeleton is decomposed into several independent connected components, forming simple line segment sets. Each line segment None of them have a branching structure.

[0038] Step 3: Global path connection based on graph optimization; Construct a fully connected graph model. Based on the distance and direction constraints between endpoints, construct a weighted undirected graph model between the endpoints of the line segments, with the endpoints of the line segments as nodes and the connection relationships between the endpoints as edges; Plan the globally optimal traversal path between the discrete simple line segment sets obtained in Step 2 using a composite cost function, forming an ordered pixel-level trajectory sequence. Specifically, based on a composite cost function that includes distance constraints and direction consistency constraints, a graph search algorithm is used to plan one or more globally optimal traversal paths between the discrete line segment sets, generating an ordered pixel-level trajectory sequence that includes work path segments and empty travel path segments.

[0039] The formula for calculating the composite cost function Cost is as follows:

[0040] in, This is the Euclidean distance between the end of the current path and the endpoint of the line segment to be connected. Let be the angle between the direction of the connecting vector and the tangent direction of the endpoint of the line segment to be connected. and is the weighting coefficient; this cost function is used to prioritize connecting line segments that are close in distance and have a smooth direction.

[0041] The graph-optimized global path connection specifically refers to: Step 3.1, Graph Model Construction: Extracting Sets Find the endpoints of all line segments in the graph, treat them as nodes of the graph, construct candidate edges between any two nodes, and calculate the connection cost.

[0042] Step 3.2, Cost Function Calculation: Calculate the cost function for any two endpoints. Connection cost between The calculation formula is as follows:

[0043] In the formula, The Euclidean distance between the two endpoints; For connection vectors The angle between the line segment and the direction of the tangent at the endpoint of the line segment; These are the weighting coefficients. This cost function tends to connect line segments that are close in distance and have a smooth direction.

[0044] Step 3.3, Path Search and Classification: Search for paths based on a greedy strategy or the minimum spanning tree algorithm. If the minimum cost between two points... If the value is less than the set repair threshold, it is determined to be a breakpoint repair, and an interpolation job path is generated (Flag=1); if If the value exceeds the repair threshold, it is determined to be a regional transfer, and an empty travel path for lifting the gun is generated (i.e., the unloaded movement path when the end sealant actuator is closed, Flag=0).

[0045] Step 3.4, Sequence Generation: Piece together all line segments and the generated connection paths in the order of access to form an ordered pixel-level trajectory sequence.

[0046] Step 4: Spatial Mapping and End-to-End Command Generation; The ordered pixel-level trajectory sequence formed in Step 3 is mapped to a spatial motion trajectory in the physical world coordinate system. Velocity planning and smoothing are performed on the spatial motion trajectory. Based on the planned velocity distribution and the response delay characteristics of the crack filling actuator (such as a crack filling gun), a time-domain compensated control signal sequence is generated. According to the spatial motion trajectory, planned velocity, and control signal sequence, an end-to-end repair operation control command stream is generated. The end-to-end repair operation control command stream includes spatial pose, timestamp, I / O status, and flow control parameters.

[0047] Taking a conventional crack-filling actuator structure as an example, the conventional crack-filling actuator generally includes a robotic arm base and a robotic arm. The robotic arm is mounted on the robotic arm base, and a crack-filling actuator is installed at the end of the robotic arm. Based on the pre-calibrated monocular camera planar homography matrix, the pixel-level trajectory sequence is mapped to a three-dimensional spatial motion trajectory in the coordinate system of the robotic arm base. The three-dimensional trajectory points are smoothed (e.g., B-spline interpolation and trapezoidal velocity curve planning) to generate dense motion control points. According to the planned robotic arm end-effector motion velocity distribution and the physical response delay characteristics of the crack-filling actuator, the switching signal is time-domain compensated (including feedforward delay adjustment of the on / off command), and the end-effector flow control parameters (e.g., pumping power) are dynamically adjusted based on the speed. The end-effector control parameters corresponding to the trajectory points are generated synchronously, forming end-effector control parameters that include spatial pose, timestamp, IO status, and flow control parameters, thus forming an end-to-end control command flow.

[0048] The time-domain compensation specifically involves: acquiring the response delay parameters of the crack grouting actuator; based on these parameters, generating an activation command before the planned spatial motion trajectory reaches the crack operation start point, and generating a deactivation command after leaving the crack operation end point. Specifically, this is based on the activation delay time of the crack grouting actuator. With closing delay time Before the robotic arm reaches the origin of the crack The start command is generated continuously after the robotic arm leaves the end of the crack. A shutdown command is generated continuously to ensure precise coverage of the potting material in the crack area.

[0049] The spatial mapping and end-to-end instruction generation are specifically as follows: Step 4.1, Coordinate Mapping: Using a pre-calibrated planar homography matrix , pixel coordinate sequence Convert to 3D world coordinates in the robot arm base coordinate system If the road surface is relatively smooth, the default setting can be used. (Or at a fixed height); In a preferred embodiment where the road surface is uneven, a distance sensor can be installed on the side of the crack sealing actuator, and the real-time height z can be obtained in conjunction with a distance sensor installed at the end to correct the coordinate mapping accuracy: .

[0050] Step 4.2, Trajectory Smoothing: Perform B-spline interpolation on discrete world coordinate points and plan a trapezoidal velocity curve to generate dense robotic arm motion control points, obtaining the linear velocity of the robotic arm's end effector at each moment. Preferably, the planned maximum linear velocity and acceleration are set according to the performance of the robotic arm and the requirements of the potting process, for example, the maximum linear velocity is 0.2 m / s and the maximum acceleration is 0.4 m / s².

[0051] Step 4.3, Timing Compensation Control: A feedforward control model is established to address the physical response delay of the feed gun. Let the feed gun activation delay be... The closing delay is The I / O switch signal is compensated in the following time domain:

[0052] in, and These are the planned times for the robotic arm's end effector to reach the crack's initiation point and leave the crack's end point, respectively.

[0053] Step 4.4, Dynamic Flow Adjustment: Based on the planned linear velocity Dynamically calculate the pumping power of the feed gun. To ensure a constant output per unit length:

[0054] In the formula Fill the pre-defined cross-sectional area of ​​the crack. This is the flow rate calibration coefficient.

[0055] By combining the planned end-effector speed, the operation parameters are dynamically adjusted to ensure the uniformity of material usage for crack repair.

[0056] Step 4.5, Instruction Output: Finally, an end-to-end control instruction stream containing timestamps, spatial poses, IO states, and PWM power is generated and sent to the lower-level machine for execution.

[0057] The present invention also provides a road crack repair system applying the above method, the system being installed on a mobile platform, comprising: A monocular vision acquisition unit is used to acquire two-dimensional digital images of the road surface to be repaired; preferably, the monocular vision acquisition unit is a monocular industrial camera.

[0058] The central processing unit includes an anti-interference segmentation module, a topology planning module, and a control generation module. It executes the end-to-end road crack repair trajectory generation method based on two-dimensional vision described in steps one through four above, generating an end-to-end control command flow. The anti-interference segmentation module performs the anti-interference crack segmentation processing in step one; the topology planning module performs the skeleton topology processing and global path planning in steps two and three; and the control generation module performs the spatial mapping and temporal compensation control in step four.

[0059] The execution unit includes a robotic arm base and a multi-degree-of-freedom robotic arm. The multi-degree-of-freedom robotic arm is mounted on the robotic arm base, and a crack sealing actuator is installed at the end of the multi-degree-of-freedom robotic arm. It receives control command streams from the central processing unit to perform road crack repair operations. Preferably, the execution unit further includes a distance sensor installed on the end of the crack sealing actuator to obtain the real-time road surface height, thereby correcting coordinate mapping accuracy.

[0060] This invention employs a strategy of skeleton extraction, topology disruption, and graph optimization connectivity to effectively solve the path ambiguity and self-intersection problems faced by traditional methods when dealing with intersecting or mesh-like cracks. This ensures that the generated robot trajectory is geometrically smooth and continuous, and logically uniquely traversed. It abandons expensive multi-sensor fusion schemes and achieves end-to-end generation of control commands containing temporal parameters based on monocular vision. This eliminates the technical gap between visual recognition and execution control, significantly improving the engineering usability and operational efficiency of the automated road crack repair system while reducing hardware costs and computing power requirements.

[0061] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A method for end-to-end generation of road crack repair trajectories based on two-dimensional vision, characterized in that... Includes the following steps: Step 1: Anti-interference crack segmentation based on frequency domain analysis; The images of the road surface to be repaired are acquired and then denoised using a pre-set deep learning network to obtain a high-precision crack binarization mask image. Step 2: Topological extraction and deconstruction of the crack skeleton; First, the morphological thinning process is performed on the crack binarized mask image obtained in Step 1 to obtain a crack skeleton with a single pixel width. Then, cross ambiguity is eliminated through topological analysis to obtain a discrete set of simple line segments. Step 3: Global path connectivity based on graph optimization; A fully connected graph model is constructed, and a globally optimal traversal path is planned among the discrete simple line segment sets obtained in step 2 using a composite cost function, forming an ordered pixel-level trajectory sequence. Step 4: Spatial mapping and end-to-end command generation; The ordered pixel-level trajectory sequence formed in Step 3 is mapped to a spatial motion trajectory in the physical world coordinate system. The spatial motion trajectory is subjected to velocity planning and smoothing. Based on the planned velocity distribution and the response delay characteristics of the grouting actuator, a time-domain compensated control signal sequence is generated. According to the spatial motion trajectory, the planned velocity, and the control signal sequence, an end-to-end repair operation control command flow is generated.

2. The end-to-end generation method for road crack repair trajectory based on two-dimensional vision according to claim 1, characterized in that... The pre-set deep learning network mentioned in step 1 is a pre-set deep learning segmentation network based on the ResUNet architecture. The segmentation network integrates an improved Discrete Cosine Transform (DCT) spectral analysis module in the feature extraction layer of the encoder.

3. The end-to-end generation method for road crack repair trajectory based on two-dimensional vision according to claim 2, characterized in that... The improved Discrete Cosine Transform (DCT) spectral analysis module works as follows: First, the input intermediate layer feature map is divided into local blocks, and a two-dimensional DCT transformation is performed on each local block to obtain frequency domain coefficients; then, a learnable frequency domain gating mask is introduced and multiplied element-wise with the frequency domain coefficients to attenuate low-frequency coefficients and retain high-frequency coefficients; finally, the spatial domain features are recovered through Inverse Discrete Cosine Transform (IDCT) and residually connected with the original input features.

4. The end-to-end generation method for road crack repair trajectory based on two-dimensional vision according to claim 1, characterized in that... The topology analysis described in step 2 is as follows: First, traverse the neighborhood connectivity of the skeleton pixels, identify the endpoints and multi-way intersecting branch points in the skeleton, then implement the topology breaking strategy to remove all identified branch points and their neighboring pixels from the skeleton, and finally forcibly decompose the complex mesh skeleton into several independent sets of simple connected domain line segments without branches.

5. The end-to-end generation method for road crack repair trajectory based on two-dimensional vision according to claim 4, characterized in that... The topology breaking strategy is as follows: calculate the number of 8-neighbor connected pixels of each pixel on the skeleton, define the point with 1 8-neighbor connected pixel as the endpoint, and the point with more than 2 8-neighbor connected pixels as the branch point. Delete the branch point and the pixels within its preset range to break the intersection structure into unconnected line segments.

6. The end-to-end generation method for road crack repair trajectory based on two-dimensional vision according to claim 1, characterized in that... The fully connected graph model described in step 3 is a weighted undirected graph model constructed between line segment endpoints based on the distance and direction constraints between endpoints, with the endpoints of the line segments as nodes and the connection relationships between the endpoints of the line segments as edges.

7. The end-to-end generation method for road crack repair trajectory based on two-dimensional vision according to claim 6, characterized in that... The formula for calculating the composite cost function is as follows: in, This is the Euclidean distance between the end of the current path and the endpoint of the line segment to be connected. Let be the angle between the direction of the connecting vector and the tangent direction of the endpoint of the line segment to be connected. and These are the weighting coefficients.

8. The end-to-end generation method for road crack repair trajectory based on two-dimensional vision according to claim 1, characterized in that... The time-domain compensation mentioned in step 4 specifically involves: obtaining the response delay parameter of the crack sealing actuator; based on the response delay parameter, generating an opening command at the moment before the planned spatial motion trajectory reaches the crack operation start point, and generating a closing command at the moment after leaving the crack operation end point.

9. The end-to-end generation method for road crack repair trajectory based on two-dimensional vision according to claim 8, characterized in that... The end-to-end repair operation control command flow includes spatial pose, timestamp, IO status, and flow control parameters.

10. A road crack repair system, characterized in that... include: A monocular vision acquisition unit is used to acquire two-dimensional digital images of the road surface to be repaired; The central processing unit includes an anti-interference segmentation module, a topology planning module, and a control generation module, used to generate an end-to-end control command flow by applying the two-dimensional vision-based end-to-end generation method for road crack repair trajectory as described in any one of claims 1 to 9. The execution unit includes a robotic arm base and a multi-degree-of-freedom robotic arm. The multi-degree-of-freedom robotic arm is mounted on the robotic arm base, and a crack filling actuator is installed at the end of the multi-degree-of-freedom robotic arm for receiving control command streams from the central processing unit to perform road crack repair operations.

Images