Road crack autonomous tracking method, system and storage medium

Abstract

The present application relates to the technical field of road maintenance, and specifically provides a road crack autonomous tracking method, a system and a storage medium, wherein the road crack autonomous tracking method comprises the following steps: collecting road images in real time; performing crack semantic segmentation by using a visual transformer network fusing self-attention and cross-attention; modeling the global structure of the crack by using self-attention to avoid missing detection of the crack; fusing multi-scale features by using cross-attention to suppress background interference, so as to obtain a high-precision crack binary mask; performing skeletonization on the crack binary mask and extracting a region of interest to calculate a deviation; adaptively setting PID parameters based on the deviation and controlling the movement of a maintenance device, so that the device can adapt to irregular trajectories of the crack, avoid overshoot and tracking loss, and realize accurate deviation correction; and thus the crack recognition accuracy on complex road surfaces is improved, the adaptive ability and stability of tracking control are enhanced, and efficient, accurate and stable real-time autonomous tracking repair of road cracks is realized.

Description

Technical Field

[0001] This invention relates to the field of road maintenance technology, and in particular to a method, system and storage medium for autonomous tracking of road cracks. Background Technology

[0002] Roads are prone to cracks and other defects during long-term service. If not repaired in time, they will deteriorate rapidly, affecting road life and driving safety. Traditional manual repair is inefficient and poses a high environmental risk. Automated crack repair robots have become the mainstream development direction. When using robots for automated crack repair, related technologies often use CNN (Convolutional Neural Network) for crack identification and PID (Proportional Integral Derivative) for robot motion control.

[0003] However, due to the limited local receptive field of the CNN used in the relevant technologies, the global feature extraction of slender and topologically complex cracks under unstructured pavement is insufficient, which easily leads to missed detection, rough fracture and segmentation, and difficulty in adapting to complex pavement backgrounds such as oil stains, shadows, and texture differences. At the same time, since the PID used has fixed parameters, it cannot adapt to the nonlinear, time-varying, and irregular characteristics of crack trajectory. It is prone to overshoot, oscillation, or even tracking loss when turning sharply or curvature changes. Therefore, the accuracy, reliability and stability of crack repair are poor. Summary of the Invention

[0004] This invention aims to at least address the technical problems of insufficient global feature extraction of slender cracks under unstructured pavements and poor control capability when using fixed-parameter proportional-integral-differential (PI-DI) crack tracking control in related technologies. Therefore, the purpose of this invention is to propose an autonomous road crack tracking method, system, and storage medium.

[0005] In a first aspect, the present invention proposes an autonomous road crack tracking method, comprising the following steps: acquiring real-time visual images of the target road; performing crack semantic segmentation on the real-time visual images based on a visual converter network that integrates self-attention and cross-attention to obtain a crack binarized mask; performing skeletonization and region of interest extraction on the crack binarized mask, and calculating the deviation between the current maintenance equipment and the target crack path; adaptively tuning the proportional-integral-derivative (PID) control parameters based on the deviation, and controlling the movement of the maintenance equipment according to the tuned PID control parameters to complete real-time crack tracking.

[0006] According to the road crack autonomous tracking method of this invention, road images are first acquired in real time to provide reliable raw visual data for crack tracking. Then, a visual converter network fusing self-attention and cross-attention is used for crack semantic segmentation. This method can utilize self-attention to model the global structure of the crack and avoid missed cracks, while cross-attention can fuse multi-scale features and suppress background interference, thereby obtaining a high-precision crack binarization mask. Next, the crack binarization mask is skeletonized and the region of interest is extracted and the deviation is calculated. While preserving the crack direction, the calculation is simplified, and the position and heading deviation between the equipment and the crack are accurately obtained, providing a stable input for control. Finally, the PID parameters are adaptively tuned based on the deviation to control the movement of the maintenance equipment. This method can adapt to the irregular trajectory of the crack, avoid overshoot and tracking loss, and achieve precise correction. Thus, it not only improves the crack recognition accuracy under complex road surfaces, but also enhances the adaptive capability and stability of tracking control, ultimately achieving efficient, accurate, and stable real-time autonomous tracking and repair of road cracks.

[0007] In addition, the road crack autonomous tracking method according to embodiments of the present invention may also have the following additional technical features:

[0008] Furthermore, the visual converter network is a dual-branch parallel structure, with two branches extracting multi-scale features from consecutive frames respectively, and feature fusion is completed at multiple scale levels by the converter module. Based on the fused features, a crack binarized mask is output. When performing feature fusion, the converter module sequentially performs self-attention calculation and cross-attention calculation. The self-attention calculation is used to model the global dependency relationship of single-branch features, and the cross-attention calculation is used to realize the interactive fusion of cross-frame features.

[0009] Furthermore, the visual converter network uses a hybrid visual converter as the encoder, which consists of an overlapping block embedding layer and an efficient self-attention layer. The encoder outputs a multi-scale crack feature map, which is then fused by a fully connected decoder to output a crack binarized mask. The efficient self-attention layer is used to perform spatial dimensionality reduction on the key matrix and value matrix to reduce the computational complexity of attention.

[0010] Furthermore, the process of skeletonizing and extracting the region of interest from the binary crack mask includes: iteratively removing boundary pixels from the binary crack mask image using a thinning algorithm to obtain a crack skeleton with a single pixel width; and defining a horizontal region of a preset height as the region of interest on the binary crack mask image.

[0011] Furthermore, the calculation of the deviation between the current maintenance equipment and the target crack path includes: calculating the centroid of the crack skeleton in the region of interest and using it as the tracking target point; based on the camera calibration parameters and coordinate transformation relationship, converting the tracking target point in the image coordinate system into physical coordinates in the maintenance equipment base coordinate system, and calculating the lateral position deviation and heading angle deviation.

[0012] Further, the adaptive tuning of the proportional-integral-derivative (PID) control parameters based on the deviation includes: inputting the deviation into a neural network and outputting the PID control parameters; wherein the neural network is a three-layer backpropagation neural network, including an input layer, a hidden layer, and an output layer; wherein the input layer is used to receive the deviation; the hidden layer uses a hyperbolic tangent activation function to perform nonlinear feature extraction and mapping on the input signal; and the output layer is used to output a non-negative PID gain parameter using a deformed sigmoid growth function to obtain the PID control parameters.

[0013] Further, the adaptive tuning of the proportional-integral-derivative (PID) control parameters based on the deviation includes: inputting the deviation into a neural network and outputting the PID control parameters; wherein the neural network is a single-hidden-layer feedforward network constructed using an extreme learning machine, including an input layer, a hidden layer, and an output layer; wherein the input layer is used to receive the deviation; the hidden layer contains multiple neurons and uses a sigmoid function or a radial basis function as the activation function, and the hidden layer is used to determine the hidden layer output matrix based on randomly generated input weights and biases; the output layer is used to calculate the PID gain vector based on the hidden layer output matrix, and solve the corresponding linear equation system using the least squares method to obtain the optimal output weights, thereby calculating the PID control parameters.

[0014] Furthermore, the step of controlling the movement of the maintenance equipment according to the tuned proportional-integral-derivative (PID) control parameters includes: calculating the steering control quantity and the lateral movement control quantity based on the incremental PID control rules and the tuned PID control parameters; and controlling the movement of the maintenance equipment according to the steering control quantity and the lateral movement control quantity.

[0015] Secondly, based on the same inventive concept, this invention also proposes an autonomous road crack tracking system, comprising: an acquisition module for acquiring real-time visual images of the target road; a segmentation module for performing crack semantic segmentation on the real-time visual images based on a visual converter network that integrates self-attention and cross-attention to obtain a crack binarized mask; a calculation module for skeletonizing and extracting the region of interest from the crack binarized mask, and calculating the deviation between the current maintenance equipment and the target crack path; and a control module for adaptively tuning proportional-integral-derivative (PID) control parameters based on the deviation, and controlling the movement of the maintenance equipment according to the tuned PID control parameters to complete real-time crack tracking.

[0016] The road crack autonomous tracking system provided in the second aspect has the same technical features as the road crack autonomous tracking method provided in the first aspect, and therefore also has similar technical effects as the first aspect, which will not be described in detail here.

[0017] Thirdly, based on the same inventive concept, the present invention also proposes a computer-readable storage medium storing a road crack autonomous tracking program, which, when executed by a processor, implements the road crack autonomous tracking method as described in the above embodiments of the present invention.

[0018] The computer-readable storage medium provided in the third aspect has the same technical features as the road crack autonomous tracking method provided in the first aspect, and therefore also has similar technical effects as the first aspect, which will not be described in detail here.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The above and additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic diagram of the framework of an autonomous road crack tracking method according to an embodiment of the present invention; Figure 2 This is a flowchart of a road crack autonomous tracking method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a visual converter network architecture according to a specific embodiment of the present invention; Figure 4 This is a schematic diagram of the configuration of a maintenance robot and an embedded system according to a specific embodiment of the present invention; Figure 5 This is a visual comparison diagram of crack segmentation results according to a specific embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the comparison results of control algorithm optimization according to a specific embodiment of the present invention; Figure 7 This is a structural block diagram of a road crack autonomous tracking system according to an embodiment of the present invention.

[0021] Figure label: 100 - Autonomous tracking system for road cracks; 110 - Acquisition module; 120 - Segmentation module; 130 - Solution module; 140 - Control module. Detailed Implementation

[0022] The embodiments of the present invention are described in detail below, and the embodiments described with reference to the accompanying drawings are exemplary.

[0023] Currently, automated repair technologies still face key challenges in unstructured road environments: 1. Insufficient Perceptual Accuracy: Traditional image processing algorithms (such as thresholding and edge detection) struggle to handle interference from complex road surface backgrounds (such as oil stains, water stains, shadows, and differences in road surface texture). While early convolutional neural network models improved accuracy, they were limited by their local receptive field. When processing long, thin road surface cracks with complex topologies and extremely high aspect ratios, they often failed to capture long-distance dependencies, leading to insufficient feature extraction and problems such as breakage, missed detections, or coarse segmentation.

[0024] 2. Poor control response lag and adaptability: Crack trajectories are highly irregular, nonlinear, and random; traditional PID controllers have fixed parameters, making it difficult to adapt to the tracking requirements of nonlinear time-varying systems; when cracks change direction sharply or curvature changes, fixed-parameter PID controllers often lead to system overshoot oscillation or tracking loss; although improved methods such as fuzzy PID have been proposed, their rule base design relies on expert experience, making it difficult to achieve optimal control, and parameter adjustment is difficult in special environments, affecting the efficiency and accuracy of path tracking.

[0025] To address the shortcomings of insufficient feature extraction capabilities and difficulties in adaptive adjustment of control parameters in related technologies, this invention provides a method, system, and storage medium for autonomous tracking of road cracks. Through a collaborative design of global feature extraction using a visual Transformer and adaptive PID tuning via a neural network, it can accurately perceive minute cracks in complex backgrounds and autonomously track and repair cracks by adaptively adjusting control parameters in real time according to changes in the crack trajectory. (See below for reference.) Figures 1-7 A method, system, and storage medium for autonomous tracking of road cracks according to embodiments of the present invention are described.

[0026] Figure 1 This is a schematic diagram of the framework of an autonomous road crack tracking method according to an embodiment of the present invention. Figure 1As shown, the framework includes a Transformer crack segmentation module and a Neural-Proportional-Integral-Derivative (PID) tracking control module. It should be noted that the road crack autonomous tracking method of this embodiment is implemented based on the above framework, which will not be described in detail below.

[0027] Figure 2 This is a flowchart of a road crack autonomous tracking method according to an embodiment of the present invention. Figure 2 As shown, a road crack autonomous tracking method according to an embodiment of the present invention includes the following steps: Step S1: Acquire real-time visual images of the target road.

[0028] In a specific embodiment, the visual sensor mounted on the maintenance equipment can be used to acquire continuous video frames or real-time image data (RGB (Red Green Blue) images and depth images) of the road surface, providing raw input for crack detection. Specifically, the image resolution can be set to 480×320 pixels or other high-resolution formats.

[0029] Specifically, the embodiments of the present invention continuously acquire the road surface scene in front of the device through real-time image acquisition, ensuring the timeliness and completeness of the perception data, providing a stable data source for subsequent crack segmentation, deviation calculation and motion control, and supporting the real-time operation of the entire closed loop.

[0030] Step S2: Based on a visual converter network that integrates self-attention and cross-attention, perform crack semantic segmentation on the real-time visual image to obtain a crack binarized mask.

[0031] In a specific embodiment, a visual Transformer is used as the segmentation architecture. A self-attention mechanism is employed to model the global contextual dependency of cracks, while a cross-attention mechanism is used to achieve cross-feature and cross-frame information interaction and fusion. This process classifies image pixels and outputs a binary mask of the crack region. Specifically, self-attention models the dependencies between different spatial locations within the same feature map, allowing the network to learn the degree of association between each point in the image and all other points. This not only captures long-distance, global contextual information about cracks, addressing the issues of fractures and discontinuities in thin cracks, but also establishes connections between pixels within the crack region, strengthening the overall structural features of the crack and suppressing local noise interference. Cross-attention uses one set of features as a query and performs attention matching and information fusion on another set of features from different sources (such as another branch or another frame image). This not only enables cross-branch and cross-frame feature interaction, fully utilizing multi-scale or temporal information complementarity, but also enhances the feature aggregation ability for complex, discontinuous, and weak-contrast cracks, improving segmentation integrity and robustness.

[0032] Specifically, the embodiments of the present invention can overcome the limitations of the local receptive field of traditional CNN (Convolutional Neural Network) through the self-attention mechanism, fully capture the long-distance spatial correlation of slender cracks, and avoid crack breakage and missed detection; through the cross-attention mechanism, feature complementarity can be enhanced and interference from complex backgrounds such as oil stains, shadows, and texture differences can be suppressed; through the synergy of the two, the network output can produce crack masks with clear boundaries and complete regions, significantly improving segmentation accuracy and robustness.

[0033] Step S3: Perform skeletonization and region of interest extraction on the binary mask of the crack, and calculate the deviation between the current maintenance equipment and the target crack path.

[0034] In a specific embodiment, the maintenance equipment is, for example, a maintenance robot, and the deviation includes, for example, lateral position deviation and heading angle deviation. For example, a single-pixel width crack skeleton can be obtained by performing a thinning process on the binarized mask, selecting an effective tracking area and calculating the target point, and obtaining the lateral position deviation and heading angle deviation between the equipment and the crack path through coordinate transformation.

[0035] Specifically, the embodiments of the present invention simplify the complex crack topology into a single-pixel centerline by skeletonization, thereby reducing the complexity of path description; extract the effective tracking range of the focusing forward look by the region of interest, and eliminate redundant information to reduce the amount of computation; by converting image deviation into device physical space deviation, the deviation can be accurately quantified, providing a stable and reliable input signal for the control module.

[0036] Step S4: Based on the deviation, adaptively tune the proportional-integral-derivative (PID) control parameters, and control the movement of the maintenance equipment according to the tuned PID control parameters to achieve real-time crack tracking.

[0037] In a specific embodiment, for example, the deviation can be used as an input vector to input the neural PID controller. The neural PID controller uses a neural network to adaptively adjust the proportional coefficient, integral coefficient, and derivative coefficient of the PID controller online. Then, based on the adjusted PID parameters, the control quantity is calculated to generate motion control commands, which drive the maintenance equipment (robot) to perform real-time tracking motion along the crack path.

[0038] Specifically, the embodiments of the present invention are based on the deviation adaptive dynamic adjustment of proportional-integral-derivative control parameters, which solves the defects of traditional fixed-parameter PID that cannot adapt to the nonlinear, time-varying, and random trajectories of cracks, and avoids overshoot, oscillation and tracking loss. By combining adaptive parameters with real-time deviation to calculate the control quantity, the maintenance equipment can quickly respond to changes in crack direction, which helps to achieve continuous, stable and accurate real-time tracking.

[0039] Therefore, the road crack autonomous tracking method according to embodiments of the present invention first acquires road images in real time to provide reliable raw visual data for crack tracking; then, a visual converter network fusing self-attention and cross-attention is used for crack semantic segmentation, which can both utilize self-attention to model the global structure of the crack and avoid missed cracks, and use cross-attention to fuse multi-scale features and suppress background interference, thereby obtaining a high-precision crack binarization mask; next, by skeletonizing the crack binarization mask and extracting the region of interest and calculating the deviation, the calculation is simplified while preserving the crack direction, and the position and heading deviation between the equipment and the crack is accurately obtained, providing a stable input for control; finally, the PID parameters are adaptively tuned based on the deviation and the movement of the maintenance equipment is controlled, which can adapt to the irregular trajectory of the crack, avoid overshoot and tracking loss, and achieve accurate correction; thus, it not only improves the crack recognition accuracy under complex road surfaces, but also enhances the adaptive capability and stability of tracking control, ultimately achieving efficient, accurate, and stable real-time autonomous tracking and repair of road cracks.

[0040] In one embodiment of the present invention, the visual converter network is a dual-branch parallel structure. The two branches extract multi-scale features of consecutive frames respectively, and the features are fused at multiple scale levels by the converter module. Based on the fused features, a crack binarized mask is output. When performing feature fusion, the converter module sequentially performs self-attention calculation and cross-attention calculation. Self-attention calculation is used to model the global dependency relationship of single-branch features, and cross-attention calculation is used to realize the interactive fusion of cross-frame features.

[0041] Figure 3 This is a schematic diagram of a visual converter network architecture according to a specific embodiment of the present invention. Figure 3 As shown, the visual converter network adopts a dual-branch structure to process the feature extraction of adjacent video frames separately. This network is an extension of the YOLOv5 backbone network and integrates the Fusion Transformer (FT) module.

[0042] Two parallel branches extract features from the current frame and the reference frame, respectively. The FT module interacts and fuses the features at three different scale levels, thereby integrating fused features containing both coarse-grained and fine-grained semantic information.

[0043] The fusion process employs a multi-head attention mechanism, including self-attention and cross-attention. The input feature map undergoes a linear transformation to generate a query vector Q, a key vector K, and a value vector V. The formula for calculating the attention score is as follows: ,in, For the attention head dimension; calculate the similarity between Q and K through dot product operation, and divide by Scaling is performed, and softmax normalization is applied to obtain weights. Finally, V is weighted and summed. The cross-attention mechanism achieves deep fusion of cross-frame features by using features from one branch as queries and features from another branch as keys and values, effectively capturing the global dependencies of cracks.

[0044] Specifically, the embodiments of the present invention extract multi-scale features from consecutive frames through two branches, preserving fine-grained details and coarse-grained semantics at different resolutions; expand the feature perception range through multi-scale hierarchical fusion; complete global modeling within a single branch through self-attention; and achieve deep interaction of cross-frame features through cross-attention; thereby improving the segmentation integrity and accuracy of complex, curved, and discontinuous cracks through multiple design collaborations.

[0045] In one embodiment of the present invention, the visual converter network uses a hybrid visual converter as the encoder. The encoder consists of an overlapping block embedding layer and an efficient self-attention layer. The encoder outputs a multi-scale crack feature map. After the multi-scale crack feature map is fused by a fully connected decoder, a crack binarized mask is output. The efficient self-attention layer is used to perform spatial dimensionality reduction on the key matrix and the value matrix to reduce the computational complexity of attention.

[0046] In a specific embodiment, the visual converter network architecture includes a hierarchical Transformer encoder and a lightweight fully connected decoder. Specifically, the encoder comprises four stages designed to generate multi-scale feature maps (with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively). Each stage employs Overlapped Patch Embeddings, embedded through convolutional layers with kernel size K=7 and stride S=4, effectively preserving the local continuity of the crack edges.

[0047] To reduce computational complexity, a sequence reduction mechanism is employed. After reshaping the input feature map X, before generating K and V, a convolutional layer with a spatial reduction ratio R is used to reduce the sequence length. , , , Therefore, the computational complexity is reduced from Reduce to This significantly improves reasoning speed and meets the real-time control requirements of maintenance equipment.

[0048] The decoder consists of an MLP (Multilayer Perceptron), which fuses multi-scale features and outputs the final binarized crack mask image.

[0049] Specifically, the embodiments of the present invention maintain the local continuity of the crack edge by embedding overlapping blocks to avoid edge blurring; and reduce the sequence length by reducing spatial dimensionality through efficient self-attention, thereby reducing the computational complexity from O(N²) to O(N² / R), thus improving the inference speed without sacrificing segmentation accuracy.

[0050] In one embodiment of the present invention, skeletonization and region of interest extraction are performed on the crack binarized mask, including: using a thinning algorithm to iteratively remove boundary pixels from the crack binarized mask image to obtain a crack skeleton with a single pixel width; and defining a horizontal region of a preset height as the region of interest on the crack binarized mask image.

[0051] In specific embodiments, for example, the Zhang-Suen thinning algorithm can be used to process the binary mask, iteratively removing boundary pixels until a crack skeleton with a single pixel width is retained; a horizontal rectangular area with a preset height is set at the bottom of the image as the ROI (Region of Interest), or the target area is selected based on the forward viewing distance.

[0052] Specifically, the embodiments of the present invention preserve the crack topology and orientation through a refinement algorithm, compressing the crack region into a single-pixel centerline, which greatly reduces the amount of subsequent calculations; by setting a horizontal ROI close to the front of the maintenance equipment, it conforms to the forward-looking tracking logic of the maintenance equipment, focuses on the nearest tracking target, and helps to improve the stability of deviation calculation.

[0053] In one embodiment of the present invention, calculating the deviation between the current maintenance equipment and the target crack path includes: calculating the centroid of the crack skeleton in the region of interest and using it as the tracking target point; based on the camera calibration parameters and coordinate transformation relationship, converting the tracking target point in the image coordinate system into physical coordinates in the maintenance equipment base coordinate system, and calculating the lateral position deviation and heading angle deviation.

[0054] In a specific embodiment, the centroid of the set of all skeleton pixels within the ROI is calculated. , ): By using the camera's intrinsic and extrinsic parameter matrices, the centroid of the image coordinate system is transformed into physical coordinates in the maintenance equipment's base coordinate system. Define lateral position deviation The x-coordinate of the target point and the deviation of the heading angle. It is the angle between the tangent direction of the crack and the direction of advance of the maintenance equipment.

[0055] Specifically, the embodiments of the present invention calculate the centroid of the crack skeleton in the region of interest and use it as the tracking target point. By utilizing the geometric uniqueness of the skeleton centroid, the tracking point is avoided from jumping due to local noise interference. The image coordinates are mapped to the device base coordinates through camera calibration and coordinate transformation, so that the deviation has physical meaning. The relative pose of the device and the crack path can be completely described by the lateral position deviation and the heading angle deviation, thereby providing comprehensive and accurate input for control.

[0056] In one embodiment of the present invention, adaptive tuning of proportional-integral-derivative (PID) control parameters based on deviation includes: inputting the deviation into a neural network and outputting PID control parameters; wherein the neural network is a three-layer backpropagation neural network, including an input layer, a hidden layer, and an output layer; wherein the input layer is used to receive the deviation; the hidden layer uses a hyperbolic tangent activation function to perform nonlinear feature extraction and mapping on the input signal; and the output layer is used to output non-negative PID gain parameters using a deformed S-shaped growth function to obtain the PID control parameters.

[0057] In a specific embodiment, a neural PID controller adjusts the PID parameters online based on the real-time error: proportional gain. Integral gain Differential gain The control law of the neural PID controller is as follows: PID control algorithm is used: Where u(k) is the steering angular velocity or lateral movement speed command of the maintenance equipment, k is the discrete time step, e(k) is the tracking error at the k-th step, e(k-1) is the error of the previous control cycle, and j is the loop variable. Specifically, the neural PID controller adopts a three-layer BP neural network (input layer, hidden layer, output layer) structure. The input layer receives three input signals, such as the set input, the actual output, and the tracking error e(k). The hidden layer contains five neuron nodes and uses the hyperbolic tangent function (Tanh) as the activation function. Where x is the input to the hidden layer neuron, e x It is an exponential function, e x Let f(x) be the reciprocal of the exponential function, f(x) be the output of the neuron, and the output layer outputs three non-negative gain parameters. , , A modified sigmoid function is used to ensure that the output is non-negative: , where u is the upper bound of the output; the network is trained online using the backpropagation algorithm to minimize the tracking error index and achieve adaptive parameter adjustment.

[0058] Specifically, the three-layer BP network structure used in this embodiment of the invention is simple and suitable for online real-time training; the hyperbolic tangent activation function can efficiently extract nonlinear error features, enhancing the network's expressive power; the modified sigmoid growth function ensures that the output parameters are non-negative, conforming to the physical constraints of PID gain; and online parameter optimization is achieved through error backpropagation, continuously minimizing the tracking error.

[0059] In one embodiment of the present invention, adaptive tuning of proportional-integral-derivative (PID) control parameters based on deviation includes: inputting the deviation into a neural network and outputting PID control parameters; wherein the neural network is a single-hidden-layer feedforward network constructed by an extreme learning machine, including an input layer, a hidden layer, and an output layer; wherein the input layer is used to receive the deviation; the hidden layer contains multiple neurons and uses a sigmoid function or a radial basis function as the activation function, and the hidden layer is used to determine the hidden layer output matrix according to randomly generated input weights and biases; the output layer is used to calculate the PID gain vector according to the hidden layer output matrix, and uses the least squares method to solve the corresponding linear equation system to obtain the optimal output weights, and then calculates the PID control parameters.

[0060] In a specific embodiment, a neural PID controller adjusts the PID parameters online based on the real-time error. , , The control law of the neural PID controller is as follows: PID control algorithm is used: Where u(k) is the steering angular velocity or lateral movement speed command of the maintenance equipment, k is the discrete time step, e(k) is the tracking error at the k-th step, e(k-1) is the error of the previous control cycle, and j is the loop variable. Specifically, an SLFN (Single-Layer Feedforward Neural Network) is constructed using ELM (Extreme Learning Machine) as the gain adjustment network. The network structure is as follows: the input vector is... It may include historical control values, where e(k) is the tracking error at the current time, Δe(k) is the error rate of change, and e(k) is the tracking error at the current time. 1) represents the error from the previous time step, e(k) 2) represents the error between the two previous time points, u(k) 1) This represents the control input from the previous time step, which is processed by the output layer through the weight matrix. Calculate the PID gain vector Solving the linear equation system using the least squares method Where H is the hidden layer output matrix, β is the output layer weight matrix, and T is the hidden layer weight matrix. target To obtain the optimal output weights for the target output matrix. ,in is the Moore-Penrose generalized inverse matrix of H; ELM has extremely fast training speed and good generalization performance, and can nonlinearly map the optimal PID parameters according to the real-time error state.

[0061] Specifically, the single hidden layer network structure used in the embodiments of the present invention is simple and has a fast computation speed; the training complexity is reduced by randomly inputting weights; the optimal output weights are solved by the least squares closed-form method, without the need for iterative parameter tuning, and the training speed is much higher than that of the traditional BP; thus, it can balance speed and generalization and adapt to the real-time tuning requirements of different platforms.

[0062] In one embodiment of the present invention, controlling the movement of maintenance equipment according to the tuned proportional-integral-derivative (PID) control parameters includes: calculating steering control quantity and lateral movement control quantity based on incremental PID control rules and the tuned PID control parameters; and controlling the movement of maintenance equipment according to the steering control quantity and lateral movement control quantity.

[0063] In a specific embodiment, the formula for calculating the control quantity is: ,in For maintenance equipment, the commands specify the angular velocity or lateral movement speed. Let be the tracking error at time k. , , Output in real time by a neural network.

[0064] Specifically, the embodiments of the present invention use incremental PID to calculate only the output increment, which has a small computational load, fast dynamic response, and no integral saturation problem; by independently outputting steering and lateral movement commands, it adapts to the movement forms of different types of maintenance equipment, ensuring smooth movement and continuous tracking of maintenance equipment.

[0065] The following is a comparative verification of the road crack autonomous tracking method of the above embodiments of the present invention, with reference to a specific embodiment. In this specific embodiment, the maintenance equipment is a maintenance robot.

[0066] Figure 4 This is a schematic diagram of a maintenance robot and embedded system configuration according to a specific embodiment of the present invention. Figure 4 As shown, in this specific embodiment, the maintenance robot is equipped with an embedded computer and a depth camera.

[0067] In this specific embodiment, the verification was performed on a CFD (Crack Forest Dataset) dataset and a maintenance robot in a real road environment.

[0068] Segmentation Accuracy: Table 1 shows the real-time segmentation results on the CFD dataset. It can be seen that compared with the original YOLOv5, this method has significantly improved in all metrics for five different model sizes (n, s, m, l, x). Taking model size "x" as an example, Precision(M) increased from 0.4726 to 0.6818, Recall(M) increased from 0.4211 to 0.5178, and mAP0.5(M) increased from 0.3756 to 0.5304. This indicates that this method can effectively generate masks that appropriately cover the target cracks, thanks to the use of self-attention and cross-attention. To further improve performance, the FT module is integrated to fuse crack features. This method can generate high-precision binary masks and is suitable for various complex scenarios.

[0069] Table 1 Real-time segmentation results on the CFD dataset

[0070] Figure 5 This is a visual comparison diagram of crack segmentation results according to a specific embodiment of the present invention. Figure 5 As shown in the visualization results, this method can segment slender crack regions more accurately than the baseline YOLOv5, effectively reducing missed detections and false detections.

[0071] Tracking accuracy: Table 2 shows the comparison results of crack tracking errors of different control methods. It can be seen that in the combined tests of three different crack paths (straight crack, curved crack, and continuous turning crack) and three different model sizes (s, m, x), the Neural-PID method (this method) achieved the smallest tracking error. Taking straight crack and model size "s" as an example, the average tracking error of Neural-PID is 4.12 mm, while that of traditional PID is 9.57 mm, Fuzzy PID is 5.73 mm, and CEAFC (Cross-Entropy Adaptive Fuzzy Control) is 4.54 mm. The experimental results show that the Neural-PID method can effectively reduce tracking error and improve tracking accuracy.

[0072] Table 2 Comparison of Crack Tracking Errors (Unit: mm)

[0073] Figure 6 This is a schematic diagram illustrating the comparison results of control algorithm optimization according to a specific embodiment of the present invention. Figure 6 As shown, experiments were conducted on real roads to verify the effectiveness of the proposed method. The comparative results show that the road crack tracking method based on Transformer fusion network and neural network PID controller can achieve stable real-time tracking in unstructured road environments, providing reliable technical support for automated road crack repair.

[0074] In summary, the road crack autonomous tracking method of the present invention has the following advantages: 1. Enhanced perception capability: The embodiments of this invention are based on a Transformer fusion network. Through self-attention and cross-attention mechanisms, it effectively solves the problem of insufficient feature extraction of slender cracks by traditional convolutional neural networks. It can achieve fine crack segmentation in complex road backgrounds, and significantly improve the segmentation accuracy and recall.

[0075] 2. Improved Control Accuracy and Adaptability: This invention is based on a Neural-PID controller, which utilizes the nonlinear mapping and online learning capabilities of neural networks to overcome the shortcomings of traditional PID controllers, such as fixed parameters and difficulty in adapting to nonlinear time-varying systems. Compared to fuzzy PID and traditional PID, Neural-PID has stronger global search capabilities, faster convergence speed, and higher tracking accuracy, effectively reducing tracking errors.

[0076] 3. Strong system integration: The embodiments of the present invention realize the synergistic integration of visual perception and motion control, and construct a closed-loop control system of "perception-decision-execution", which can realize real-time, stable and efficient automatic tracking and repair of cracks in unstructured road environment.

[0077] In summary, the road crack autonomous tracking method of this invention has the following significant advantages compared with related technologies: 1. Perception level: The Transformer global attention mechanism completely solves the problem of insufficient feature extraction of thin cracks, and significantly improves the segmentation accuracy under complex road surfaces.

[0078] 2. Control level: The neural PID online adaptive tuning enables the controller to have nonlinear adaptation capability, and the tracking error is much lower than that of traditional PID and fuzzy PID.

[0079] 3. Real-time aspect: High-efficiency attention and lightweight network ensure that the segmentation and control calculation speed meets the real-time operation requirements of maintenance equipment.

[0080] 4. Robustness: The entire process is designed to resist interference, which can adapt to unstructured road surface environments such as oil stains, shadows, complex textures, and irregular crack directions.

[0081] 5. Engineering level: The method and system are modular and programmable, compatible with mainstream maintenance robot hardware, and can directly support the industrial application of automated crack repair.

[0082] A further embodiment of the present invention discloses an autonomous road crack tracking system. Figure 7 This is a structural block diagram of a road crack autonomous tracking system according to an embodiment of the present invention. Figure 7 As shown, in one embodiment of the present invention, the road crack autonomous tracking system 100 includes: a data acquisition module 110, a segmentation module 120, a calculation module 130, and a control module 140.

[0083] Specifically, the acquisition module 110 is used to acquire real-time visual images of the target road.

[0084] The segmentation module 120 is used to perform crack semantic segmentation on real-time visual images based on a visual converter network that integrates self-attention and cross-attention, and obtain a crack binarized mask.

[0085] The calculation module 130 is used to skeletonize the crack binarization mask and extract the region of interest, and calculate the deviation between the current maintenance equipment and the target crack path.

[0086] The control module 140 is used to adaptively tune the proportional-integral-derivative (PID) control parameters based on the deviation, and to control the movement of the maintenance equipment according to the tuned PID control parameters in order to achieve real-time crack tracking.

[0087] It should be noted that the specific implementation of the road crack autonomous tracking system 100 in this embodiment of the invention is similar to the specific implementation of the road crack autonomous tracking method described in the above embodiment of the invention, and therefore has similar technical effects. For details, please refer to the description of the road crack autonomous tracking method section. To reduce redundancy, it will not be repeated here.

[0088] A further embodiment of the present invention discloses a computer-readable storage medium storing a road crack autonomous tracking program, which, when executed by a processor, implements the road crack autonomous tracking method as described in any of the above embodiments of the present invention.

[0089] It should be noted that the specific implementation of the computer-readable storage medium in the embodiments of the present invention is similar to the specific implementation described in the road crack autonomous tracking method of the above embodiments of the present invention, and therefore has similar technical effects. For details, please refer to the description of the road crack autonomous tracking method section. To reduce redundancy, it will not be repeated here.

[0090] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0091] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for autonomously tracking road cracks, characterized in that, Includes the following steps: Acquire real-time visual images of the target road; Based on a visual converter network that integrates self-attention and cross-attention, the real-time visual image is subjected to crack semantic segmentation to obtain a crack binarization mask. The crack binarized mask is skeletonized and the region of interest is extracted to calculate the deviation between the current maintenance equipment and the target crack path; Based on the aforementioned deviation, the proportional-integral-derivative (PID) control parameters are adaptively tuned, and the movement of the maintenance equipment is controlled according to the tuned PID parameters to achieve real-time crack tracking.

2. The road crack autonomous tracking method according to claim 1, characterized in that, The visual converter network has a dual-branch parallel structure. The two branches extract multi-scale features from consecutive frames, and feature fusion is performed at multiple scale levels by the converter module. Based on the fused features, a binary crack mask is output. When performing feature fusion, the converter module sequentially performs self-attention calculation and cross-attention calculation. The self-attention calculation is used to model the global dependency relationship of single-branch features, and the cross-attention calculation is used to realize the interactive fusion of cross-frame features.

3. The road crack autonomous tracking method according to claim 1, characterized in that, The visual converter network uses a hybrid visual converter as the encoder, which consists of an overlapping block embedding layer and a high-efficiency self-attention layer. The encoder outputs a multi-scale crack feature map, which is then fused by a fully connected decoder to output a binary crack mask. The efficient self-attention layer is used to perform spatial dimensionality reduction on the key matrix and value matrix to reduce the computational complexity of attention.

4. The road crack autonomous tracking method according to claim 1, characterized in that, The process of skeletonizing and extracting the region of interest from the binary mask of the crack includes: A thinning algorithm is used to iteratively remove boundary pixels from the binary mask image of the crack to obtain a crack skeleton with a single pixel width. On the binary mask image of the crack, a horizontal region of a preset height is defined as the region of interest.

5. The road crack autonomous tracking method according to claim 1 or 4, characterized in that, The calculation of the deviation between the current maintenance equipment and the target crack path includes: Calculate the centroid of the crack skeleton within the region of interest and use it as the tracking target point; Based on the relationship between camera calibration parameters and coordinate transformation, the tracking target point in the image coordinate system is converted into the physical coordinates in the maintenance equipment base coordinate system, and the lateral position deviation and heading angle deviation are calculated.

6. The road crack autonomous tracking method according to claim 1, characterized in that, The adaptive tuning of the proportional-integral-derivative control parameters based on the deviation includes: The deviation is input into a neural network, which outputs the proportional-integral-derivative (PID) control parameters; wherein... The neural network is a three-layer backpropagation neural network, including an input layer, a hidden layer, and an output layer; wherein, The input layer is used to receive the deviation; The hidden layer employs the hyperbolic tangent activation function for nonlinear feature extraction and mapping of the input signal; The output layer is used to output non-negative proportional-integral-derivative gain parameters using a deformed S-shaped growth function, thereby obtaining the proportional-integral-derivative control parameters.

7. The road crack autonomous tracking method according to claim 1, characterized in that, The adaptive tuning of the proportional-integral-derivative control parameters based on the deviation includes: The deviation is input into a neural network, which outputs the proportional-integral-derivative (PID) control parameters; wherein... The neural network is a single-hidden-layer feedforward network constructed using an extreme learning machine, comprising an input layer, a hidden layer, and an output layer; wherein, The input layer is used to receive the deviation; The hidden layer contains multiple neurons and uses a sigmoid function or radial basis function as the activation function. The hidden layer is used to determine the hidden layer output matrix based on randomly generated input weights and biases. The output layer is used to calculate the proportional-integral-derivative gain vector based on the output matrix of the hidden layer, and to solve the corresponding linear equation system using the least squares method to obtain the optimal output weight, thereby calculating the proportional-integral-derivative control parameters.

8. The road crack autonomous tracking method according to claim 1, characterized in that, The control of the maintenance equipment movement based on the adjusted proportional-integral-derivative control parameters includes: Based on the incremental proportional-integral-derivative (PID) control rule, the steering control quantity and lateral movement control quantity are calculated using the tuned PID control parameters. The maintenance equipment is moved according to the steering control amount and the lateral movement control amount.

9. A road crack autonomous tracking system, characterized in that, include: The acquisition module is used to acquire real-time visual images of the target road. The segmentation module is used to perform crack semantic segmentation on the real-time visual image based on a visual converter network that integrates self-attention and cross-attention, and obtain a crack binarization mask. The calculation module is used to skeletonize and extract the region of interest from the binary mask of the crack, and to calculate the deviation between the current maintenance equipment and the target crack path. The control module is used to adaptively tune the proportional-integral-derivative (PID) control parameters based on the deviation, and control the movement of the maintenance equipment according to the tuned PID control parameters to achieve real-time crack tracking.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a road crack autonomous tracking program, which, when executed by a processor, implements the road crack autonomous tracking method as described in any one of claims 1-8.

Images