End-to-end lightweight road crack sensing method, system, equipment and medium

Abstract

The invention discloses an end-to-end lightweight road crack sensing method, system and device and a medium, belongs to the technical field of computer vision and deep learning, and aims to solve the technical problem of how to quickly and accurately detect road cracks and improve the road crack detection efficiency. According to the technical scheme, the method comprises the following steps: image standardization processing: carrying out standardization preprocessing on a collected road image to obtain a road image after standardization preprocessing; multi-scale dynamic labeling: labeling the road image after standardization preprocessing by adopting a multi-scale dynamic labeling mechanism to construct a training label; light-weight road crack perception: based on an improved DETR model, introducing a cross attention mechanism into a Transform encoder to improve the perception ability of crack local features, and adopting a serialization interaction mechanism in the Transform encoder to realize crack integrity reasoning; and transfer learning incremental training: improving the generalization performance of the model through a transfer learning strategy combining public data set pre-training and own data set incremental training.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, specifically to an end-to-end lightweight method, system, device, and medium for sensing road cracks. Background Technology

[0002] Timely and accurate detection of road defects and cracks is crucial for ensuring road safety, extending service life, and reducing maintenance costs. With the acceleration of urbanization and the increase in traffic volume, road cracks and defects are becoming increasingly serious, affecting not only driving safety but also increasing the burden of road maintenance.

[0003] Currently, the detection of road defects and cracks mainly relies on manual inspections. Inspectors check along the road, visually observe cracks, take photos, and record data. However, the inspection area that inspectors can typically complete each day is only 8-10 kilometers, which is inefficient. Moreover, due to the subjectivity of manual inspection, it is easy to miss or misjudge, with a miss rate as high as 25%-35%.

[0004] Therefore, how to quickly and accurately detect road cracks and improve the efficiency of road crack detection is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The technical objective of this invention is to provide an end-to-end lightweight road crack detection method, system, device, and medium to address the problem of how to quickly and accurately detect road cracks and improve the efficiency of road crack detection.

[0006] The technical objective of this invention is achieved as follows: an end-to-end lightweight road crack detection method, the specific method of which is as follows:

[0007] Image standardization processing: Acquire road images and perform standardization preprocessing on the acquired road images to obtain standardized preprocessed road images;

[0008] Multi-scale dynamic annotation: A multi-scale dynamic annotation mechanism is used to annotate the standardized preprocessed road images to construct training labels;

[0009] Lightweight road crack perception: Based on the improved DETR model, a cross-attention mechanism is introduced in the Transformer encoder to enhance the perception of local crack features, and a serialized interaction mechanism is used in the Transformer decoder to realize crack integrity reasoning.

[0010] Incremental training for transfer learning: This strategy improves the generalization performance of a model by combining pre-training on a public dataset with incremental training on a proprietary dataset.

[0011] As a preferred method, the image normalization process is as follows:

[0012] Size standardization: The image is uniformly cropped into multiple 600*600 pixel sub-images to ensure that the spatial dimension of each training sample is uniform, so that the improved DETR model can stably learn the spatial feature rules and avoid feature extraction deviation caused by size fluctuations.

[0013] Pixel normalization: Normalize the pixel values ​​of the sub-image to the range of [0,1] to obtain the sub-image after pixel normalization. This prevents the weight update imbalance of the improved DETR model caused by the difference in pixel value range of different images (such as 0-255 or 0-65535). By scaling the pixel value scale, the improved DETR model can converge quickly in the early stage of training, thus improving training efficiency.

[0014] Adaptive Histogram Equalization: Histogram equalization is performed on the sub-image after pixel normalization. To address the problem of uneven lighting and low contrast in the original image, which leads to blurred crack features, the gray-level distribution of the image is automatically adjusted to enhance the gray-level difference between the crack and the background, making the crack features more prominent. This helps the improved DETR model to capture target area information more clearly and improves the ability to identify weak features.

[0015] As a preferred option, multi-scale dynamic annotation is as follows:

[0016] Composite label construction: Cracks longer than 10cm are segmented into overlapping local bounding boxes, which are then combined with global bounding boxes to form composite labels. This solves the problem that a single global bounding box is insufficient for depicting the local details of long cracks. The global bounding box ensures that the improved DETR model grasps the overall shape and location of the crack, while the overlapping local bounding boxes focus on the detailed features of small-scale crack segments. This allows the improved DETR model to learn both global context and local detail information, thereby improving its ability to detect minute defects in long cracks.

[0017] Adaptive Local Annotation Box Size Mechanism: An adaptive annotation box size determination mechanism that integrates crack length and width variations is introduced. For crack sections with large width variations, the local annotation box size is appropriately increased. This adaptive local annotation box size mechanism solves the problem that traditional fixed-size annotation boxes are prone to missing key features when crack width changes abruptly. By dynamically adjusting the annotation box size, it ensures that crack details in the width variation area are fully included, avoiding feature truncation caused by excessively small annotation boxes. This enables the improved DETR model to learn the crack width variation pattern more accurately, further enhancing the completeness and effectiveness of annotation information.

[0018] As a preferred option, lightweight road crack detection is specifically as follows:

[0019] Optimize the Transformer encoder structure based on cross-attention enhancement: Introduce a cross-attention mechanism before the feedforward network (FFN) of the Transformer encoder layer. Actively combine external information of the original image texture and scene context. When the improved DETR model processes small cracks, it selectively retrieves and enhances the feature representation related to crack details to obtain auxiliary cues, which can accurately locate crack areas and enhance its own features.

[0020] Serialization Interactive Decoding and Crack Integrity Reasoning: A lightweight serialization interaction module is introduced after the standard Transformer decoder layer. This module performs an additional round of self-attention interaction on the output of all object queries. During this process, the improved DETR model learns the spatial and semantic relationships between crack fragments, i.e., it determines whether two bounding boxes should be connected end-to-end. This further guides the improved DETR model to predict a "connection confidence," and the bounding boxes with high confidence are merged into a complete crack in post-processing.

[0021] More preferably, in each standard Transformer encoder layer, image features first capture global contextual dependencies through a multi-head self-attention (MSA) mechanism; then, the normalized features are used as a query and cross-attention is performed with a set of learnable local prior feature keys to further enhance the interaction with relevant features of the target region; finally, the enhanced encoded features are output through a feedforward network for nonlinear transformation and feature integration.

[0022] After embedding a cross-attention mechanism in the Transformer encoder layer, the improved DETR model is guided to focus on salient regions through a set of optimizable local prior feature libraries. During training, the prior features gradually characterize the typical local patterns of cracks (such as edge response, texture irregularity, etc.), enabling the Transformer encoder structure layer to have local structure enhancement capabilities on the basis of global perception.

[0023] As a preferred method, incremental training of transfer learning is specifically as follows:

[0024] During the pre-training phase, the improved DETR model was trained on a public dataset of 20,000 road crack images, enabling it to learn rich general visual features and possess strong feature extraction and semantic representation capabilities. This significantly improved the sensitivity of the improved DETR model to typical targets of cracks. The improved DETR model achieved an accuracy of 88.4% in detecting road targets on the public dataset, but its generalization ability was weak, with an accuracy of only 46.3% in real-world scenarios. However, it already possessed the basic feature extraction capabilities for various types of cracks, laying an important foundation for subsequent domain-adaptive optimization.

[0025] After entering the incremental training phase, a transfer learning mechanism was introduced, and the pre-trained model was further optimized using 6,000 proprietary road image data. During this process, the weights of the first two layers of the CNN backbone network were frozen to retain the general feature representations learned in the pre-training phase. At the same time, a cosine annealing learning rate scheduling strategy was adopted to dynamically adjust the learning rate during the training process of the improved DETR model, promoting the improved DETR model to efficiently adapt to the distribution characteristics of crack features in the target dataset. Through the incremental training strategy, the improved DETR model, while inheriting the original generalization features, further enhances its ability to identify and represent the unique morphology and fine structure of road disease cracks.

[0026] The road image to be detected is input into the improved DETR model after incremental training. The improved DETR model outputs the bounding box coordinates of road cracks in the image end-to-end, realizing accurate localization and detection of the disease area. Based on the obtained crack location information, the improved DETR model not only completes the initial localization, but also provides a stable and reliable foundation for further fine classification (such as distinguishing between transverse cracks, longitudinal cracks, alligator cracks and block cracks). It provides key data support for subsequent disease assessment, maintenance decision-making and long-term pavement performance research. After the standardized images are stitched together, the final detection results are obtained.

[0027] An end-to-end lightweight road crack sensing system, comprising:

[0028] The image standardization processing module is used to acquire road images and perform standardization preprocessing on the acquired road images to obtain standardized preprocessed road images;

[0029] The multi-scale dynamic annotation module is used to annotate and construct training labels for standardized preprocessed road images using a multi-scale dynamic annotation mechanism.

[0030] A lightweight road crack perception module is used to improve the perception of local crack features by introducing a cross-attention mechanism in the Transformer encoder based on the improved DETR model, and to realize crack integrity inference by using a serialized interaction mechanism in the Transformer decoder.

[0031] The transfer learning incremental training module is used to improve the generalization performance of models by combining pre-training on public datasets with incremental training on proprietary datasets.

[0032] As a preferred option, the lightweight road crack sensing module includes:

[0033] The optimization submodule introduces a cross-attention mechanism before the feedforward network (FFN) of the Transformer encoder layer. This actively combines external information from the original image texture and scene context. When processing small cracks, the improved DETR model selectively retrieves and enhances feature representations related to crack details, obtaining auxiliary cues to both accurately locate crack regions and strengthen its own features. In each standard Transformer encoder layer, image features first capture global contextual dependencies through a multi-head self-attention (MSA) mechanism. Subsequently, the normalized features are used as a query. The Transformer encoder layer performs cross-attention calculations with a set of learnable local prior feature key-value pairs to further enhance the interaction with relevant features of the target region. Finally, it undergoes nonlinear transformation and feature integration via a feedforward network to output the enhanced encoded features. After embedding the cross-attention mechanism, the Transformer encoder layer guides the improved DETR model to focus on salient regions through a set of optimizable local prior feature libraries. During training, the prior features gradually characterize the typical local patterns of cracks (such as edge response, texture irregularities, etc.), enabling the Transformer encoder structure layer to have local structure enhancement capabilities on the basis of global perception.

[0034] The inference submodule introduces a lightweight serialization interaction module after the standard Transformer decoder layer. This module performs an additional round of self-attention interaction on the output of all object queries. During this process, the improved DETR model learns the spatial and semantic relationships between the crack fragments, i.e., it determines whether two bounding boxes should be connected end-to-end. This further guides the improved DETR model to predict a "connection confidence," merging bounding boxes with high confidence into a complete crack in post-processing.

[0035] An electronic device includes: a memory and at least one processor;

[0036] The memory contains computer programs;

[0037] The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the end-to-end lightweight road crack sensing method as described above.

[0038] A computer-readable storage medium storing a computer program that can be executed by a processor to implement the end-to-end lightweight road crack sensing method as described above.

[0039] The end-to-end lightweight road crack sensing method, system, device, and medium of the present invention have the following advantages:

[0040] (I) This invention significantly improves the overall performance of road crack detection through several key technologies. Under the same experimental conditions, using 100 kilometers of actual data collected from a main road as the inference object, compared with the unimproved DETR algorithm and the current mainstream detection model YOLOv11, this invention shows significant advantages in accuracy, recall, and inference speed. The specific beneficial effects are as follows:

[0041] ① Image standardization improves model robustness and generalization ability: By combining size standardization, pixel normalization and adaptive histogram equalization, the distribution of input images is effectively unified, image contrast is enhanced, environmental interference such as lighting and shadows is reduced, and the adaptability and detection stability of the model in different road scenarios are improved.

[0042] ② Multi-scale dynamic annotation method enhances label expressiveness: By adopting composite label construction and adaptive local annotation box mechanism, more refined annotation of cracks of different shapes and scales is achieved, which significantly improves the model's ability to perceive and recognize fine cracks and complex crack patterns, and lays the label foundation for high-precision detection.

[0043] ③ Lightweight road crack perception algorithm achieves efficient and accurate detection: By using a cross-attention enhanced encoder structure, the model's ability to collaboratively model local crack features and global context information is strengthened, improving the quality of feature representation; a serialized interactive decoding and crack integrity inference mechanism is introduced to enhance the decoder's ability to perceive and detect fractured and discontinuous cracks in a structured and coherent manner; a public dataset pre-training and transfer learning incremental training strategy is adopted, which has both general feature extraction and specific scene adaptation capabilities, effectively improving the model's generalization performance; this invention performs excellently in three key indicators: accuracy, recall, and inference speed, as shown in Table 1;

[0044] Table 1 Model Comparison Table

[0045]

[0046] As shown in Table 1, improvements in annotation methods and training strategies can increase the accuracy of traditional models by approximately 10%. While maintaining a high inference speed (35 images / second), this invention achieves an accuracy of 87.1% and a recall of 90.7%, significantly outperforming the comparison algorithms. It is particularly suitable for road inspection scenarios with high requirements for real-time performance and accuracy, and can effectively support embedded platform deployment and practical road inspection engineering applications, providing reliable technical support for intelligent transportation maintenance systems.

[0047] (II) This invention preprocesses the input road image using standardization and constructs training labels using a multi-scale dynamic annotation mechanism. Based on the improved DETR architecture, a cross-attention enhancement module is introduced in the Encoder to improve the perception of local crack features, and a serialized interaction mechanism is used in the Decoder to realize crack integrity reasoning. The generalization performance of the model is improved by combining pre-training on a public dataset with incremental training on a proprietary dataset. Experiments show that this invention achieves an accuracy of 87.1% and a recall of 90.7% while maintaining a detection speed of 35 FPS, which is significantly better than traditional detection methods. It effectively solves the problem of balancing accuracy and real-time performance in crack detection under complex road conditions and provides reliable technical support for intelligent road maintenance. Attached Figure Description

[0048] The invention will be further described below with reference to the accompanying drawings.

[0049] Appendix Figure 1 This is a schematic diagram of image acquisition and encoding.

[0050] Appendix Figure 2 A schematic diagram for image annotation;

[0051] Appendix Figure 3 This is a schematic diagram of the DETR algorithm architecture;

[0052] Appendix Figure 4 This is a schematic diagram of the TransFormer Encoder structure;

[0053] Appendix Figure 5 This is a schematic diagram illustrating the detection results. Detailed Implementation

[0054] The end-to-end lightweight road crack sensing method, system, device, and medium of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1:

[0056] This embodiment provides an end-to-end lightweight road crack detection method, which is as follows:

[0057] S1. Image standardization processing: Acquire road images and perform standardization preprocessing on the acquired road images to obtain standardized preprocessed road images;

[0058] S2. Multi-scale dynamic annotation: A multi-scale dynamic annotation mechanism is used to annotate the standardized preprocessed road images to construct training labels;

[0059] S3. Lightweight road crack perception: Based on the improved DETR model, a cross-attention mechanism is introduced in the Transformer encoder to enhance the perception of local crack features, and a serialized interaction mechanism is used in the Transformer decoder to realize crack integrity reasoning.

[0060] S4. Incremental Training for Transfer Learning: Improve the generalization performance of the model by combining pre-training on public datasets with incremental training on proprietary datasets.

[0061] The image normalization process in step S1 of this embodiment is as follows:

[0062] S101, Size Standardization: The image is uniformly cropped into multiple 600*600 pixel sub-images to ensure that the spatial dimension of each training sample is uniform, so that the improved DETR model can stably learn the spatial feature rules and avoid feature extraction deviation caused by size fluctuations.

[0063] S102, Pixel Normalization: Normalize the pixel values ​​of the sub-image to the range of [0,1] to obtain the pixel-normalized sub-image. This prevents the weight update imbalance of the improved DETR model caused by the difference in pixel value range of different images (such as 0-255 or 0-65535). By scaling the pixel value scale, the improved DETR model can converge quickly in the early stage of training, thereby improving training efficiency.

[0064] S103 Adaptive Histogram Equalization: Histogram equalization is performed on the sub-image after pixel normalization. To address the problem of uneven lighting and low contrast in the original image causing blurred crack features, the gray-level distribution of the image is automatically adjusted to enhance the gray-level difference between the crack and the background, making the crack features more prominent. This helps the improved DETR model to capture target area information more clearly and improves the ability to identify weak features.

[0065] Through the above standardization process, the image data achieves uniform spatial dimension, numerical range, and feature contrast, providing stable and consistent training input for the model and laying a data foundation for subsequent feature learning, as shown in the attached figure. Figure 1 As shown.

[0066] The multi-scale dynamic annotation in step S2 of this embodiment is as follows:

[0067] S201. Composite Label Construction: Cracks longer than 10cm are divided into overlapping local annotation boxes, which are then combined with global annotation boxes to form composite labels (see attached). Figure 2 (a) is the global annotation box, attached Figure 2 (b) is an overlapping local annotation box), which solves the problem that a single global annotation is insufficient to characterize the local details of long cracks. The global annotation box ensures that the improved DETR model grasps the overall shape and location of the crack, while the overlapping local annotation boxes focus on the detailed features of small-scale crack segments, enabling the improved DETR model to learn both global context and local detail information at the same time, thereby improving the ability to detect small defects in long cracks.

[0068] S202. Adaptive Local Annotation Box Size Mechanism: An adaptive annotation box size determination mechanism that integrates crack length and width changes is introduced. For crack sections with large width changes, the local annotation box size is appropriately increased. The adaptive local annotation box size mechanism solves the problem that traditional fixed-size annotation boxes are prone to missing key features when crack width changes abruptly. By dynamically adjusting the annotation box size, it ensures that crack details in the width-changing area are fully included, avoiding feature truncation caused by excessively small annotation boxes. This enables the improved DETR model to learn the crack width change pattern more accurately, further improving the completeness and effectiveness of annotation information.

[0069] DETR (Detection Transformer) is an end-to-end object detection model based on Transformer. It utilizes a self-attention mechanism for global feature modeling and directly outputs detection results through ensemble prediction. This avoids the reliance on complex manual components such as anchor box design and non-maximum suppression (NMS) in traditional methods, simplifying the detection process. The algorithm framework is attached. Figure 3As shown, firstly, the input image is processed by a convolutional neural network (CNN) backbone to extract image feature sets. Then, the feature maps are enhanced with spatial location encoding modules to enhance positional information, flattened, and fed into a Transformer encoder. A self-attention mechanism is used for global context modeling, enabling each pixel to perceive the entire image. Next, the Transformer decoder receives a fixed set of learnable location vectors as input and uses the encoder's output as the key and value, repeatedly interacting through a cross-attention mechanism. Finally, each object query outputs a prediction result (including category and bounding box). Finally, a unique ground truth value is directly assigned to each prediction using bipartite graph matching loss, eliminating the need for non-maximum suppression post-processing and directly outputting the final prediction set. However, the original DETR model suffers from weak perception of small object features, slow convergence speed, and high computational cost, resulting in poor performance when handling road crack detection tasks with subtle shapes and complex backgrounds.

[0070] The lightweight road crack detection in step S3 of this embodiment is as follows:

[0071] S301. Optimize the Transformer encoder structure based on cross-attention enhancement: Introduce a cross-attention mechanism before the feedforward network (FFN) of the Transformer encoder layer. Actively combine external information of the original image texture and scene context. When the improved DETR model processes small cracks, it selectively retrieves and enhances the feature representation related to crack details to obtain auxiliary cues, which can accurately locate the crack area and enhance its own features.

[0072] S302, Serialization Interactive Decoding and Crack Integrity Reasoning: A lightweight serialization interaction module is introduced after the standard Transformer decoder layer. This module performs an additional round of self-attention interaction on the output of all object queries. During this process, the improved DETR model learns the spatial and semantic relationships between crack fragments, i.e., it determines whether two bounding boxes should be connected end-to-end. This further guides the improved DETR model to predict a "connection confidence," and the bounding boxes with high confidence are merged into a complete crack in post-processing.

[0073] In this embodiment, as shown in the appendix Figure 4As shown, in each standard Transformer encoder layer, image features first capture global contextual dependencies through a multi-head self-attention (MSA) mechanism; then, the normalized features are used as queries and cross-attention is calculated with a set of learnable local prior feature keys to further enhance the interaction with relevant features of the target region; finally, the enhanced encoded features are output through a feedforward network for nonlinear transformation and feature integration.

[0074] After embedding a cross-attention mechanism in the Transformer encoder layer, the improved DETR model is guided to focus on salient regions through a set of optimizable local prior feature libraries. During training, the prior features gradually characterize the typical local patterns of cracks (such as edge response, texture irregularity, etc.), enabling the Transformer encoder structure layer to have local structure enhancement capabilities on the basis of global perception.

[0075] The incremental training of transfer learning in step S4 of this embodiment is as follows:

[0076] S401. In the pre-training stage, the improved DETR model is trained on a public dataset of 20,000 road cracks, enabling the improved DETR to learn rich general visual features and possess strong feature extraction and semantic representation capabilities. This significantly improves the sensitivity of the improved DETR model to typical targets of cracks. The improved DETR model achieves a detection accuracy of 88.4% for road targets on the public dataset, but its generalization ability is weak, with an accuracy of only 46.3% in real-world scenarios. However, it already possesses the basic feature extraction capabilities for various types of cracks, laying an important foundation for subsequent domain-adaptive optimization.

[0077] S402. After entering the incremental training phase, a transfer learning mechanism is introduced, and 6,000 proprietary road image data are used to further optimize the pre-trained model. During this process, the weights of the first two layers of the CNN backbone network are frozen to retain the general feature representations learned in the pre-training phase. At the same time, a cosine annealing learning rate scheduling strategy is adopted to dynamically adjust the learning rate during the training process of the improved DETR model, so as to promote the improved DETR model to efficiently adapt to the distribution characteristics of crack features in the target dataset. Through the incremental training strategy, the improved DETR model further enhances its ability to identify and represent the unique morphology and fine structure of road disease cracks on the basis of inheriting the original generalization features.

[0078] S403. Input the road image to be detected into the improved DETR model after incremental training. The improved DETR model outputs the bounding box coordinates of road cracks in the image end-to-end, achieving accurate localization and detection of the damaged area. Based on the acquired crack location information, the improved DETR model not only completes the initial localization but also provides a stable and reliable foundation for further refined classification (such as distinguishing between transverse cracks, longitudinal cracks, alligator cracks, and block cracks). It provides key data support for subsequent damage assessment, maintenance decisions, and long-term pavement performance studies. After stitching the standardized images, the final detection results are obtained, as shown in the attached figure. Figure 5 As shown.

[0079] Example 2:

[0080] This embodiment provides an end-to-end lightweight road crack detection system, which includes:

[0081] The image standardization processing module is used to acquire road images and perform standardization preprocessing on the acquired road images to obtain standardized preprocessed road images;

[0082] The multi-scale dynamic annotation module is used to annotate and construct training labels for standardized preprocessed road images using a multi-scale dynamic annotation mechanism.

[0083] A lightweight road crack perception module is used to improve the perception of local crack features by introducing a cross-attention mechanism in the Transformer encoder based on the improved DETR model, and to realize crack integrity inference by using a serialized interaction mechanism in the Transformer decoder.

[0084] The transfer learning incremental training module is used to improve the generalization performance of models by combining pre-training on public datasets with incremental training on proprietary datasets.

[0085] Original images often exhibit variations in size, pixel value range, and lighting contrast, requiring the model to learn additional data distribution differences during training, increasing training difficulty and reducing feature extraction stability. Before model training, a series of standardization operations transform the image data into a unified format suitable for model training. The image standardization module's workflow is as follows:

[0086] (1) Size standardization: The image is uniformly cropped into multiple 600*600 pixel sub-images to ensure that the spatial dimension of each training sample is uniform, so that the model can stably learn the spatial feature rules and avoid feature extraction deviation caused by size fluctuation.

[0087] (2) Pixel normalization: Normalize the pixel values ​​to the range of [0,1] to prevent the model weight update imbalance caused by the difference in the range of pixel values ​​of different images (such as 0-255 or 0-65535). By scaling the pixel value scale, the model can converge quickly in the early stage of training, thus improving training efficiency.

[0088] (3) Adaptive Histogram Equalization: Histogram equalization is performed on the image to address the problem of uneven lighting and low contrast in the original image, which leads to blurred crack features. The gray-level distribution of the image is automatically adjusted to enhance the gray-level difference between the crack and the background, making the crack features more prominent. This helps the model to capture target area information more clearly and improves the ability to recognize weak features.

[0089] Through the above standardization process, the image data achieves uniform spatial dimension, numerical range, and feature contrast, providing stable and consistent training input for the model and laying a data foundation for subsequent feature learning, as shown in the attached figure. Figure 1 As shown.

[0090] Traditional single-labeling methods lack sufficient feature coverage for complex targets such as long cracks and cracks with varying widths, resulting in low sensitivity of the model to small-scale crack features and a tendency for missed or false detections. This method achieves refined labeling of crack features through composite label construction and adaptive bounding box adjustment, thereby improving the model's sensitivity and detection accuracy for small-scale and cracks with varying widths. The specific working process of the multi-scale dynamic labeling module in this embodiment is as follows:

[0091] (1) Composite label construction: Cracks longer than 10cm are divided into overlapping local annotation boxes, which are then combined with global annotation boxes to form composite labels (see attached). Figure 2 (a) is the global annotation box, attached Figure 2 (b) Overlapping local annotation boxes). This method solves the problem that a single global annotation is insufficient for depicting the local details of long cracks. The global annotation boxes ensure that the model grasps the overall shape and location of the crack, while the overlapping local annotation boxes focus on the detailed features of small-scale crack segments, enabling the model to learn both global context and local detail information at the same time, thereby improving the ability to detect small defects in long cracks.

[0092] (2) Adaptive Local Annotation Box Size Mechanism: An adaptive annotation box size determination mechanism that integrates crack length and width variations is introduced, appropriately increasing the size of the local annotation box for crack sections with significant width variations. This mechanism solves the problem that traditional fixed-size annotation boxes easily miss key features when crack width changes abruptly. By dynamically adjusting the annotation box size, it ensures that crack details in the width-varying areas are fully included, avoiding feature truncation due to excessively small annotation boxes. This allows the model to learn the crack width variation pattern more accurately, further improving the completeness and effectiveness of the annotation information.

[0093] Traditional manual inspection methods are not only inefficient and costly, but also highly susceptible to subjective factors. In recent years, the demand for intelligent road inspection has been increasing. However, road cracks are characterized by complex morphology, extremely small size, and strong interference in the detection environment. Furthermore, practical applications place stringent demands on the computational efficiency of algorithms. These factors collectively lead to the poor performance of existing detection models, making it difficult to accurately capture and correlate weak crack features. They are also prone to false detections due to complex background interference and insufficient speed on high-resolution images, failing to meet the demands for real-time, efficient, and automated detection. To achieve high-accuracy and high-efficiency intelligent detection of road cracks, this paper adopts the DETR (DetectionTransformer) model based on the Transformer architecture as the basic framework.

[0094] DETR (Detection Transformer) is an end-to-end object detection model based on Transformer. It utilizes a self-attention mechanism for global feature modeling and directly outputs detection results through ensemble prediction. This avoids the reliance on complex manual components such as anchor box design and non-maximum suppression (NMS) in traditional methods, simplifying the detection process. The algorithm framework is attached. Figure 3 As shown in the figure, firstly, the input image is processed by a convolutional neural network (CNN) backbone to extract image feature sets. Then, the feature maps are enhanced with spatial location encoding modules to enhance positional information, flattened, and fed into a Transformer encoder. A self-attention mechanism is used for global context modeling, enabling each pixel to perceive the entire image information. Next, the Transformer decoder receives a fixed number of learnable location vectors as input and uses the encoder's output as the key and value, repeatedly interacting through a cross-attention mechanism. Finally, each object query outputs a prediction result (including category and bounding box). Finally, a unique ground truth value is directly assigned to each prediction using bipartite graph matching loss, eliminating the need for non-maximum suppression post-processing and directly outputting the final prediction set. However, the original DETR model suffers from weak perception of small object features, slow model convergence, and high computational overhead, resulting in poor performance when handling road crack detection tasks with subtle morphology and complex backgrounds. To address these issues, this embodiment proposes a lightweight road crack perception algorithm that improves the DETR framework.

[0095] (1) Encoder structure optimization based on cross-attention enhancement

[0096] Traditional Transformer Encoders' self-attention only focuses on internal feature dependencies. However, the features of tiny road cracks are weak and easily obscured by the background. To enhance the extraction of tiny crack features and suppress background interference, this paper introduces a cross-attention mechanism before the feedforward network (FFN) in the Transformer Encoder layer. This mechanism actively combines external information such as the original image texture and scene context, allowing the model to selectively retrieve and enhance feature representations related to crack details when processing tiny cracks. This provides auxiliary cues, accurately locating crack regions while strengthening their own features. The Encoder structure is attached. Figure 4 As shown, in each standard Encoder layer, image features first capture global contextual dependencies through a multi-head self-attention (MSA) mechanism; then, the normalized features are used as queries and cross-attention calculations are performed with a set of learnable local prior feature keys to further enhance the interaction with relevant features of the target region; finally, nonlinear transformation and feature integration are performed through FFN to output the enhanced encoded features.

[0097] This encoder structure, after embedding a cross-attention mechanism, guides the model to focus on salient regions through a set of optimizable local prior features. These prior features gradually characterize typical local patterns of cracks (such as edge responses and texture irregularities) during training, enabling the encoder to enhance local structure on top of global perception. This design significantly improves the representational quality of the encoder's output features, making it more sensitive to minute crack features and more robust to background interference such as lighting changes and oil stains. It also improves model convergence speed and detection accuracy, providing a clear direction for optimization, allowing the model to learn the essential features of cracks more quickly, while maintaining a simple and compact model structure. The increase in computational cost is completely controllable, meeting the requirements of lightweight design.

[0098] (2) Serialization Interactive Decoding and Crack Integrity Reasoning

[0099] Object queries in the standard Decoder layer are processed in parallel and are independent of each other, lacking explicit interaction. This causes the multiple crack boxes predicted by the model to be unaware of each other's existence, potentially outputting a series of broken, overlapping fragment boxes instead of complete crack shapes. To solve the above problem, this paper introduces a lightweight serialization interaction module after the standard Decoder layer.

[0100] This module performs an additional round of self-attention interaction on the outputs of all Object Queries. During this process, the model can learn the spatial and semantic relationships between crack fragments (determining whether two bounding boxes should be connected end-to-end), further guiding the model to predict a "connection confidence," and merging bounding boxes with high confidence into a complete crack in post-processing.

[0101] Through the above improvements, the correlation between crack segments is explicitly modeled, enabling the model to reason about crack integrity. Its output is no longer an isolated box, but a set of associated predictions that can express the overall topological structure of the crack, significantly improving the consistency and accuracy of the detection results.

[0102] (3) Pre-training on public datasets and incremental training using transfer learning

[0103] In road inspection tasks, if a model is directly trained using a limited-scale domain-specific dataset, it often struggles to learn sufficient and robust feature representations. This can lead to problems such as poor fitting of the training set distribution, weak generalization ability, and insufficient extraction of features from small targets, resulting in a significant decrease in detection performance in real-world scenarios. This embodiment employs a strategy of pre-training on a public dataset and incremental training using transfer learning.

[0104] First, during the pre-training phase, the model was trained on a public dataset of 20,000 road crack images, enabling it to learn rich general visual features and possess strong feature extraction and semantic representation capabilities. This significantly improved the model's sensitivity to typical targets such as cracks. The model achieved an accuracy of 88.4% in detecting road targets on the public dataset, but its generalization ability was weak, with an accuracy of only 46.3% in real-world scenarios. However, it already possessed the basic feature extraction capabilities for various types of cracks, laying an important foundation for subsequent domain-adaptive optimization.

[0105] After entering the incremental training phase, this study introduces a transfer learning mechanism, using 6000 proprietary road image data to further optimize the pre-trained model. During this process, the image is frozen... Figure 3 The weights of the first two layers of the CNN backbone network shown are used to retain the general feature representations learned during the pre-training stage. Simultaneously, a cosine annealing learning rate scheduling strategy is employed to dynamically adjust the learning rate during model training, promoting efficient adaptation of the model to the distribution characteristics of crack features in the target dataset. Through this incremental training strategy, the model, while inheriting the original generalized features, further enhances its ability to identify and represent the unique morphology and fine structure of road disease cracks.

[0106] The road image to be detected is input into the incrementally trained DETR model. This model can output the bounding box coordinates of road cracks in the image end-to-end, achieving accurate localization and detection of the affected area. Based on the acquired crack location information, the model not only completes the initial localization but also provides a stable and reliable foundation for further refined classification (such as distinguishing between transverse cracks, longitudinal cracks, alligator cracks, and block cracks). This result provides key data support for subsequent damage assessment, maintenance decisions, and long-term pavement performance studies. The detection results after standardizing and stitching the images are shown in the attached figure. Figure 5 As shown.

[0107] Example 3:

[0108] This embodiment also provides an electronic device, including: a memory and a processor;

[0109] The memory stores the instructions executed by the computer.

[0110] The processor executes computer execution instructions stored in the memory, causing the processor to perform the end-to-end lightweight road crack sensing method in any embodiment of the present invention.

[0111] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor or any conventional processor.

[0112] Memory is used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, memory can also include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart memory cards (SMC), secure digital cards (SD cards), flash memory cards, at least one disk storage device, flash memory devices, or other volatile solid-state storage devices.

[0113] Example 4:

[0114] This embodiment also provides a computer-readable storage medium storing a plurality of instructions, which are loaded by a processor to cause the processor to execute the end-to-end lightweight road crack sensing method of any embodiment of the present invention. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the above embodiments can be provided, and the computer (or CPU or MPU) of the system or apparatus can read and execute the program code stored in the storage medium.

[0115] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0116] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0117] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0118] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An end-to-end lightweight road crack detection method, characterized in that, The method is as follows: Image standardization processing: Acquire road images and perform standardization preprocessing on the acquired road images to obtain standardized preprocessed road images; Multi-scale dynamic annotation: A multi-scale dynamic annotation mechanism is used to annotate the standardized preprocessed road images to construct training labels; Lightweight road crack perception: Based on the improved DETR model, a cross-attention mechanism is introduced in the Transformer encoder to enhance the perception of local crack features, and a serialized interaction mechanism is used in the Transformer decoder to realize crack integrity reasoning. Incremental training for transfer learning: This strategy improves the generalization performance of a model by combining pre-training on a public dataset with incremental training on a proprietary dataset.

2. The end-to-end lightweight road crack detection method according to claim 1, characterized in that, The image normalization process is as follows: Size standardization: The image is uniformly cropped into multiple 600*600 pixel sub-images to ensure that the spatial dimension of each training sample is uniform; Pixel normalization: Normalize the pixel values ​​of the sub-image to the range of [0,1] to obtain the pixel-normalized sub-image. By scaling the pixel value scale, the improved DETR model can converge quickly in the early stage of training. Adaptive Histogram Equalization: Histogram equalization is performed on the sub-image after pixel normalization to automatically adjust the gray-level distribution of the image, enhance the gray-level difference between the crack and the background, make the crack features more prominent, and help the improved DETR model capture target area information more clearly.

3. The end-to-end lightweight road crack detection method according to claim 1, characterized in that, The multi-scale dynamic annotation is as follows: Composite label construction: Cracks with a length >10cm are segmented into overlapping local bounding boxes, which are combined with global bounding boxes to form composite labels. The global bounding boxes ensure that the improved DETR model grasps the overall shape and location of the cracks, while the overlapping local bounding boxes focus on the detailed features of small-scale crack segments, enabling the improved DETR model to learn both global context and local detail information at the same time. Adaptive Local Annotation Box Size Mechanism: An adaptive annotation box size determination mechanism that integrates crack length and width variations is introduced. For crack sections with large width variations, the local annotation box size is appropriately increased. By dynamically adjusting the annotation box size, the crack details in the width variation area are fully included, avoiding feature truncation caused by excessively small annotation boxes. This enables the improved DETR model to learn the crack width variation pattern more accurately, further enhancing the completeness and effectiveness of annotation information.

4. The end-to-end lightweight road crack detection method according to claim 1, characterized in that, The specific details of lightweight road crack detection are as follows: Optimize the Transformer encoder structure based on cross-attention enhancement: Introduce a cross-attention mechanism before the feedforward network of the Transformer encoder layer to actively combine external information such as the original texture of the image and the scene context. When the improved DETR model processes small cracks, it selectively retrieves and enhances the feature representation related to crack details to obtain auxiliary cues, which can accurately locate the crack area and enhance its own features. Serialization Interactive Decoding and Crack Integrity Reasoning: A lightweight serialization interaction module is introduced after the standard Transformer decoder layer. This module performs an additional round of self-attention interaction on the output of all object queries. During this process, the improved DETR model learns the spatial and semantic relationships between crack fragments, i.e., it determines whether two bounding boxes should be connected end-to-end. This further guides the improved DETR model to predict a "connection confidence," and the bounding boxes with high confidence are merged into a complete crack in post-processing.

5. The end-to-end lightweight road crack detection method according to claim 4, characterized in that, In each standard Transformer encoder layer, image features first capture global contextual dependencies through a multi-head self-attention mechanism; then, the normalized features are used as queries and cross-attention is performed with a set of learnable local prior feature keys to further enhance the interaction with relevant features of the target region; finally, the enhanced encoded features are output through a feedforward network for nonlinear transformation and feature integration. After embedding the cross-attention mechanism in the Transformer encoder layer, the improved DETR model is guided to focus on salient regions through a set of optimizable local prior features. During training, the prior features gradually characterize the typical local patterns of cracks, enabling the Transformer encoder structure layer to have local structure enhancement capabilities on the basis of global perception.

6. The end-to-end lightweight road crack detection method according to claim 1, characterized in that, The incremental training of transfer learning is as follows: During the pre-training phase, the improved DETR model was trained on a public dataset of 20,000 road crack images, enabling the improved DETR to learn rich general visual features, possess strong feature extraction and semantic representation capabilities, and significantly improve the sensitivity of the improved DETR model to typical targets of cracks. After entering the incremental training phase, a transfer learning mechanism was introduced, and the pre-trained model was further optimized using 6,000 proprietary road image data. During this process, the weights of the first two layers of the CNN backbone network were frozen to retain the general feature representations learned in the pre-training phase. At the same time, a cosine annealing learning rate scheduling strategy was adopted to dynamically adjust the learning rate during the training process of the improved DETR model, promoting the improved DETR model to efficiently adapt to the distribution characteristics of crack features in the target dataset. Through the incremental training strategy, the improved DETR model, while inheriting the original generalization features, further enhances its ability to identify and represent the unique morphology and fine structure of road disease cracks. The road image to be detected is input into the improved DETR model after incremental training. The improved DETR model outputs the bounding box coordinates of road cracks in the image end-to-end, realizing accurate localization and detection of the disease area. Based on the obtained crack location information, the improved DETR model not only completes the initial localization, but also provides a stable and reliable foundation for further fine classification. After the standardized images are stitched together, the final detection result is obtained.

7. An end-to-end lightweight road crack detection system, characterized in that, The system includes: The image standardization processing module is used to acquire road images and perform standardization preprocessing on the acquired road images to obtain standardized preprocessed road images; The multi-scale dynamic annotation module is used to annotate and construct training labels for standardized preprocessed road images using a multi-scale dynamic annotation mechanism. A lightweight road crack perception module is used to improve the perception of local crack features by introducing a cross-attention mechanism in the Transformer encoder based on the improved DETR model, and to realize crack integrity inference by using a serialized interaction mechanism in the Transformer decoder. The transfer learning incremental training module is used to improve the generalization performance of models by combining pre-training on public datasets with incremental training on proprietary datasets.

8. The end-to-end lightweight road crack detection system according to claim 7, characterized in that, The lightweight road crack detection module includes: The optimization submodule introduces a cross-attention mechanism before the feedforward network of the Transformer encoder layer. This mechanism actively combines external information from the original image texture and scene context. When processing minute cracks, the improved DETR model selectively retrieves and enhances feature representations related to crack details, acquiring auxiliary cues to accurately locate crack regions while strengthening its own features. In each standard Transformer encoder layer, image features first capture global contextual dependencies through a multi-head self-attention mechanism. Subsequently, normalized features are used as queries and cross-attention calculations are performed with a set of learnable local prior feature keys to further enhance the interaction with relevant features of the target region. Finally, the feedforward network performs nonlinear transformation and feature integration to output enhanced encoded features. After embedding the cross-attention mechanism in the Transformer encoder layer, a set of optimizable local prior feature libraries guides the improved DETR model to focus on salient regions. During training, the prior features gradually characterize typical local patterns of cracks, enabling the Transformer encoder structure layer to possess local structural enhancement capabilities on the basis of global perception. The inference submodule introduces a lightweight serialization interaction module after the standard Transformer decoder layer. This module performs an additional round of self-attention interaction on the output of all object queries. During this process, the improved DETR model learns the spatial and semantic relationships between the crack fragments, i.e., it determines whether two bounding boxes should be connected end-to-end. This further guides the improved DETR model to predict a "connection confidence," merging the bounding boxes with high confidence into a complete crack in post-processing.

9. An electronic device, characterized in that, include: Memory and at least one processor; The memory contains computer programs; The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the end-to-end lightweight road crack sensing method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the end-to-end lightweight road crack detection method as described in any one of claims 1 to 6.

Images