Lightweight road crack detection method and system

Abstract

The embodiment of the invention discloses a lightweight road crack detection method and system, and the method comprises the following steps: obtaining a road surface image, inputting the road surface image into a trunk feature extraction network, so as to extract a multi-scale basic feature map set, the multi-scale basic feature map set comprises a plurality of basic feature maps output by the trunk feature extraction network under at least two different spatial resolutions; according to the scale distribution features of the crack target, adaptively selecting an intermediate scale feature map of which the spatial resolution is in a preset range from the plurality of basic feature maps, and performing coupling feature modeling processing on the intermediate scale feature map to generate a collaborative feature map; performing lightweight feature compensation processing by taking the collaborative feature map as an input to obtain an output feature; executing spatial re-calibration processing on the output features; and multi-scale target detection is executed based on the re-calibrated output features, a crack detection result is obtained, and detection branches of different scales share the same group of detection head parameters.

Description

Technical Field

[0001] This application belongs to the field of image detection technology, specifically relating to a lightweight method and system for detecting road cracks. Background Technology

[0002] With the accelerating pace of urbanization, roads, as the infrastructure of the transportation system, directly impact urban operational efficiency and traffic safety. Road surface cracks, as the most common type of early damage, are characterized by their sudden onset and rapid spread. If not identified and repaired in time, they will further develop into serious defects such as potholes, subsidence, and fractures, even causing irreversible damage to the road structure. Therefore, accurate and efficient automated crack detection is a crucial step in achieving intelligent road maintenance and ensuring traffic safety.

[0003] Traditional pavement crack detection relies primarily on manual inspections, where the location and type of cracks are determined visually. This method is costly, inefficient, and subjective, making it unsuitable for the efficient maintenance needs of today's large-scale road networks. To improve the automation of detection, existing technologies are gradually incorporating image processing techniques to assist in the initial identification of crack areas. For example, threshold segmentation and image morphology methods are used for crack preprocessing and contour extraction. While these methods reduce labor costs to some extent, they lack adaptability to environmental complexity and the diversity of crack morphologies, resulting in insufficient overall robustness and difficulty in meeting the complex detection requirements of real-world road scenarios.

[0004] Application content

[0005] The purpose of this application is to provide a lightweight road crack detection method and system to address the shortcomings of existing technologies in meeting the complex detection needs of real-world road scenarios.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows:

[0007] Firstly, a lightweight road crack detection method is provided, comprising the following steps:

[0008] S1. Acquire a road surface image and input the road surface image into a backbone feature extraction network to extract a multi-scale basic feature map set, wherein the multi-scale basic feature map set includes multiple basic feature maps output by the backbone feature extraction network at at least two different spatial resolutions.

[0009] S2. Based on the scale distribution characteristics of the crack target, adaptively select an intermediate scale feature map with a spatial resolution within a preset range from the multiple basic feature maps, and perform coupled feature modeling processing on the intermediate scale feature map to generate a collaborative feature map.

[0010] The coupled feature modeling process includes: dividing the intermediate-scale feature map into at least two feature branches along the channel dimension; in the first feature branch, performing local convolution operations on the intermediate-scale feature map to obtain local feature representations, and generating query vectors, key vectors, and value vectors based on the local feature representations; obtaining global feature representations through multi-head self-attention operations, wherein the local feature representations are used to modulate the attention weights in the multi-head self-attention operations to suppress global responses unrelated to the crack structure; in the second feature branch, performing parallel feature transfer on the intermediate-scale feature map to preserve the original spatial structure information; and fusing the local feature representations, the global feature representations, and the intermediate-scale feature map to generate the collaborative feature map.

[0011] S3. Using the collaborative feature map as input, perform lightweight feature compensation processing to obtain the output features;

[0012] The lightweight feature compensation process includes: generating a basic feature subset through a first convolution operation; generating a redundant feature subset based on the basic feature subset through a channel-wise linear transformation or element-wise linear mapping with a computational complexity lower than that of the first convolution operation; and concatenating the basic feature subset with the redundant feature subset to obtain the output feature.

[0013] S4. Perform spatial recalibration processing on the output features to enhance the saliency of the crack region and suppress background texture interference;

[0014] The spatial recalibration process includes: generating a spatial weight map and weighting the output features based on the spatial weight map;

[0015] S5. Perform multi-scale target detection based on the recalibrated output features to obtain crack detection results. Detection branches of different scales share the same set of detection head parameters.

[0016] Secondly, a lightweight road crack detection system is provided, comprising:

[0017] An acquisition module is used to acquire a road surface image and input the road surface image into a backbone feature extraction network to extract a multi-scale basic feature map set, wherein the multi-scale basic feature map set includes multiple basic feature maps output by the backbone feature extraction network at at least two different spatial resolutions.

[0018] The generation module is used to adaptively select intermediate-scale feature maps with spatial resolution within a preset range from the multiple basic feature maps according to the scale distribution characteristics of the crack target, and perform coupled feature modeling processing on the intermediate-scale feature maps to generate collaborative feature maps.

[0019] The coupled feature modeling process includes: dividing the intermediate-scale feature map into at least two feature branches along the channel dimension; in the first feature branch, performing local convolution operations on the intermediate-scale feature map to obtain local feature representations, and generating query vectors, key vectors, and value vectors based on the local feature representations; obtaining global feature representations through multi-head self-attention operations, wherein the local feature representations are used to modulate the attention weights in the multi-head self-attention operations to suppress global responses unrelated to the crack structure; in the second feature branch, performing parallel feature transfer on the intermediate-scale feature map to preserve the original spatial structure information; and fusing the local feature representations, the global feature representations, and the intermediate-scale feature map to generate the collaborative feature map.

[0020] The compensation module is used to perform lightweight feature compensation processing on the collaborative feature map as input to obtain the output features;

[0021] The lightweight feature compensation process includes: generating a basic feature subset through a first convolution operation; generating a redundant feature subset based on the basic feature subset through a channel-wise linear transformation or element-wise linear mapping with a computational complexity lower than that of the first convolution operation; and concatenating the basic feature subset with the redundant feature subset to obtain the output feature.

[0022] The processing module is used to perform spatial recalibration processing on the output features to enhance the saliency of the crack region and suppress background texture interference;

[0023] The spatial recalibration process includes: generating a spatial weight map and weighting the output features based on the spatial weight map;

[0024] The detection module is used to perform multi-scale target detection based on the recalibrated output features to obtain crack detection results. Detection branches of different scales share the same set of detection head parameters.

[0025] This application's embodiments, by introducing a feature selection mechanism based on crack scale distribution, a global modeling method with local structural constraints, and a lightweight feature compensation and spatial recalibration strategy, can improve the targeting of crack characterization, enhance the ability to perceive slender cracks, prevent crack details from being weakened during the modeling process, and effectively reduce computational complexity while ensuring detection accuracy. This significantly improves the accuracy, robustness, and engineering practicality of road crack detection in complex environments. Attached Figure Description

[0026] Figure 1 This is a flowchart of a lightweight road crack detection method provided in an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of the structure of an improved lightweight detection model provided in an embodiment of this application;

[0028] Figure 3 This is a schematic diagram of a multi-head self-attention mechanism for a Transformer provided in an embodiment of this application;

[0029] Figure 4 This is a schematic diagram of the structure of a CSPMHSA module provided in an embodiment of this application;

[0030] Figure 5 This is a schematic diagram of the core design of GhostNet provided in the embodiments of this application;

[0031] Figure 6 This is a schematic diagram of the CSPGhost convolutional structure based on the GhostNet concept provided in the embodiments of this application;

[0032] Figure 7 This is a schematic diagram of the large kernel space attention mechanism provided in an embodiment of this application;

[0033] Figure 8 This is a schematic diagram of the structure of a lightweight detection head provided in an embodiment of this application;

[0034] Figure 9 This is a schematic diagram of the structure of a lightweight road crack detection system provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] To address the shortcomings of existing technologies, this application selects YOLOv8, which has a relatively balanced performance in detection accuracy and inference speed, as the baseline model in the YOLO (You Only Look Once) series. It designs and proposes a lightweight road crack detection model based on the improved YOLOv8, which effectively improves crack detection accuracy while maintaining real-time performance.

[0037] Specifically, this application's embodiments innovatively integrate the advantages of Transformer structures and CNNs to design a CSPMHSA convolutional structure, effectively improving the detection accuracy of road surface cracks of different morphologies and achieving a synergistic enhancement of local feature extraction and global modeling capabilities. Addressing the issue of traditional methods relying on large parameters to improve accuracy, this application's embodiments propose a CSPGhost module, which significantly reduces model complexity while enhancing the model's feature fusion effect, achieving the dual goals of lightweight design and performance improvement. For road surface crack detection in complex environments, this application's embodiments introduce an LSKA mechanism, significantly enhancing the model's robustness in recognizing crack features in complex backgrounds and improving detection reliability in complex scenarios. By optimizing the detection head structure, this application's embodiments design and propose a lightweight detection head, LBoxHead, effectively enhancing the characterization ability of crack features and further improving overall detection performance.

[0038] The following description, in conjunction with the accompanying drawings, details a lightweight road crack detection method provided by this application through specific embodiments and application scenarios.

[0039] like Figure 1 The diagram shown is a flowchart of a lightweight road crack detection method provided in an embodiment of this application. The method includes the following steps:

[0040] Step 101: Obtain a road surface image and input the road surface image into the backbone feature extraction network to extract a multi-scale basic feature map set.

[0041] The backbone feature extraction network can employ a convolutional neural network structure, such as a lightweight residual network or a depthwise separable convolutional network. Through a multi-level downsampling structure, it outputs multiple basic feature maps at at least two different spatial resolutions, forming a multi-scale basic feature map set {F}. (1) ,F (2) ,…,F (l)}. Multi-scale basic feature map set {F (1) ,F (2) ,…,F (l) This includes multiple base feature maps output by the backbone feature extraction network at at least two different spatial resolutions. (l) This represents the basic feature map at the l-th scale, with different downsampling ratios corresponding to different scales.

[0042] Step 102: Based on the scale distribution characteristics of the crack target, adaptively select an intermediate scale feature map with a spatial resolution within a preset range from the multiple basic feature maps, and perform coupled feature modeling processing on the intermediate scale feature map to generate a collaborative feature map.

[0043] Given that crack targets are characterized by being slender, continuous, and concentrated at medium resolution, this embodiment does not directly use all scale features, but instead performs crack response analysis on the basic feature maps at each scale.

[0044] Specifically, the crack response intensity score of each basic feature map can be calculated based on the edge continuity, linear structure density, and gradient consistency of each basic feature map; feature maps whose scores meet preset conditions are selected from the multiple basic feature maps as intermediate scale feature maps, and the preset conditions are determined based on the scale distribution characteristics of the crack target.

[0045] In this embodiment, the following formula can be used to calculate the basic feature map F at the l-th scale. (l) Spatial gradient calculation is performed to obtain the gradient magnitude map G. (l) (x,y):

[0046]

[0047] The gradient magnitude map is thinned and connected component analysis is performed, and the edge continuity of the basic feature map at the l-th scale is determined by the following formula:

[0048]

[0049] Among them, E (l) For edge continuity, E (l) The larger the value of N, the more the corresponding basic feature map tends to represent a slender, continuous crack structure; l L represents the number of edge segments at the l-th scale; k A is the length of the k-th edge segment; k Let be the area enclosed by the k-th edge segment;

[0050] For the basic feature map F at the l-th scale (l) Directional filtering is performed to obtain a set of directional response maps. Based on the set of directional response maps, the linear structure density of the basic feature map at the l-th scale is determined using the following formula:

[0051]

[0052] Among them, H l and W l These represent the height and width of the base feature map at the l-th scale, respectively. Different scales correspond to different downsampling ratios, satisfying H1>H2>…>H L W1>W2>…>W L ; The set of directional response maps; D (l) The density of the linear structure represents the response intensity to the linear structure per unit area; This indicates that the maximum value is taken in all directions to highlight the linear structure;

[0053] The gradient direction consistency of the base feature map at the l-th scale is calculated using the following formula:

[0054]

[0055] Among them, H l and W l These are the height and width of the base feature map at the l-th scale, respectively; φ represents the local neighborhood centered at position (x, y); x,y C represents the gradient direction at position (x, y); (l) For gradient direction consistency, C (l) A larger value indicates a more consistent gradient direction, which better matches the characteristics of crack propagation.

[0056] The basic feature map F at the l-th scale is obtained using the following formula. (l) The edge continuity, linear structure density, and gradient consistency are weighted and fused to obtain the basic feature map F at the l-th scale. (l) Crack response strength score:

[0057] S (l) =αE (l) +βD (l) +γC (l)

[0058] Among them, E (l) For edge continuity, D (l) For linear structure density, C (l) To ensure gradient direction consistency, α, β, and γ are the first, second, and third weighting coefficients, respectively, and α+β+γ=1. α, β, and γ can be adaptively adjusted according to different road surface materials.

[0059] Accordingly, the normalized scaling factor for the l-th scale can be calculated using the following formula:

[0060]

[0061] Where, r (l) H is the normalized scaling factor for the l-th scale. l and W l H1 and W1 are the height and width of the base feature map at the l-th scale, respectively; H1 and W1 are the height and width of the base feature map at the first scale, respectively.

[0062] Based on the center location of the target scale distribution of the crack, the scale penalty function is determined using the following formula:

[0063] P(l) =exp(-λ(logr) (l) -μ) 2 )

[0064] Among them, P (l) Let r be the scaling penalty function. (l) λ is the normalized scale factor for the l-th scale, λ is the penalty intensity coefficient which can change dynamically with the input image resolution, and μ is the center position of the crack target scale distribution.

[0065] The weighted scale penalty score of the base feature map at the l-th scale is calculated using the following formula:

[0066]

[0067] in, The weighted scale penalty score, P, of the base feature map at the l-th scale. (l) Let S be the scaling penalty function. (l) The crack response intensity score is given for the basic feature map at the l-th scale.

[0068] If the weighted scale penalty score of the base feature map at the l-th scale is greater than or equal to a preset threshold, then the base feature map at the l-th scale is used as the intermediate scale feature map.

[0069] Further, the coupled feature modeling process includes: dividing the intermediate-scale feature map into at least two feature branches along the channel dimension; in the first feature branch, performing local convolution operations on the intermediate-scale feature map to obtain local feature representations, and generating query vectors, key vectors, and value vectors based on the local feature representations; obtaining global feature representations through multi-head self-attention operations, wherein the local feature representations are used to modulate the attention weights in the multi-head self-attention operations to suppress global responses unrelated to the crack structure; in the second feature branch, performing parallel feature transfer on the intermediate-scale feature map to preserve the original spatial structure information; and fusing the local feature representations, the global feature representations, and the intermediate-scale feature map to generate the collaborative feature map.

[0070] In the multi-head self-attention operation, the weights of each attention head are modulated by the structure-guided weights generated by the local feature representation to enhance the response along the crack extension direction and suppress global associations in non-crack regions. The structure-guided weights act on at least one of the query vector, key vector, or attention output.

[0071] or,

[0072] The weights of each attention head are modulated by structural constraint factors generated from the local feature representation to enhance the global response consistent with the local structure of the crack and suppress attention allocation to irrelevant regions. The structural constraint factors are generated based on the crack's directional information, connectivity information, or a combination thereof in the local feature representation and are used to weight or gate the attention weights.

[0073] Specifically, the structure-guided weights can be generated based on the local feature representation using the following formula:

[0074] W s =σ(Conv(F) local ))

[0075] Among them, W s The structure guides the weights, σ ​​is the Sigmoid activation function, Conv(·) is a 1×1 convolution, and F... local Representation as local features;

[0076] Based on the structure-guided weights, the weights of each attention head in the multi-head self-attention operation are modulated using the following formula:

[0077]

[0078] Among them, W s The structure guides the weights, where Q is the query vector, K is the key vector, V is the value vector, and d is the weights. k The dimension of the key vector;

[0079] Alternatively, based on the local feature representation, the structural constraint factor can be generated using the following formula:

[0080] C s =σ(Pool(F) local ))

[0081] Among them, C s σ is the structural constraint factor, σ is the Sigmoid activation function, Pool(·) is the pooling operation, and F local Representation as local features;

[0082] Based on the structural constraint factor, the weights of each attention head in the multi-head self-attention operation are modulated using the following formula:

[0083]

[0084] Among them, C s Let Q be the structural constraint factor generated from the local feature representation, K be the key vector, V be the value vector, and d be the value vector. k The dimension of the key vector.

[0085] Step 103: Using the collaborative feature map as input, perform lightweight feature compensation processing to obtain the output features.

[0086] The lightweight feature compensation process includes: generating a basic feature subset through a first convolution operation; generating a redundant feature subset based on the basic feature subset through a channel-wise linear transformation or element-wise linear mapping with a computational complexity lower than that of the first convolution operation; and concatenating the basic feature subset with the redundant feature subset to obtain the output feature.

[0087] Step 104: Perform spatial recalibration on the output features to enhance the saliency of the crack region and suppress background texture interference.

[0088] The spatial recalibration process includes generating a spatial weight map and weighting the output features based on the spatial weight map.

[0089] Step 105: Perform multi-scale target detection based on the recalibrated output features to obtain crack detection results. Detection branches of different scales share the same set of detection head parameters.

[0090] This application's embodiments, by introducing a feature selection mechanism based on crack scale distribution, a global modeling method with local structural constraints, and a lightweight feature compensation and spatial recalibration strategy, can improve the targeting of crack characterization, enhance the ability to perceive slender cracks, prevent crack details from being weakened during the modeling process, and effectively reduce computational complexity while ensuring detection accuracy. This significantly improves the accuracy, robustness, and engineering practicality of road crack detection in complex environments.

[0091] In this application embodiment, to address the problems of insufficient accuracy in pavement crack detection and excessive model complexity, an improved lightweight detection model (Improved-YOLO) is proposed, such as... Figure 2 As shown. First, in the Neck section, this embodiment designs a novel convolutional module (CSPMHSA) based on the Transformer architecture, aiming to combine the local feature extraction capabilities of CNNs with the global modeling advantages of Transformers, thereby improving crack detection accuracy. Second, addressing the limitation of traditional methods that increase parameter scale to improve detection accuracy, this embodiment introduces the CSPGhost module, which reduces model complexity while enhancing the feature fusion effect in the Neck section. Furthermore, to improve the robustness of crack recognition in complex environments, this embodiment also introduces the LSKA mechanism. Finally, in the Head section, this embodiment designs a lighter LBoxHead detection head to enhance the model's ability to represent crack features.

[0092] like Figure 3As shown, this is the Multi-Head Self-Attention (MHSA) mechanism of the Transformer, which models the system by calculating the global dependencies between features. Assuming an input X, the 2D feature map is first processed using a 1×1 convolution to generate the query (q) and key (q) respectively. k ) and value (v). Next, the spatial dimensions of the feature map are decoupled, respectively at height (R) and value (v). h ) and width (R) w In two dimensions, a relative positional representation is learned for each position, enabling attention to simultaneously capture content association and positional relationships. Then, the attention score is calculated based on content-content interaction (qk). T ) and content-location interaction (qr) T Together, they constitute the attention logits, which is qk. T +qr T After multi-head splitting and aggregation, the global aggregated feature Y is output.

[0093] However, although MHSA can effectively model long-range dependencies at the global level, its global computation method is inefficient in capturing local fine features and is relatively lacking in modeling local spatial correlations.

[0094] To address the issue of insufficient detection accuracy for road cracks of different shapes, this application introduces a CSPMHSA module, the structure of which is as follows: Figure 4 As shown, after the input features are initially encoded by the CBS module to obtain channel and spatial features, the feature flow is decoupled into two parallel branches through a Split operation. The main branch introduces a Bottleneck MHSA unit, which combines 3×3 convolution with the MHSA mechanism. The former uses the local receptive field to capture fine-grained spatial information, while the latter models global contextual dependencies, achieving hierarchical fusion of local and global features. The secondary branch maintains parallel feature propagation to preserve the original information. Finally, the two feature streams are aggregated by Concat, and the final CBS module performs cross-channel information recalibration and spatial feature enhancement, thereby achieving effective fusion of multi-scale features and global dependencies.

[0095] like Figure 5 As shown, the core idea of ​​GhostNet is to first obtain intrinsic feature maps using a small number of standard convolutions, then generate additional redundant feature maps from these intrinsic feature maps through a linear transformation with low computational cost, and then fuse the two to maintain the model's feature representation ability while significantly reducing the number of parameters and computational cost.

[0096] To reduce convolutional computation redundancy while maintaining feature representation capability, embodiments of this application design a CSPGhost convolutional structure based on the GhostNet concept, such as... Figure 6As shown, the input features are first pre-encoded by the CBS module to obtain channel and spatial features, and then split into two parallel branches through a Split operation. The main branch consists of two cascaded BottleneckGhost units, each containing two GhostConvs. The first layer generates a partial feature map through standard convolution and performs normalization and non-linear mapping using BN and SiLU activation functions. The second layer uses a low-cost operation to generate the remaining feature map, and then concatenates it with the previous part of the features through a Concatenation operation to form a complete output. This output is then added element-wise with the residual edge concatenation result to achieve feature fusion. The other branch directly passes the original features to retain basic information. Finally, the two features are aggregated in the Concat operation, and the CBS module at the end performs cross-channel information recalibration and spatial feature enhancement, thereby obtaining a feature representation that is both lightweight and highly expressive.

[0097] To improve the identifiability and detection robustness of cracks in complex backgrounds, embodiments of this application introduce a large-kernel spatial attention mechanism (LSKA), such as... Figure 7 As shown. First, to address the need for a large receptive field, the original input features are... Large kernel decomposition is performed to reduce the computational overhead of directly using large convolution kernels. Let the kernel size and dilation rate of the i-th depthwise separable convolution be k, respectively. i and d i Its receptive field RF i This can be expressed by a recursive formula as follows:

[0098] k i-1 ≤k i ,d1=1,d i-1 <d i ≤RF i-1

[0099] RF1 = k1, RF i =d i (k i -1)+RF i-1

[0100] Based on this, two groups of large kernel branches with different receptive fields were selected to extract long-range dependency features, resulting in...

[0101]

[0102] Among them, F dw This represents a depthwise separable convolution, where (k1, k2) correspond to different kernel sizes for different receptive fields. This operation allows for the acquisition of multi-scale spatial context features with low computational complexity, providing an informational basis for subsequent background suppression.

[0103] Then, the two features U1 and U2 are concatenated along the channel dimension and global average pooling and max pooling are performed to obtain...

[0104] f avg =AvgPool(Concat[U1,U2])

[0105] f max =MaxPool(Concat[U1,U2])

[0106] Next, the pooling results are concatenated along the channel dimension, and then mapped to an N-channel spatial attention weight map through convolution, normalization, and SiLU activation combination, resulting in...

[0107] SA=σ(F 2→N (Concat[f avg ,f max ]))

[0108] Where σ(·) is the Sigmoid activation function, F 2→N This represents a convolutional layer that maps the input 2-channel features to N-channel features. This weight map assigns higher response values ​​to salient crack regions in complex backgrounds, thereby enhancing the discriminative power of the features.

[0109] In the spatial selection stage, attention weights are used to weight and fuse the main and secondary branch features to obtain...

[0110]

[0111] Then, feature recalibration is performed by element-wise multiplication with the input feature X to obtain the final feature representation.

[0112]

[0113] in, This represents the element-wise multiplication operation. This process adaptively fuses global and local spatial information, effectively suppressing the interference of complex road surface textures and environmental noise on crack detection, and significantly improving the model's detection reliability in complex environments.

[0114] In existing technologies, the original detection head design of YOLOv8 still has room for optimization: First, the parameter scale of this part is relatively large, and the computational cost accounts for a high proportion of the entire network. Each detection head uses a concatenated structure consisting of two 3×3 convolutional layers and one 1×1 convolutional layer to extract and transform features. This repeated stacking not only increases computational complexity but also significantly increases the number of parameters in the overall model. Second, the single-scale prediction mode used by this detection head is difficult to take into account the detection needs of targets of different sizes. It only relies on the current feature layer for prediction and ignores supplementary information from feature maps of other scales, thus creating a performance bottleneck when processing multi-scale targets.

[0115] To address the aforementioned shortcomings, this embodiment proposes a lightweight detection head (LBoxHead), such as... Figure 8 As shown in the diagram, this structure introduces a GroupNorm normalization strategy into the convolutional units. This method has been validated in related studies as effectively enhancing the classification and localization capabilities of features. In terms of design, the original two-layer standard convolutions of the three detection heads are replaced with shared GroupNorm convolutions, and a scaling layer at the output adjusts the feature ratio to match the target size of different detection branches. This improvement significantly reduces redundant parameters and computational load, while enhancing the multi-scale perception capability of the detection heads, making them more suitable for deployment in computationally limited applications.

[0116] like Figure 9 The diagram shown is a structural schematic of a lightweight road crack detection system provided in an embodiment of this application, comprising:

[0117] The acquisition module 910 is used to acquire a road surface image and input the road surface image into the backbone feature extraction network to extract a multi-scale basic feature map set, wherein the multi-scale basic feature map set includes multiple basic feature maps output by the backbone feature extraction network at at least two different spatial resolutions.

[0118] The generation module 920 is used to adaptively select intermediate-scale feature maps with spatial resolution within a preset range from the multiple basic feature maps according to the scale distribution characteristics of the crack target, and perform coupled feature modeling processing on the intermediate-scale feature maps to generate collaborative feature maps.

[0119] The coupled feature modeling process includes: dividing the intermediate-scale feature map into at least two feature branches along the channel dimension; in the first feature branch, performing local convolution operations on the intermediate-scale feature map to obtain local feature representations, and generating query vectors, key vectors, and value vectors based on the local feature representations; obtaining global feature representations through multi-head self-attention operations, wherein the local feature representations are used to modulate the attention weights in the multi-head self-attention operations to suppress global responses unrelated to the crack structure; in the second feature branch, performing parallel feature transfer on the intermediate-scale feature map to preserve the original spatial structure information; and fusing the local feature representations, the global feature representations, and the intermediate-scale feature map to generate the collaborative feature map.

[0120] Specifically, the generation module 920 includes:

[0121] The computational submodule is used to calculate the crack response intensity score for each basic feature map based on the edge continuity, linear structure density, and gradient consistency of each basic feature map.

[0122] The selection submodule is used to select feature maps whose scores meet preset conditions from the multiple basic feature maps as intermediate scale feature maps. The preset conditions are determined based on the scale distribution characteristics of the crack target.

[0123] Specifically, the computation submodule is used to process the basic feature map F at the l-th scale using the following formula. (l) Spatial gradient calculation is performed to obtain the gradient magnitude map G. (l) (x,y):

[0124]

[0125] The gradient magnitude map is thinned and connected component analysis is performed, and the edge continuity of the basic feature map at the l-th scale is determined by the following formula:

[0126]

[0127] Among them, E (l) For edge continuity, E (l) The larger the value of N, the more the corresponding basic feature map tends to represent a slender, continuous crack structure; l L represents the number of edge segments at the l-th scale; k A is the length of the k-th edge segment; k Let be the area enclosed by the k-th edge segment;

[0128] For the basic feature map F at the l-th scale (l) Directional filtering is performed to obtain a set of directional response maps. Based on the set of directional response maps, the linear structure density of the basic feature map at the l-th scale is determined using the following formula:

[0129]

[0130] Among them, H l and W l These represent the height and width of the base feature map at the l-th scale, respectively. Different scales correspond to different downsampling ratios, satisfying H1>H2>…>H L W1>W2>…>W L ; The set of directional response maps; D (l) The density of the linear structure represents the response intensity to the linear structure per unit area; This indicates that the maximum value is taken in all directions to highlight the linear structure;

[0131] The gradient direction consistency of the base feature map at the l-th scale is calculated using the following formula:

[0132]

[0133] Among them, H l and W l These are the height and width of the base feature map at the l-th scale, respectively; φ represents the local neighborhood centered at position (x, y); x,y C represents the gradient direction at position (x, y); (l) For gradient direction consistency, C (l) A larger value indicates a more consistent gradient direction, which better matches the characteristics of crack propagation.

[0134] The basic feature map F at the l-th scale is obtained using the following formula. (l) The edge continuity, linear structure density, and gradient consistency are weighted and fused to obtain the basic feature map F at the l-th scale. (l) Crack response strength score:

[0135] S (l) =αE (l) +βD (l) +γC (l)

[0136] Among them, E (l) For edge continuity, D (l) For linear structure density, C (l) To ensure gradient direction consistency, α, β, and γ are the first weight coefficient, the second weight coefficient, and the third weight coefficient, respectively, and α+β+γ=1;

[0137] Accordingly, a submodule is selected, specifically for calculating the normalized scaling factor at the l-th scale using the following formula:

[0138]

[0139] Where, r (l) H is the normalized scaling factor for the l-th scale. l and W l H1 and W1 are the height and width of the base feature map at the l-th scale, respectively; H1 and W1 are the height and width of the base feature map at the first scale, respectively.

[0140] Based on the center location of the target scale distribution of the crack, the scale penalty function is determined using the following formula:

[0141] P (l) =exp(-λ(logr) (l) -μ) 2 )

[0142] Among them, P (l) Let r be the scaling penalty function. (l) λ is the normalized scale factor for the l-th scale, λ is the penalty intensity coefficient, and μ is the center position of the crack target scale distribution.

[0143] The weighted scale penalty score of the base feature map at the l-th scale is calculated using the following formula:

[0144]

[0145] in, The weighted scale penalty score, P, of the base feature map at the l-th scale. (l) Let S be the scaling penalty function. (l) The crack response intensity score is given for the basic feature map at the l-th scale.

[0146] If the weighted scale penalty score of the base feature map at the l-th scale is greater than or equal to a preset threshold, then the base feature map at the l-th scale is used as the intermediate scale feature map.

[0147] In the multi-head self-attention operation of this embodiment, the weight of each attention head is modulated by the structure-guided weight generated by the local feature representation to enhance the response along the crack extension direction and suppress the global association of non-crack regions. The structure-guided weight acts on at least one of the query vector, key vector or attention output.

[0148] or,

[0149] The weights of each attention head are modulated by structural constraint factors generated from the local feature representation to enhance the global response consistent with the local structure of the crack and suppress attention allocation to irrelevant regions. The structural constraint factors are generated based on the crack's directional information, connectivity information, or a combination thereof in the local feature representation and are used to weight or gate the attention weights.

[0150] Specifically, the generation module 920 is used to generate structure-guided weights based on the local feature representation using the following formula:

[0151] W s =σ(Conv(F) local ))

[0152] Among them, W s The structure guides the weights, σ ​​is the Sigmoid activation function, Conv(·) is a 1×1 convolution, and F... local Representation as local features;

[0153] Based on the structure-guided weights, the weights of each attention head in the multi-head self-attention operation are modulated using the following formula:

[0154]

[0155] Among them, W s The structure guides the weights, where Q is the query vector, K is the key vector, V is the value vector, and d is the weights. k The dimension of the key vector;

[0156] Alternatively, based on the local feature representation, the structural constraint factor can be generated using the following formula:

[0157] C s =σ(Pool(F) local ))

[0158] Among them, C s σ is the structural constraint factor, σ is the Sigmoid activation function, Pool(·) is the pooling operation, and F local Representation as local features;

[0159] Based on the structural constraint factor, the weights of each attention head in the multi-head self-attention operation are modulated using the following formula:

[0160]

[0161] Among them, C s Let Q be the structural constraint factor generated from the local feature representation, K be the key vector, V be the value vector, and d be the value vector. k The dimension of the key vector.

[0162] The compensation module 930 is used to perform lightweight feature compensation processing on the collaborative feature map as input to obtain output features.

[0163] The lightweight feature compensation process includes: generating a basic feature subset through a first convolution operation; generating a redundant feature subset based on the basic feature subset through a channel-wise linear transformation or element-wise linear mapping with a computational complexity lower than that of the first convolution operation; and concatenating the basic feature subset with the redundant feature subset to obtain the output feature.

[0164] Processing module 940 is used to perform spatial recalibration processing on the output features to enhance the saliency of the crack region and suppress background texture interference.

[0165] The spatial recalibration process includes: generating a spatial weight map and weighting the output features based on the spatial weight map;

[0166] The detection module 950 is used to perform multi-scale target detection based on the recalibrated output features to obtain crack detection results. Detection branches of different scales share the same set of detection head parameters.

[0167] This application's embodiments, by introducing a feature selection mechanism based on crack scale distribution, a global modeling method with local structural constraints, and a lightweight feature compensation and spatial recalibration strategy, can improve the targeting of crack characterization, enhance the ability to perceive slender cracks, prevent crack details from being weakened during the modeling process, and effectively reduce computational complexity while ensuring detection accuracy. This significantly improves the accuracy, robustness, and engineering practicality of road crack detection in complex environments.

[0168] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described lightweight road crack detection method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0169] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0170] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0171] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A lightweight method for detecting road cracks, characterized in that, Includes the following steps: S1. Acquire a road surface image and input the road surface image into a backbone feature extraction network to extract a multi-scale basic feature map set, wherein the multi-scale basic feature map set includes multiple basic feature maps output by the backbone feature extraction network at at least two different spatial resolutions. S2. Based on the scale distribution characteristics of the crack target, adaptively select an intermediate scale feature map with a spatial resolution within a preset range from the multiple basic feature maps, and perform coupled feature modeling processing on the intermediate scale feature map to generate a collaborative feature map. The coupled feature modeling process includes: dividing the intermediate-scale feature map into at least two feature branches along the channel dimension; in the first feature branch, performing local convolution operations on the intermediate-scale feature map to obtain local feature representations, and generating query vectors, key vectors, and value vectors based on the local feature representations; obtaining global feature representations through multi-head self-attention operations, wherein the local feature representations are used to modulate the attention weights in the multi-head self-attention operations to suppress global responses unrelated to the crack structure; in the second feature branch, performing parallel feature transfer on the intermediate-scale feature map to preserve the original spatial structure information; and fusing the local feature representations, the global feature representations, and the intermediate-scale feature map to generate the collaborative feature map. S3. Using the collaborative feature map as input, perform lightweight feature compensation processing to obtain the output features; The lightweight feature compensation process includes: generating a basic feature subset through a first convolution operation; generating a redundant feature subset based on the basic feature subset through a channel-wise linear transformation or element-wise linear mapping with a computational complexity lower than that of the first convolution operation; and concatenating the basic feature subset with the redundant feature subset to obtain the output feature. S4. Perform spatial recalibration processing on the output features to enhance the saliency of the crack region and suppress background texture interference; The spatial recalibration process includes: generating a spatial weight map and weighting the output features based on the spatial weight map; S5. Perform multi-scale target detection based on the recalibrated output features to obtain crack detection results. Detection branches of different scales share the same set of detection head parameters.

2. The method according to claim 1, characterized in that, The step of adaptively selecting an intermediate-scale feature map with a spatial resolution within a preset range from the multiple basic feature maps based on the scale distribution characteristics of the crack target specifically includes: Based on the edge continuity, linear structure density, and gradient consistency of each basic feature map, the crack response intensity score of each basic feature map is calculated. Feature maps whose scores meet preset conditions are selected from the multiple basic feature maps as intermediate scale feature maps. The preset conditions are determined based on the scale distribution characteristics of the crack target.

3. The method according to claim 2, characterized in that, The crack response intensity score for each basic feature map is calculated based on its edge continuity, linear structure density, and gradient consistency. Specifically, this includes: The following formula is used to evaluate the basic feature map F at the l-th scale. (l) Spatial gradient calculation is performed to obtain the gradient magnitude map G. (l) (x,y): The gradient magnitude map is thinned and connected component analysis is performed, and the edge continuity of the basic feature map at the l-th scale is determined by the following formula: Among them, E (l) For edge continuity, E (l) The larger the value of N, the more the corresponding basic feature map tends to represent a slender, continuous crack structure; l L represents the number of edge segments at the l-th scale; k A is the length of the k-th edge segment; k Let be the area enclosed by the k-th edge segment; For the basic feature map F at the l-th scale (l) Directional filtering is performed to obtain a set of directional response maps. Based on the set of directional response maps, the linear structure density of the basic feature map at the l-th scale is determined using the following formula: Among them, H l and W l These represent the height and width of the base feature map at the l-th scale, respectively. Different scales correspond to different downsampling ratios, satisfying H1>H2>…>H L W1>W2>…>W L ; The set of directional response maps; D (l) The density of the linear structure represents the response intensity to the linear structure per unit area; This indicates that the maximum value is taken in all directions to highlight the linear structure; The gradient direction consistency of the base feature map at the l-th scale is calculated using the following formula: Among them, H l and W l These are the height and width of the base feature map at the l-th scale, respectively; φ represents the local neighborhood centered at position (x, y); x,y C represents the gradient direction at position (x, y); (l) For gradient direction consistency, C (l) A larger value indicates a more consistent gradient direction, which better matches the characteristics of crack propagation. The basic feature map F at the l-th scale is obtained using the following formula. (l) The edge continuity, linear structure density, and gradient consistency are weighted and fused to obtain the basic feature map F at the l-th scale. (l) Crack response strength score: S (l) =αE (l) +βD (l) +γC (l) Among them, D (l) For edge continuity, D (l) For linear structure density, C (l) To ensure gradient direction consistency, α, β, and γ are the first weight coefficient, the second weight coefficient, and the third weight coefficient, respectively, and α+β+γ=1; The step of selecting feature maps whose scores meet preset conditions from the plurality of basic feature maps as intermediate-scale feature maps specifically includes: The normalized scaling factor for the l-th scale is calculated using the following formula: Where, r (l) H is the normalized scaling factor for the l-th scale. l and W l H1 and W1 are the height and width of the base feature map at the l-th scale, respectively; H1 and W1 are the height and width of the base feature map at the first scale, respectively. Based on the center location of the target scale distribution of the crack, the scale penalty function is determined using the following formula: P (l) =exp(-λ(logr (l) -m) 2 ) Among them, P (l) Let r be the scaling penalty function. (l) λ is the normalized scale factor for the l-th scale, λ is the penalty intensity coefficient, and μ is the center position of the crack target scale distribution. The weighted scale penalty score of the base feature map at the l-th scale is calculated using the following formula: in, The weighted scale penalty score, P, of the base feature map at the l-th scale. (l) Let S be the scaling penalty function. (l) The crack response intensity score is given for the basic feature map at the l-th scale. If the weighted scale penalty score of the base feature map at the l-th scale is greater than or equal to a preset threshold, then the base feature map at the l-th scale is used as the intermediate scale feature map.

4. The method according to claim 1, characterized in that, In the multi-head self-attention operation, the weights of each attention head are modulated by the structure-guided weights generated by the local feature representation to enhance the response along the crack extension direction and suppress global associations in non-crack regions. The structure-guided weights act on at least one of the query vector, key vector, or attention output. or, The weights of each attention head are modulated by structural constraint factors generated from the local feature representation to enhance the global response consistent with the local structure of the crack and suppress attention allocation to irrelevant regions. The structural constraint factors are generated based on the crack's directional information, connectivity information, or a combination thereof in the local feature representation and are used to weight or gate the attention weights.

5. The method according to claim 4, characterized in that, The process of obtaining a global feature representation through multi-head self-attention operation, and using the local feature representation to modulate the attention weights in the multi-head self-attention operation, specifically includes: Based on the local feature representation, the structure-guided weights are generated using the following formula: W s =σ(Conv(F local )) Among them, W s The structure guides the weights, σ ​​is the Sigmoid activation function, Conv(·) is a 1×1 convolution, and F... local Representation as local features; Based on the structure-guided weights, the weights of each attention head in the multi-head self-attention operation are modulated using the following formula: Among them, W s The structure guides the weights, where Q is the query vector, K is the key vector, V is the value vector, and d is the weights. k The dimension of the key vector; Alternatively, based on the local feature representation, the structural constraint factor can be generated using the following formula: C s <σ(Pool(F) local )) Among them, C s σ is the structural constraint factor, σ is the Sigmoid activation function, Pool(·) is the pooling operation, and F local Representation as local features; Based on the structural constraint factor, the weights of each attention head in the multi-head self-attention operation are modulated using the following formula: Among them, C s Let Q be the structural constraint factor generated from the local feature representation, K be the key vector, V be the value vector, and d be the value vector. k The dimension of the key vector.

6. A lightweight road crack detection system, characterized in that, include: An acquisition module is used to acquire a road surface image and input the road surface image into a backbone feature extraction network to extract a multi-scale basic feature map set, wherein the multi-scale basic feature map set includes multiple basic feature maps output by the backbone feature extraction network at at least two different spatial resolutions. The generation module is used to adaptively select intermediate-scale feature maps with spatial resolution within a preset range from the multiple basic feature maps according to the scale distribution characteristics of the crack target, and perform coupled feature modeling processing on the intermediate-scale feature maps to generate collaborative feature maps. The coupled feature modeling process includes: dividing the intermediate-scale feature map into at least two feature branches along the channel dimension; in the first feature branch, performing local convolution operations on the intermediate-scale feature map to obtain local feature representations, and generating query vectors, key vectors, and value vectors based on the local feature representations; obtaining global feature representations through multi-head self-attention operations, wherein the local feature representations are used to modulate the attention weights in the multi-head self-attention operations to suppress global responses unrelated to the crack structure; in the second feature branch, performing parallel feature transfer on the intermediate-scale feature map to preserve the original spatial structure information; and fusing the local feature representations, the global feature representations, and the intermediate-scale feature map to generate the collaborative feature map. The compensation module is used to perform lightweight feature compensation processing on the collaborative feature map as input to obtain the output features; The lightweight feature compensation process includes: generating a basic feature subset through a first convolution operation; generating a redundant feature subset based on the basic feature subset through a channel-wise linear transformation or element-wise linear mapping with a computational complexity lower than that of the first convolution operation; and concatenating the basic feature subset with the redundant feature subset to obtain the output feature. The processing module is used to perform spatial recalibration processing on the output features to enhance the saliency of the crack region and suppress background texture interference; The spatial recalibration process includes: generating a spatial weight map and weighting the output features based on the spatial weight map; The detection module is used to perform multi-scale target detection based on the recalibrated output features to obtain crack detection results. Detection branches of different scales share the same set of detection head parameters.

7. The system according to claim 6, characterized in that, The generation module specifically includes: The computational submodule is used to calculate the crack response intensity score for each basic feature map based on the edge continuity, linear structure density, and gradient consistency of each basic feature map. The selection submodule is used to select feature maps whose scores meet preset conditions from the multiple basic feature maps as intermediate scale feature maps. The preset conditions are determined based on the scale distribution characteristics of the crack target.

8. The system according to claim 7, characterized in that, The computational submodule is specifically used to process the basic feature map F at the l-th scale using the following formula. (l) Spatial gradient calculation is performed to obtain the gradient magnitude map G. (l) (x,y): The gradient magnitude map is thinned and connected component analysis is performed, and the edge continuity of the basic feature map at the l-th scale is determined by the following formula: Among them, E (l) For edge continuity, E (l) The larger the value of N, the more the corresponding basic feature map tends to represent a slender, continuous crack structure; l L represents the number of edge segments at the l-th scale; k A is the length of the k-th edge segment; k Let be the area enclosed by the k-th edge segment; For the basic feature map F at the l-th scale (l) Directional filtering is performed to obtain a set of directional response maps. Based on the set of directional response maps, the linear structure density of the basic feature map at the l-th scale is determined using the following formula: Among them, H l and W l These represent the height and width of the base feature map at the l-th scale, respectively. Different scales correspond to different downsampling ratios, satisfying H1>H2>…>H L W1>W2>…>W L ; The set of directional response maps; D (l) The density of the linear structure represents the response intensity to the linear structure per unit area; This indicates that the maximum value is taken in all directions to highlight the linear structure; The gradient direction consistency of the base feature map at the l-th scale is calculated using the following formula: Among them, H l and W l These are the height and width of the base feature map at the l-th scale, respectively; φ represents the local neighborhood centered at position (x, y); x,y C represents the gradient direction at position (x, y); (l) For gradient direction consistency, C (l) A larger value indicates a more consistent gradient direction, which better matches the characteristics of crack propagation. The basic feature map F at the l-th scale is obtained using the following formula. (l) The edge continuity, linear structure density, and gradient consistency are weighted and fused to obtain the basic feature map F at the l-th scale. (l) Crack response strength score: S (l) =αE (l) +βD (l) +γC (l) Among them, E (l) For edge continuity, D (l) For linear structure density, C (l) To ensure gradient direction consistency, α, β, and γ are the first weight coefficient, the second weight coefficient, and the third weight coefficient, respectively, and α+β+γ=1; The selection submodule is specifically used to calculate the normalized scaling factor for the l-th scale using the following formula: Where, r (l) H is the normalized scaling factor for the l-th scale. l and W l H1 and W1 are the height and width of the base feature map at the l-th scale, respectively; H1 and W1 are the height and width of the base feature map at the first scale, respectively. Based on the center location of the target scale distribution of the crack, the scale penalty function is determined using the following formula: P (l) =exp(-λ(logr (l) -m) 2 ) Among them, P (l) Let r be the scaling penalty function. (l) λ is the normalized scale factor for the l-th scale, λ is the penalty intensity coefficient, and μ is the center position of the crack target scale distribution. The weighted scale penalty score of the base feature map at the l-th scale is calculated using the following formula: in, The weighted scale penalty score, P, of the base feature map at the l-th scale. (l) Let S be the scaling penalty function. (l) The crack response intensity score is given for the basic feature map at the l-th scale. If the weighted scale penalty score of the base feature map at the l-th scale is greater than or equal to a preset threshold, then the base feature map at the l-th scale is used as the intermediate scale feature map.

9. The system according to claim 6, characterized in that, In the multi-head self-attention operation, the weights of each attention head are modulated by the structure-guided weights generated by the local feature representation to enhance the response along the crack extension direction and suppress global associations in non-crack regions. The structure-guided weights act on at least one of the query vector, key vector, or attention output. or, The weights of each attention head are modulated by structural constraint factors generated from the local feature representation to enhance the global response consistent with the local structure of the crack and suppress attention allocation to irrelevant regions. The structural constraint factors are generated based on the crack's directional information, connectivity information, or a combination thereof in the local feature representation and are used to weight or gate the attention weights.

10. The system according to claim 9, characterized in that, The generation module is specifically used to generate structure-guided weights based on the local feature representation using the following formula: W s =σ(Conv(F local )) Among them, W s The structure guides the weights, σ ​​is the Sigmoid activation function, Conv(·) is a 1×1 convolution, and F... local Representation as local features; Based on the structure-guided weights, the weights of each attention head in the multi-head self-attention operation are modulated using the following formula: Among them, W s The structure guides the weights, where Q is the query vector, K is the key vector, V is the value vector, and d is the weights. k The dimension of the key vector; Alternatively, based on the local feature representation, the structural constraint factor can be generated using the following formula: C s <σ(Pool(F) local )) Among them, C s σ is the structural constraint factor, σ is the Sigmoid activation function, Pool(·) is the pooling operation, and F local Representation as local features; Based on the structural constraint factor, the weights of each attention head in the multi-head self-attention operation are modulated using the following formula: Among them, C s Let Q be the structural constraint factor generated from the local feature representation, K be the key vector, V be the value vector, and d be the value vector. k The dimension of the key vector.

Images