Image crack identification model generation method for identifying road surface cracks, and application and system thereof

Abstract

The application relates to the technical field of image recognition, and discloses a kind of image crack identification model generation method and application and system for identifying road surface crack, comprising: feature extraction and feature splicing are carried out to original image to obtain pseudo-modal feature set and collaborative input tensor;Pseudo-modal feature set and collaborative input tensor are modulated to obtain gate enhanced shallow features;The gate enhanced shallow features are extracted, encoded and decoded to obtain a crack prediction probability map;A pixel in the crack prediction probability map is used to establish a model loss function, the image crack identification model is trained by the model loss function, and a trained image crack identification model is obtained;The application solves the problem of low recognition accuracy and poor stability of the existing road surface image crack identification method.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, application, and system for generating image crack recognition models for identifying road surface cracks. Background Technology

[0002] Road surface cracks are a common form of road damage during service, and their formation and expansion can affect the integrity of the road structure and driving safety. Obtaining crack information is crucial for road technical condition assessment and maintenance decisions. Timely identification of cracked areas helps to understand the development of road damage and provides a basis for subsequent maintenance. Traditional crack detection relies heavily on manual inspection or image interpretation. While these methods are intuitive, they are labor-intensive for long-distance, large-scale road inspections, and the results are easily affected by personnel experience and shooting conditions. Therefore, utilizing image acquisition equipment to acquire road surface images and automatically identifying cracks using computer vision methods has become an important development direction for road damage detection.

[0003] Existing image-based methods for road crack detection mainly include traditional image processing methods and deep learning methods. Traditional methods typically identify cracks based on differences in grayscale, edges, or texture between the crack and the road surface background. While effective under simple backgrounds and stable lighting conditions, these methods tend to falter when faced with interference from rough textures, shadows, stains, or road markings. In recent years, convolutional neural networks, U-Net and its improved structures, attention mechanisms, and multi-scale feature fusion have been used for crack detection and segmentation. These methods can automatically learn crack features from samples, improving the completeness and accuracy of crack region extraction compared to traditional methods.

[0004] However, the acquisition environment for real-world road images is often unstable, especially after rain, when there is a water film, standing water, or strong reflection on the road surface, which significantly alters the image representation of cracks. Wet road surfaces typically reduce the local contrast between cracks and surrounding textures, making fine crack structures difficult to identify. Standing water or a water film causes refraction and scattering effects, resulting in blurred crack boundaries, discontinuous contours, or weakened local edge responses. Strongly reflective areas easily produce striped highlights and pseudo-linear structures, whose morphology resembles fine cracks, leading to false detections and oversegmentation. Furthermore, low-cost road inspection scenarios typically rely primarily on ordinary RGB images, lacking auxiliary modal information such as depth, infrared, or polarization, making reflective areas, shadow areas, and real cracks more easily confused in pixel and feature spaces. In addition, pixel-level crack annotation samples for wet, water-filled, and strongly reflective conditions remain relatively limited, restricting the systematic analysis and stability evaluation of crack segmentation methods in complex environments. Therefore, existing road image crack recognition methods suffer from low accuracy and poor stability. Summary of the Invention

[0005] This invention provides a method and system for generating image crack recognition models for identifying road surface cracks, in order to solve the problems of low recognition accuracy and poor stability in existing road surface image crack recognition methods.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] In a first aspect, the present invention provides a method for generating an image crack recognition model for identifying road surface cracks, comprising: S1. Obtain the original image of the target road surface, extract features from the original image to obtain a pseudo-modal feature set, and concatenate the features of the original image to obtain a collaborative input tensor; S2. Modulate the pseudo-modal feature set and the collaborative input tensor to obtain gated enhanced shallow features; S3. Perform feature extraction, encoding, and decoding on the gated enhanced shallow features to obtain the crack prediction probability map; S4. Construct the original image crack recognition model, establish the model loss function based on the pixels in the crack prediction probability map, and train the original image crack recognition model through the model loss function to obtain the trained image crack recognition model.

[0008] Optionally, in S1, feature extraction is performed on the original image to obtain a pseudo-modal feature set, including: Acquire the RGB image of the target road surface as the original image, and convert the RGB image to a grayscale image; Dark structure responses are extracted from grayscale images using local maximum morphological operations at different scales, yielding grayscale structure features and dark structure features. The grayscale structure features satisfy the following relationship: ; Dark structure features satisfy the following relationship: ; In the formula, Represents grayscale structural features. Indicates dark structure features, Representing scale Local maximum morphological operations under the following conditions Local neighborhood scale of grayscale structure The local neighborhood scale represents the feature scale of dark structures. Represents a grayscale image; Gaussian smoothing is applied to the grayscale image, and the Laplacian negative response is calculated to obtain the Laplacian response features, which satisfy the following relationship: ; In the formula, Indicates the Laplace response characteristics, Indicates the Gaussian smoothing kernel. This represents the convolution operation. Represents the discrete Laplace operator. Represents a non-linear activation function; Reflectance-related response features are constructed based on grayscale images using local minimum morphological operations, where the reflection-related response features satisfy the following relationship: ; In the formula, Indicates the reflective response characteristics, The neighborhood scale representing local differences in reflectance. Indicated in scale Local minimal morphological operations under; The grayscale structural features, dark structural features, Laplace response features, and reflectivity-related response features are normalized, and the normalized features are combined into a four-channel pseudo-modal feature set.

[0009] Optionally, in S1, feature concatenation is performed on the original image to obtain a co-input tensor, including: The pseudomodal feature set is concatenated with the RGB image along the channel direction to obtain the co-input tensor, which satisfies the following relationship: ; In the formula, Represents the collaborative input tensor. This indicates a channel splicing operation. Represents an RGB image. This represents the set of pseudomodal features.

[0010] Optionally, acquire the original image of the target road surface, including: Original RGB images were captured using road image acquisition equipment with RGB image acquisition capabilities, including those of wet road surfaces, waterlogged road surfaces, and highly reflective road surfaces. The crack regions in the acquired original RGB image are annotated at the pixel level to generate crack mask labels corresponding to the original RGB image. In the crack mask labels, the crack regions and the background regions are distinguished by different pixel values. The original RGB image and its corresponding crack mask label are preprocessed to construct a sample set for model training, validation and testing. The preprocessing includes one or more of the following: image size normalization, pixel value normalization, data augmentation and sample set partitioning. The data augmentation includes one or more of the following: random flipping, random rotation, random cropping, brightness perturbation and contrast perturbation.

[0011] Optionally, in S2, feature modulation is performed on the pseudo-modal feature set and the co-input tensor to obtain gated enhanced shallow features, including: A structure-aware gated graph is generated based on a set of pseudo-modal features, wherein the structure-aware gated graph satisfies the following relationship: ; ; In the formula, Represents a structure-aware gated graph. This represents the convolution operation. This indicates a batch normalization operation. Represents a non-linear activation function. This represents the Sigmoid activation function. and Represents the convolution weights. Indicates the bias term. This represents the intermediate features obtained from the pseudomodal feature mapping; Shallow features are generated based on collaborative input tensors, where the shallow features satisfy the following relationship: ; In the formula, Represents the collaborative input tensor. Indicates shallow features. This indicates the first-stage encoder; By performing element-wise modulation on shallow features using a structure-aware gating map, gated enhanced shallow features are obtained, which satisfy the following relationship: ; In the formula, This represents element-wise multiplication. This represents the number of channels in the shallow feature map. This indicates gating enhancement of shallow features.

[0012] Optionally, in S3, feature extraction, encoding, and decoding are performed on the gated enhanced shallow features to obtain a crack prediction probability map, including: Constructing a five-level coding phase, for the first Each encoding stage, its feature extraction process satisfies the following relationship: ; In the formula, This indicates a downsampling operation. Indicates the first Convolutional transformation in each encoding stage, Indicates the first Level feature map; The first-level feature map in the first encoding stage is replaced with gated augmented shallow features. Through the feature extraction process of each encoding stage, the corresponding feature map is obtained. After the five-level encoding stage, a global augmented feature is generated based on the fifth-level feature map. The global augmented feature satisfies the following relationship: ; In the formula, This indicates that global dependency modeling is performed on low-resolution deep features. This represents the 5th level feature map. Indicates global enhancement features; The global enhancement features are upsampled and then concatenated with the level 4 feature map to obtain the fused features, which satisfy the following relationship: ; In the formula, This indicates an upsampling operation or a transposed convolution operation. This represents the 4th level feature map. Indicates fusion characteristics; The fused features are decoded to obtain the decoded enhanced features, which satisfy the following relationship: ; In the formula, This represents the first stage of the five-level encoding process, the decoding convolution operation. This refers to the first stage of the five-level encoding process, the decoding Transformer module. Indicates decoding enhancement features; Spatial resolution is gradually restored by performing progressive upsampling, skip connections, and convolutional decoding operations on the decoded enhancement features, and a crack prediction probability map is output.

[0013] Optionally, in S4, a model loss function is established using pixels in the crack prediction probability map. The image crack recognition model is trained using the model loss function to obtain the trained image crack recognition model, including: Determine the predicted probability of each pixel in the crack prediction probability map, and construct a surface distance loss function based on the predicted probability of each pixel; A total loss function is constructed based on the surface distance loss function, Focal Tversky loss function, binary cross-entropy loss function, and boundary-aware loss function, wherein the total loss function satisfies the following relationship: ; In the formula, This represents the Focal Tversky loss function. Represents the cross-entropy loss function. This represents the boundary-aware loss function. Represents the surface distance loss function. ( i =1,2,3,4) represents the balance coefficients of each loss term, Indicates the linear preheating factor. Represents the total loss function; The linear preheating factor satisfies the following relationship: ; In the formula, Indicates the current training round. Indicates the preheating cycle; The image crack recognition model is trained by using the total loss function to obtain the trained image crack recognition model.

[0014] Optionally, the surface distance loss function satisfies the following relationship: ; In the formula, Represents the set of image pixels. Represents pixels The signed distance to the nearest real crack boundary Represents the absolute value of the corresponding distance. Indicates the distance cutoff threshold. Represents pixels Predicted probability of belonging to the crack region; The Focal Tversky loss function satisfies the following relationship: ; ; The binary cross-entropy loss function satisfies the following relationship: ; The boundary-aware loss function satisfies the following relationship: ; In the formula, Represents the set of image pixels. Indicates the total number of pixels in the image. Represents pixels in an image. Represents pixels The predicted probability of belonging to the crack region. Represents pixels The true label, This represents the Tversky coefficient. This represents the weighting coefficient for the false positive term. This represents the weighting coefficient of the false negative term. Indicates the focusing coefficient. This indicates a smoothing term to prevent the denominator from being zero. This represents the soft edge label and edge weight obtained from the gradient of the real crack mask edge. This indicates that the edge prediction branch is at the pixel point. The output logit, This represents the Sigmoid function.

[0015] Secondly, the present invention also provides an application of the image crack recognition model obtained by the generation method in the first aspect for crack recognition in complex road surface images, and after steps S1-S4, it further includes the following step S5: The wet, reflective pavement image to be identified is input into the trained image crack recognition model to obtain a binary crack mask image. This binary crack mask image is then used as the result generated by the image crack recognition model for identifying pavement cracks. Specifically, this includes: The RGB image of the wet, reflective pavement to be identified is used as the input of the trained image crack recognition model. The trained image crack recognition model generates a corresponding pseudo-modal feature set based on the RGB image of the wet, reflective pavement to be identified. The pseudo-modal feature set is then modulated to obtain shallow visual features. The shallow visual features are then encoded and decoded to obtain the crack prediction probability map corresponding to the RGB image of the wet reflective road surface to be identified. Based on a preset threshold or post-processing strategy, the crack prediction probability map is converted into a binary crack mask map.

[0016] Thirdly, the present invention also provides a crack identification system for identifying road surface cracks, including a processor and a memory; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps as described in the second aspect.

[0017] Beneficial effects: This invention provides an image crack recognition model generation method for identifying road surface cracks. It constructs four pseudo-modal features—grayscale structure, dark structure, Laplacian response, and reflectivity-related response—through morphological and Laplacian operations, and concatenates these features with the original RGB image to form a seven-channel collaborative input tensor. This strategy effectively compensates for the low contrast between cracks and background and the degradation of texture information in a single RGB image under wet reflective conditions. Without increasing hardware costs, it significantly enhances the expression of structural prior information such as crack dark lines, edges, and reflectivity differences, improving the separability of cracks from interference areas under complex lighting conditions. A spatial structure-aware gating map is generated through pseudo-modal features, and shallow visual features are modulated element-wise. This gating mechanism adaptively highlights crack-related structural regions while suppressing interference from water film reflection, specular highlights, and complex background textures on shallow feature encoding, reducing the risk of false detection from the source and improving the feature purity of the model under interference scenarios. By employing CNNs to extract multi-scale local texture details and introducing a Transformer module to model global long-range dependencies in the bottleneck layer and decoding stage, the continuous structure of thin cracks, branch cracks, and low-contrast cracks can be effectively recovered. This overcomes the breakage or omission problem caused by the limited receptive field of traditional pure CNN models, outputting a more complete and continuous crack prediction probability map. A composite loss function including Focal Tversky loss, cross-entropy loss, boundary-aware loss, and surface distance loss is constructed, and a linear warm-up mechanism is introduced. This loss function not only alleviates the extreme imbalance between crack pixels and background pixels but also directly optimizes the spatial deviation between the predicted region and the real crack boundary through boundary distance constraints. This significantly improves the boundary blurring and positioning offset problems caused by water film or water accumulation, enhancing the edge sharpness and spatial consistency of the segmentation results. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the image crack recognition model generation method for identifying road surface cracks provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the pseudo-multimodal generation module structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure-aware gating module provided in an embodiment of the present invention; Figure 4 A schematic diagram of the CNN-Transformer hybrid segmentation network structure provided in an embodiment of the present invention; Figure 5 This is a schematic diagram showing the crack segmentation results under wet road surface, waterlogged road surface, and highly reflective road surface conditions provided in the embodiments of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms "an" or "a" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," "right," etc., are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship also changes accordingly.

[0021] like Figure 1 As shown, this embodiment provides a method for generating an image crack recognition model for identifying road surface cracks. The method includes a model training phase and a model application phase. In the model training phase, RGB images of cracks on a wet, reflective road surface are acquired and a sample set is constructed. Then, pseudo-multimodal generation, structure-aware gating, CNN-Transformer hybrid segmentation, and structure-aware boundary loss are sequentially performed to train the crack segmentation model, resulting in a trained crack segmentation model. In the model application phase, the RGB image of the wet, reflective road surface to be detected is input into the trained crack segmentation model, which outputs a crack prediction probability map and further obtains a binary crack mask map.

[0022] S1. Acquire images and construct a sample set. Acquire RGB images of cracks on wet, reflective road surfaces. The RGB images can be acquired by mobile phones, vehicle-mounted cameras, drones, or other road image acquisition devices with RGB image acquisition capabilities. The acquisition objects include crack images under conditions of wet road surfaces, waterlogged road surfaces, and highly reflective road surfaces.

[0023] The crack regions in the acquired RGB images are annotated at the pixel level to generate crack mask labels corresponding to the original RGB images. In these crack mask labels, crack regions and background regions are distinguished by different pixel values ​​to serve as supervision information during model training.

[0024] The RGB images and their corresponding crack mask labels are preprocessed to construct a sample set for model training, validation, and testing. The preprocessing includes one or more of image size normalization, pixel value normalization, data augmentation, and sample set partitioning. The data augmentation includes one or more of random flipping, random rotation, random cropping, brightness perturbation, and contrast perturbation.

[0025] In one specific embodiment, a sample set of WetCracks images of cracked pavement under wet and reflective conditions is constructed. This sample set includes 1800 RGB pavement crack images, collected under wet, flooded, and highly reflective pavement conditions, with a spatial resolution of 240×360 pixels. Specifically, there are 660 images of wet pavement, 660 images of flooded pavement, and 480 images of highly reflective pavement. Each image is annotated pixel-level by a domain expert to generate corresponding crack mask labels. During annotation, emphasis is placed on preserving the structure of small cracks, branch connectivity, and the integrity of crack boundaries under optical degradation conditions. To ensure fairness in model training and evaluation, the WetCracks sample set is stratified according to pavement condition and divided into training, validation, and test sets in a 6:2:2 ratio. It should be noted that the number of images, image size, collection location, and sample division ratio are only one implementation method in this embodiment and do not constitute a limitation on the scope of protection of this invention.

[0026] S2, perform pseudo-multimodal generation. For example... Figure 2 As shown, the preprocessed RGB image is input into the pseudo-multimodal generation module, and a pseudo-modal feature set is constructed based on a single RGB image. The pseudo-modal feature set includes grayscale structural features, dark structural features, Laplacian response features, and reflectivity-related response features. The pseudo-multimodal generation module constructs crack-related structural features based on morphological top-hat operations. Road surface cracks in images typically exhibit a lower local grayscale response relative to the surrounding area, i.e., local minimum intensity features or geometric concavity structures; while water film reflection, water accumulation reflection, or oil glare typically exhibit local maximum intensity features and surface-added brightness effects. Based on the above optical geometric features, the difference between the local maximum morphological operation and the original grayscale image is used to construct a dark structural response, enhancing the expressive ability of crack dark lines, local concavities, and crack structures at different scales; simultaneously, the difference between the original grayscale image and the result of the local minimum morphological operation is used to construct reflectivity-related response features, enhancing the brightness difference of the wet reflective area relative to the surrounding area; and combined with Laplacian response features to characterize crack edge changes, providing structural prior information for subsequent structure perception gating and feature fusion.

[0027] Let the input RGB image be: ; in, and These represent the image height and width, respectively. The three color channels of an RGB image are represented as follows: , and .

[0028] Convert the RGB image to a grayscale image. This is to reduce the impact of color variations on crack structure extraction. Defined by pixels. The local spatial neighborhood centered on the center is ,in Represents the local neighborhood scale; Define For scale Local maximum morphological operation under, defined For scale Local minimal morphological operations.

[0029] Based on the characteristic that crack regions typically exhibit lower local gray levels than surrounding areas, local maximal morphological operations at different scales are used to extract dark structure responses, thereby obtaining gray-level structure features. and dark structure features : ; ; in, and These represent different local neighborhood scales. By utilizing the dark structure responses at two scales, the expressive power of fine cracks, wider cracks, and local gray-scale depressions can be enhanced.

[0030] For grayscale images Gaussian smoothing is performed, and the Laplace negative response is calculated to obtain the Laplace response characteristics. : ; in, Indicates the Gaussian smoothing kernel. This represents the convolution operation. Represents the discrete Laplace operator. This represents a nonlinear activation function. The Laplace response characteristics are used to characterize crack edges, slender structures, and local intensity variations.

[0031] To address the issue of interference between reflective areas and crack structures in images of wet, waterlogged, or highly reflective road surfaces, local minimum morphological operations are employed to construct reflectivity-related response features. : ; in, The neighborhood scale is used to characterize local differences in reflectivity. Indicated in scale The local minimum morphological operation is performed. The reflective correlation response features are used to enhance the response that is brighter than the surrounding local area, thereby highlighting potential reflective pseudostructures and helping the network distinguish between real crack structures and reflective interference areas.

[0032] Will , , and Normalization is performed to ensure that the numerical ranges of each pseudomodal feature are consistent, and a four-channel pseudomodal feature set is formed: ; The pseudomodal feature set Compared with the original RGB image By concatenating along the channel direction, we obtain the co-input tensor: ; in, This indicates a channel concatenation operation. Through the above steps, the network input is expanded from a single RGB appearance feature to a seven-channel collaborative input tensor that simultaneously contains appearance information and structural prior information.

[0033] In this embodiment, based on the physical characteristics of cracks in images, such as localized extremely low grayscale, geometric concavity, and edge variations, morphological top-hat operations and edge response operators are used to explicitly construct multiple prior features strongly correlated with crack structures from a single RGB image without relying on additional sensors; at different scales and The model enhances fine cracks and wider cracks respectively, strengthens edges and slender structures with the Laplacian response, and quantifies local brightness differences with the reflective response to distinguish real cracks from reflective artifacts. After normalization, the numerical range of each feature is made consistent, which facilitates subsequent network fusion. This effectively makes up for the lack of appearance information under wet reflective conditions and improves the model's ability to distinguish low-contrast cracks and interference areas.

[0034] Meanwhile, the co-input tensor concatenation method preserves the color and texture appearance information of the original RGB image, while the four generated structural prior features are input into the network as additional channels in parallel, allowing the network to access appearance features and structural response features simultaneously at shallow layers. Compared to using pseudomodal features as separate branches or auxiliary supervision, the channel concatenation operation is simple and efficient, requiring no additional network branches or parameters, and can ensure that the features are fully aligned in the spatial dimension, facilitating subsequent joint convolution processing by the encoder. This significantly enhances the amount of input information without increasing hardware costs, providing richer discrimination criteria for subsequent gating modulation and feature extraction.

[0035] S3. Perform structure-aware gating. For example... Figure 3 As shown, the pseudomodal feature set Input the structure-aware gating module to generate a structure-aware gating graph. The structure-aware gating module is used to convert pseudo-modal structural information into a learnable spatial attention map. This spatial attention map acts as a soft gating mechanism to modulate shallow visual features, thereby highlighting crack-related structural regions and suppressing interference from wet textures, strong reflections, and complex background noise on shallow feature encoding.

[0036] The structure-aware gating graph The generation process is as follows: ; ; in, This represents the convolution operation. This indicates a batch normalization operation. Represents a non-linear activation function. This represents the Sigmoid activation function. and Represents the convolution weights. Indicates the bias term. This represents the intermediate features obtained from the pseudomodal feature mapping.

[0037] The structure-aware gating graph This is a single-channel spatial weight map with pixel values ​​ranging from (0, 1), used to represent the structural importance of different spatial locations in the image. Through this gating map, the network can enhance the response of crack-related regions and reduce the effects of reflections, wet textures, and background noise during the shallow encoding stage.

[0038] Cooperative input tensor Input first stage encoder shallow features are obtained. : ; Using structure-aware gating graphs shallow visual features Each channel is modulated element-wise: ; in, This represents element-wise multiplication. This represents the number of channels in the shallow feature map; when For single-channel feature maps When creating multi-channel feature maps, for After broadcasting to each channel, the features are multiplied by the features of each channel to obtain the gated enhanced shallow features. Through the aforementioned structure-aware gating modulation, the network can highlight crack-related structural regions in the early stages of encoding, while reducing the impact of reflective regions, complex texture regions, and non-crack linear structures on subsequent segmentation results.

[0039] In this embodiment, the structural prior information in the pseudo-modal features is transformed into spatial attention weights through the gating mechanism. The pixel values ​​of the gating map G are between (0, 1), which can adaptively indicate the importance of each spatial location in the image to crack identification. By performing soft gating on shallow features through element-wise multiplication, the response intensity of crack-related regions (such as dark structures and regions with strong edge responses) can be enhanced in the early stage of encoding, while suppressing the response of non-structural interference regions such as reflective highlights, water film textures, and stains. This allows the subsequent encoder to start from purer features, reduce the propagation of background noise to deeper layers, thereby improving the effectiveness of feature representation and reducing the risk of false detection and over-segmentation.

[0040] S4. Perform CNN-Transformer hybrid segmentation. For example... Figure 4 As shown, shallow features modulated by structure-aware gating are input into a CNN-Transformer hybrid segmentation network. The CNN-Transformer hybrid segmentation network includes a CNN encoder, a Transformer enhancement module, a decoder, and a segmentation head. The CNN encoder extracts multi-scale local texture features. The Transformer enhancement module is placed in the bottleneck layer and the first decoding stage to establish long-range dependencies in the low-resolution feature space and alleviate semantic ambiguity after feature fusion. The decoder restores spatial resolution through stepwise upsampling, skip connections, and convolutional decoding. The segmentation head outputs a crack prediction probability map.

[0041] The CNN encoder employs a hierarchical structure to extract multi-scale local texture features. In this embodiment, the encoder includes five encoding stages. For the first stage... Each encoding stage, its feature extraction process is represented as follows: ; in, This indicates a downsampling operation. Indicates the first Convolutional transformation in each encoding stage, Indicates the first Multi-scale feature maps. Through progressive downsampling and convolutional transformation, the encoder obtains multi-scale feature representations ranging from shallow texture to deep semantics, where... This represents the bottleneck feature at the lowest resolution.

[0042] In the encoding process, the first stage of encoding features Structure-aware gating graph After modulation, gated enhancement of shallow features is formed. and participate in subsequent skip connections; the output of the remaining encoding stages and As skip connection features at the corresponding scale, they are used for feature fusion in the decoding stage.

[0043] Bottleneck characteristics Input bottleneck layer Transformer module This yields the global enhanced features: ; in, It is used for global dependency modeling of low-resolution deep features to enhance the continuous representation of thin cracks, branch cracks and weakly textured cracks.

[0044] The global augmentation features are input into the Transformer module of the decoding stage and upsampled step-by-step by the decoder. During the decoding stage, the decoded features at each level are concatenated and fused with the corresponding scale-encoded features through skip connections to recover spatial detail information. Specifically, the global augmentation features are upsampled and combined with the corresponding scale-encoded features. Perform splicing and fusion: ; in, This indicates an upsampling operation or a transposed convolution operation.

[0045] The fused features are processed by the first-stage decoding convolution and the decoding stage Transformer module to obtain the decoded enhanced features: ; in, This indicates the first stage of decoding the convolution operation. This indicates the first stage of decoding the Transformer module.

[0046] Subsequently, the decoder gradually restores the spatial resolution through stepwise upsampling, skip connections, and convolutional decoding operations, and outputs a crack prediction probability map through a segmentation head. The segmentation head can be implemented using 1×1 convolution to map the decoded features into pixel-level crack prediction results; in the crack prediction probability map, each pixel value represents the probability that the pixel belongs to a crack region.

[0047] In this embodiment, the five-level hierarchical structure of the CNN encoder can extract multi-scale features from local texture to deep semantics, but the receptive field of a pure CNN is limited, which can easily lead to the breakage of thin cracks; by introducing a bottleneck layer Transformer module, low-resolution deep features can be extracted. Global dependency modeling captures long-range spatial relationships between crack branches, enhancing continuity and integrity. In the first decoding stage, the Transformer module is introduced again to further optimize the global context of the fused features, alleviating semantic ambiguity caused by skip connections. Upsampling and skip connections restore spatial resolution step by step, and the final segmentation head outputs a pixel-level probability map, thereby effectively restoring the crack structure that is prone to breakage under wet and reflective conditions while maintaining the accuracy of local details, improving segmentation integrity and connectivity.

[0048] S5. Perform structure-aware boundary loss training. To alleviate the imbalance between the number of crack pixels and background pixels, and to improve the crack boundary localization accuracy under wet, waterlogged, and reflective conditions, this embodiment uses the structure-aware boundary loss function to supervise the training of the CNN-Transformer hybrid segmentation network.

[0049] The structure-aware boundary loss function consists of Focal Tversky loss, cross-entropy loss, boundary-aware loss, and surface distance loss, and its total loss function is expressed as: ; in, This indicates Focal Tversky's loss. Represents cross-entropy loss, This represents boundary-aware loss. Indicates surface distance loss, This represents the balance coefficient for each loss term. This represents the linear preheating factor, used to gradually adjust the strength of the boundary-dependent loss term in the total loss function.

[0050] In a preferred embodiment, the linear preheating factor is expressed as: ; in, Indicates the current training round. Indicates the preheating cycle.

[0051] The Focal Tversky loss is used to alleviate the imbalance between the number of crack pixels and background pixels, and to enhance the model's ability to learn about small and weak crack regions. It satisfies the following relationship: The Focal Tversky loss satisfies the following relationship: ; Among them, the Tversky coefficient The following relationship must be satisfied: ; In the formula, Represents the set of image pixels. Represents pixels in an image. Represents pixels The predicted probability of belonging to the crack region. Represents pixels The true label, This represents the weighting coefficient for the false positive term. This represents the weighting coefficient of the false negative term. Indicates the focusing coefficient. This indicates a smoothing term to prevent the denominator from being zero. The binary cross-entropy loss is used to provide pixel-level classification supervision, and it satisfies the following relationship: ; In the formula, This indicates the total number of pixels in the image.

[0052] During the training phase, the CNN-Transformer hybrid segmentation network also includes an edge prediction branch for boundary supervision. This edge prediction branch obtains the edge prediction logit from the decoded features through convolutional mapping. The edge prediction logit It is only used to calculate the boundary-aware loss; during the model application stage, the edge prediction branch may not be used as the final output, and the final output is still the crack prediction probability map and the binary crack mask map converted from it.

[0053] The boundary-aware loss is used to enhance the supervision strength of the real crack boundary region. During the training phase, a real crack mask is used. Generate soft edge labels and edge weights The edge prediction branch outputs the edge prediction logit. The boundary-aware loss satisfies the following relationship: ; In the formula, This represents the soft edge label and edge weight obtained from the gradient of the real crack mask edge. This indicates that the edge prediction branch is at the pixel point. The output logit, This represents the Sigmoid function. Represents pixels The predicted probability of belonging to the crack boundary region.

[0054] In one alternative implementation, the Sobel operator is used to calculate the actual crack mask. Horizontal and vertical gradients: ; ; And calculate pixel points Edge gradient magnitude at: ; Normalize the edge gradient magnitudes to obtain soft edge labels and edge weights: ; In the formula, and These represent the Sobel horizontal convolution kernel and the Sobel vertical convolution kernel, respectively. This represents the convolution operation. and Representing pixels The horizontal and vertical gradients at the location, Represents pixels Edge gradient magnitude at the location, Represents the set of image pixels Any pixel in, This represents the truncation function. Through the boundary-aware loss described above, the model focuses more on the real crack boundary region during the training phase, thereby improving the problems of blurred crack boundaries and boundary localization errors under wet, waterlogged, or reflective conditions.

[0055] The surface distance loss is used to constrain the spatial deviation between the predicted region and the actual crack boundary, and it satisfies the following relationship: ; In the formula, Represents pixels The signed distance to the nearest real crack boundary Represents the absolute value of the corresponding distance. Indicates the distance cutoff threshold. Represents pixels The predicted probability of belonging to the crack region.

[0056] In this embodiment, for the composite loss function, and This separately mitigates the extreme class imbalance between cracks and the background and provides pixel-by-pixel classification supervision; Enhance the supervision intensity of the crack boundary neighborhood, so that the model focuses on the edge region; By directly constraining the spatial deviation between the predicted probability and the actual crack boundary using a signed distance function, the positioning offset caused by water film blurring is reduced; the linear preheating factor is reduced in the early stage of training. and The weights are adjusted to prevent boundary supervision from interfering with the learning of core features too early. The role of boundary supervision is gradually increased in the later stages, so that the boundary can be finely optimized after the model has reached stable convergence. This solves the problems of class imbalance, boundary ambiguity and localization offset at the same time, and improves the edge sharpness and spatial consistency of the segmentation results.

[0057] In the surface distance loss function, for each pixel p, its predicted probability is multiplied by a truncation value of the signed distance to the nearest real crack boundary, and the result is averaged over all pixels in the image. The physical meaning of this loss function is that pixels far from the real crack boundary (larger distance) are penalized even with higher predicted probabilities, thus suppressing high-confidence false detections in background areas (especially reflective artifacts or complex texture areas); simultaneously, the distance truncation threshold... By limiting the influence of distant pixels, the loss term primarily applies to the region near the boundary, avoiding excessive constraint on the distant background. By directly coupling the predicted probability with the spatial distance, this loss term can guide the network to output a probability distribution that is closer to the true boundary geometry. It is particularly suitable for scenarios with blurred boundaries and shifted predicted regions under wet and reflective conditions, effectively improving the localization accuracy and shape fidelity of the segmentation results.

[0058] In one specific embodiment, model training is implemented using the PyTorch framework, and the training device is an NVIDIA RTX3080 GPU. The initial learning rate is set to 1×10⁻⁶. -4 The optimizer used is AdamW, with a weight decay coefficient of 0.01, a batch size of 4, and an early stopping strategy based on validation set IoU, with an early stopping patience value of 20. The above training parameters are merely one optional implementation in this embodiment and do not constitute a limitation of the present invention.

[0059] This application also provides an application of an image crack recognition model based on the generation method in the first aspect for crack recognition in complex road surface images: S6. Output crack segmentation results. After model training is completed, the RGB image of the wet reflective road surface to be detected is input into the trained pseudo-multimodal collaborative segmentation model. First, the pseudo-multimodal generation module generates the corresponding pseudo-modal feature set, then the structure-aware gating module generates a gating map and modulates shallow visual features, and finally the crack prediction probability map is output through the CNN-Transformer hybrid segmentation network.

[0060] Based on a preset threshold or post-processing strategy, the crack prediction probability map is converted into a binary crack mask map, resulting in pixel-level segmentation of the wet, reflective pavement crack image. The crack prediction probability map is a continuous pixel probability value map, representing the probability that each pixel belongs to a crack region; the binary crack mask map is the final segmentation result, representing the crack region and the background region. The binary crack mask map can be used for crack region extraction, crack connectivity analysis, crack area statistics, crack width estimation, or road distress evaluation.

[0061] In this embodiment, after the RGB image to be identified is input into the trained model, a corresponding pseudo-modal feature set is first generated. Then, through modulation, encoding, and decoding, a crack prediction probability map is obtained. Finally, it is converted into a binary crack mask image through a preset threshold or post-processing. This application process is completely consistent with the forward propagation in the training phase, ensuring semantic alignment between training and inference. The pseudo-modal generation module does not rely on any auxiliary hardware and can construct structural priors in real time through mathematical operations, resulting in high inference efficiency. Post-processing strategies (such as threshold segmentation, morphological opening and closing, etc.) can be flexibly adjusted according to specific engineering needs, converting the continuous probability map into a clear binary mask, which facilitates subsequent crack area statistics, width estimation, connectivity analysis, and road maintenance evaluation. This realizes a complete automated link from the original image to a segmentation result that can be directly used for engineering decisions.

[0062] like Figure 5 As shown, to verify the effectiveness of the method in this embodiment, crack images under wet road surface, waterlogged road surface and highly reflective road surface conditions are selected for segmentation results. Figure 5 The segmentation result of this invention is a binary crack mask image obtained by threshold segmentation or post-processing of the crack prediction probability map, which is used to intuitively characterize the final crack segmentation effect.

[0063] In one specific embodiment, the method of this embodiment is compared with segmentation methods such as U-Net, DeepLabv3+, and SegFormer-B3. Evaluation metrics include one or more of the following: Cross-Union Ratio (CUNR), Dice coefficient, precision, recall, boundary F1 score, and mean CUNR. On the test set, this method achieved a CUNR of 63.40%, a Dice coefficient of 77.60%, a precision of 73.39%, a recall of 82.33%, a boundary F1 score of 76.36%, and a mean CUNR of 81.05%. These results demonstrate that the present invention can improve the accuracy and boundary integrity of crack region extraction under wet reflective conditions.

[0064] In the three sub-scenes of wet road surface, waterlogged road surface, and highly reflective road surface, this method maintains good segmentation results. For wet road surface, this method can enhance the recognition ability of low-contrast crack structures; for waterlogged road surface, this method can improve the problem of blurred crack boundaries caused by water film or water accumulation; for highly reflective road surface, this method can reduce false detections caused by specular reflection, bright stripes, and non-crack linear structures.

[0065] Furthermore, module ablation experiments revealed that the pseudo-multimodal generation module, the structure-aware gating module, and the structure-aware boundary loss function all improve the crack segmentation results. Specifically, the pseudo-multimodal generation module enhances the expression of crack structure; the structure-aware gating module suppresses unstructured reflective interference; and the structure-aware boundary loss function improves crack boundary localization. The synergistic effect of these three components gives the invention better segmentation stability and engineering applicability under humid, waterlogged, and reflective conditions.

[0066] This application embodiment also provides a crack identification system for identifying road surface cracks, including a processor and a memory; Memory, used to store computer programs; The processor, when executing the program stored in memory, implements the application steps of the image crack recognition model in crack recognition of complex road surface images.

[0067] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for generating an image crack recognition model for identifying road surface cracks, characterized in that, include: S1. Obtain the original image of the target road surface, extract features from the original image to obtain a pseudo-modal feature set, and concatenate the features of the original image to obtain a collaborative input tensor; The process of extracting features from the original image yields a set of pseudomodal features, including: The process involves acquiring an RGB image of the target road surface as the original image and converting it to a grayscale image; extracting dark structure responses from the grayscale image using local maximum morphological operations at different scales to obtain grayscale structure features and dark structure features; performing Gaussian smoothing on the grayscale image and calculating the Laplacian negative response to obtain Laplacian response features; constructing reflectivity-related response features based on the grayscale image using local minimum morphological operations; normalizing the grayscale structure features, dark structure features, Laplacian response features, and reflectivity-related response features, and then combining the normalized features into a four-channel pseudomodal feature set. S2. Modulate the pseudo-modal feature set and the cooperative input tensor to obtain gated enhanced shallow features; S3. Perform feature extraction, encoding and decoding on the gated enhanced shallow features to obtain a crack prediction probability map; S4. Construct an original image crack recognition model. Establish a model loss function based on the pixels in the crack prediction probability map. Train the original image crack recognition model using the model loss function to obtain a trained image crack recognition model.

2. The image crack recognition model generation method for identifying road surface cracks according to claim 1, characterized in that, In S1: The grayscale structural features satisfy the following relationship: ； Dark structure features satisfy the following relationship: ； In the formula, Represents grayscale structural features. Indicates dark structure features, Representing scale Local maximum morphological operations under the following conditions Represents the local neighborhood scale of grayscale structure. The local neighborhood scale represents the feature scale of dark structures. Represents a grayscale image; The Laplace response characteristics satisfy the following relationship: ； In the formula, Indicates the Laplace response characteristics, Indicates the Gaussian smoothing kernel. This represents the convolution operation. Represents the discrete Laplace operator. Represents a non-linear activation function; The reflectivity-related response characteristics satisfy the following relationship: ； In the formula, Indicates the reflective response characteristics, The neighborhood scale representing local differences in reflectance. Indicated in scale Local minimal morphological operations.

3. The image crack recognition model generation method for identifying road surface cracks according to claim 2, characterized in that, In step S1, feature concatenation of the original image is performed to obtain a collaborative input tensor, including: The pseudo-modal feature set is concatenated with the RGB image along the channel direction to obtain a collaborative input tensor, wherein the collaborative input tensor satisfies the following relationship: ； In the formula, Represents the collaborative input tensor. This indicates a channel splicing operation. Represents an RGB image. This represents the set of pseudomodal features.

4. The method for generating an image crack recognition model for identifying road surface cracks according to any one of claims 1-3, characterized in that, The acquisition of the original image of the target road surface includes: Original RGB images were captured using road image acquisition equipment with RGB image acquisition capabilities, including those of wet road surfaces, waterlogged road surfaces, and highly reflective road surfaces. The crack regions in the acquired original RGB image are annotated at the pixel level to generate crack mask labels corresponding to the original RGB image. In the crack mask labels, the crack regions and the background regions are distinguished by different pixel values. The original RGB image and its corresponding crack mask label are preprocessed to construct a sample set for model training, validation and testing; the preprocessing includes one or more of image size normalization, pixel value normalization, data augmentation and sample set partitioning; the data augmentation includes one or more of random flipping, random rotation, random cropping, brightness perturbation and contrast perturbation.

5. The method for generating an image crack recognition model for identifying road surface cracks according to any one of claims 1-3, characterized in that, In step S2, feature modulation is performed on the pseudo-modal feature set and the cooperative input tensor to obtain gated enhanced shallow features, including: A structure-aware gated graph is generated based on the pseudo-modal feature set, wherein the structure-aware gated graph satisfies the following relationship: ； ； In the formula, Represents a structure-aware gated graph. This represents the convolution operation. This indicates a batch normalization operation. Represents a non-linear activation function. This represents the Sigmoid activation function. and Represents the convolution weights. Indicates the bias term. This represents the intermediate features obtained from the pseudomodal feature mapping; Shallow features are generated based on the collaborative input tensor, wherein the shallow features satisfy the following relationship: ； In the formula, Represents the collaborative input tensor. Indicates shallow features. This indicates the first-stage encoder; The shallow features are element-modulated using the structure-aware gating map to obtain gated enhanced shallow features, which satisfy the following relationship: ； In the formula, This represents element-wise multiplication. This represents the number of channels in the shallow feature map. This indicates gating enhancement of shallow features.

6. The method for generating an image crack recognition model for identifying road surface cracks according to any one of claims 1-3, characterized in that, In step S3, feature extraction, encoding, and decoding are performed on the gated enhanced shallow features to obtain a crack prediction probability map, including: Constructing a five-level coding phase, for the first Each encoding stage, its feature extraction process satisfies the following relationship: ； In the formula, This indicates a downsampling operation. Indicates the first Convolutional transformation in each encoding stage, Indicates the first Level feature map; The first-level feature map in the first encoding stage is replaced with the gated enhanced shallow features. Through the feature extraction process of each encoding stage, the corresponding feature map is obtained. After the five-level encoding stage is completed, a global enhanced feature is generated based on the fifth-level feature map. The global enhanced feature satisfies the following relationship: ； In the formula, This indicates that global dependency modeling is performed on low-resolution deep features. This represents the 5th level feature map. Indicates global enhancement features; The global enhancement features are upsampled and then concatenated with the level 4 feature map to obtain the fused features, which satisfy the following relationship: ； In the formula, This indicates an upsampling operation or a transposed convolution operation. This represents the 4th level feature map. Indicates fusion characteristics; The fused features are decoded to obtain decoded enhanced features, which satisfy the following relationship: ； In the formula, This represents the first stage of the five-level encoding process, the decoding convolution operation. This refers to the first stage of the five-level encoding process, the decoding Transformer module. Indicates decoding enhancement features; Spatial resolution is gradually restored by performing stepwise upsampling, skip connections, and convolutional decoding operations on the decoded enhancement features, and a crack prediction probability map is output.

7. The method for generating an image crack recognition model for identifying road surface cracks according to any one of claims 1-3, characterized in that, In step S4, a model loss function is established based on the pixels in the crack prediction probability map. The image crack recognition model is trained using the model loss function to obtain the trained image crack recognition model, including: Determine the predicted probability of each pixel in the crack prediction probability map, and construct a surface distance loss function based on the predicted probability of each pixel; A total loss function is constructed based on the surface distance loss function, Focal Tversky loss function, binary cross-entropy loss function, and boundary-aware loss function, wherein the total loss function satisfies the following relationship: ； In the formula, This represents the Focal Tversky loss function. Represents the cross-entropy loss function. This represents the boundary-aware loss function. Represents the surface distance loss function. ( i =1,2,3,4) represents the balance coefficients of each loss term, Indicates the linear preheating factor. Represents the total loss function; The linear preheating factor satisfies the following relationship: ； In the formula, Indicates the current training round. Indicates the preheating cycle; The image crack recognition model is trained by using the total loss function to obtain the trained image crack recognition model.

8. The method for generating an image crack recognition model for identifying road surface cracks according to claim 7, characterized in that, The surface distance loss function satisfies the following relationship: ； In the formula, Represents the set of image pixels. Represents pixels The signed distance to the nearest real crack boundary Represents the absolute value of the corresponding distance. Indicates the distance cutoff threshold. Represents pixels Predicted probability of belonging to the crack region; The Focal Tversky loss function satisfies the following relationship: ； ； The binary cross-entropy loss function satisfies the following relationship: ； The boundary-aware loss function satisfies the following relationship: ； In the formula, Represents the set of image pixels. Indicates the total number of pixels in the image. Represents pixels in an image. Represents pixels The predicted probability of belonging to the crack region. Represents pixels The true label, This represents the Tversky coefficient. This represents the weighting coefficient for the false positive term. This represents the weighting coefficient of the false negative term. Indicates the focusing coefficient. This indicates a smoothing term to prevent the denominator from being zero. This represents the soft edge label and edge weight obtained from the gradient of the real crack mask edge. This indicates that the edge prediction branch is at the pixel point. The output logit, This represents the Sigmoid function.

9. An application of an image crack recognition model obtained based on the generation method of any one of claims 1-8 in crack recognition of complex road surface images, characterized in that, After steps S1-S4, also Includes the following steps S5: The wet, reflective pavement image to be identified is input into the trained image crack recognition model to obtain a binary crack mask image. This binary crack mask image is then used as the result generated by the image crack recognition model for identifying pavement cracks. Specifically, this includes: The RGB image of the wet, reflective pavement to be identified is used as the input of the trained image crack recognition model. The trained image crack recognition model generates a corresponding pseudo-modal feature set based on the RGB image of the wet, reflective pavement to be identified. The pseudo-modal feature set is then modulated to obtain shallow visual features. The shallow visual features are then encoded and decoded to obtain the crack prediction probability map corresponding to the RGB image of the wet reflective road surface to be identified. Based on a preset threshold or post-processing strategy, the crack prediction probability map is converted into a binary crack mask map.

10. A crack identification system for identifying road surface cracks, characterized in that, Including processor and memory; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the application as described in claim 9.

Images