Road crack segmentation model based on Transform enhancement, construction method and use method

Abstract

The invention discloses a road crack segmentation model based on Transform enhancement, a construction method and a use method, and relates to the technical field of image processing. According to the method, a multi-scale cross-level module (MSAF) is provided, and overall image global semantics and 2 * 2 local block features are explicitly fused by embedding a multi-head attention mechanism of two-dimensional sine position coding. The module not only retains edge details of the crack, but also captures long-range context information, so that the crack detection capability of the model is remarkably improved, and additional parameters do not need to be added; a global-local double-flow coding architecture, namely MSAFNet, is designed, a parallel double-flow coding architecture is adopted, global flow processing is performed on the whole image, local branch processing is performed on 2 * 2 cutting, and seamless recombination is performed after the features of the whole image and the local branch processing are fused through an MSAF module, so that fine-grained texture information of cracks is reserved, and global context information is captured; therefore, the high-precision crack segmentation capability can be maintained in different scenes.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a road crack segmentation model based on Transformer enhancement, its construction method, and its usage method. Background Technology

[0002] Cracks are the most typical and easily evolving early structural defects in asphalt and concrete pavements. Their width, length, and connectivity directly determine the remaining lifespan of the road, the level of driving safety, and the priority of maintenance. Traditional manual inspections rely on experience-based judgment, resulting in low efficiency, high subjectivity, and a high rate of missed detections, which can no longer meet the daily maintenance needs of a highway network spanning tens of thousands of kilometers. Meanwhile, the rapid proliferation of inspection vehicles, drones, and handheld smart terminals has made real-time analysis of massive amounts of high-definition imagery possible. Pixel-level crack segmentation can accurately depict the geometric contours of cracks, and then, in conjunction with BIM, digital twin platforms, and spatiotemporal big data, provide quantitative decision-making basis for predictive maintenance, budget allocation, and robotic inspections. Therefore, researching crack segmentation algorithms that combine high accuracy and strong generalization ability has become an important cutting-edge topic in the interdisciplinary field of computer vision, intelligent transportation, and civil engineering.

[0003] Despite this, existing deep learning semantic segmentation models still face a "triple dilemma" in real-world road scenarios. First, cracks account for a very small percentage of pixels (usually less than 3%), making them easily lost in textured backgrounds and leading to numerous false negatives. Second, public datasets are small in size and suffer from severe annotation noise: taking the largest dataset to date, Crack500, as an example, the 250 training images commonly contain missing labels for thin cracks, misaligned boundaries, and false positive labels. Human annotators struggle to accurately identify fine cracks due to variations in resolution, sharpness, motion blur, and lighting, resulting in systematic biases in the supervision signal. Third, cross-scene domain shifts caused by lighting, shooting angle, road surface material, stains, and seasonal changes cause models trained on Crack500 to experience a sharp drop in IoU exceeding 10% in unseen scenarios such as CFD. While unsupervised, semi-supervised, or adversarial methods reduce the reliance on accurate labels, a trade-off between accuracy, speed, and deployment cost remains difficult. Therefore, how to simultaneously improve the accuracy and robustness of crack segmentation under the constraints of "small data, high noise, and strong cross-domain challenges" remains a critical problem that urgently needs to be solved.

[0004] Therefore, a new solution is needed to address the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide a road crack segmentation model based on Transformer enhancement, as well as a method for its construction and use, in order to solve the technical problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for constructing a road crack segmentation model based on Transformer enhancement, comprising at least the following steps:

[0007] The MSAFNet framework is built. The MSAFNet framework is a global-local dual-stream coding architecture. It extends the U-Net architecture by introducing a dual-stream encoder, which can process global and local features.

[0008] The global stream encodes the entire input image to capture long-range contextual information, while the local stream processes 2×2 image patches to preserve fine-grained crack details;

[0009] A MSAF module is built to fuse global and local streams. This MSAF module uses a multi-head attention mechanism with 2D sinusoidal position encoding to align and integrate multi-scale features.

[0010] A decoder is built, and the fused features are then passed to the decoder to generate the final segmentation map;

[0011] The Dice-Focal dynamic weighted loss function is introduced to mitigate annotation noise and class imbalance. The input image is processed to output a binary segmentation map, where H and W represent the height and width, respectively.

[0012] Furthermore, the dual-stream encoder consists of two parallel branches: a global encoder and a local encoder.

[0013] Both branches use a series of DoubleConv modules, each containing two 3×3 convolutions with batch normalization (BN) and ReLU activation functions, defined as follows:

[0014] DoubleConv(X)=ReLU(BN(Conv(ReLU(Conc(X))))) (1)

[0015] Where BN is batch normalization and ReLU is the activation function;

[0016] The global encoder processes the input image X∈R3×H×W through four dual convolutional modules with 64, 128, 256 and 512 channels respectively, and intersperses 2×2 max pooling (stride 2) between the modules.

[0017] The output of the ii-th global encoder is:

[0018]

[0019] Where Ei is the feature map of the i-th global encoder. It is the height and width after i max pooling operations, C i It is the number of channels in the i-th layer;

[0020] Deepest features Reduced to 256 channels via 1×1 convolution, generating a global feature map for MSAF fusion:

[0021]

[0022] The local encoder divides the input image X into 2×2 non-overlapping blocks and reshapes it as follows:

[0023]

[0024] Where B is the batch size;

[0025] These blocks are processed through three double convolutional modules with 64, 128, and 256 channels respectively, interspersed with max pooling, to generate:

[0026]

[0027] Among them, E 3,loc It is a local encoder feature map after passing through three modules, which retains more crack details. This dual-stream coding design ensures that crack edges can be accurately detected while capturing the global context, effectively addressing the challenge of low crack pixel ratio.

[0028] Furthermore, the MSAF module integrates global and local features by employing a multi-head attention mechanism with 2D sinusoidal positional encoding;

[0029] in, and These represent local features and global features, respectively.

[0030] The operation procedure of the MSAF module is as follows:

[0031] To incorporate spatial information, 2D sinusoidal position coding is applied to L and G. For a feature map of size B×H×W, the position coding... The definition method is as follows:

[0032]

[0033] Where x and y are normalized spatial coordinates, θ = 10000, and C = 256 is the channel dimension;

[0034] Encoding features include:

[0035] L=L+PE(B·4,H / 8,W / 8),G=G+PE(B,H / 8,W / 8)(8)

[0036] The local feature L is reshaped into B×4×256×H / 8×W / 8, and each patch L[:,i,:,:,:] (i=0,1,2,3) is flattened into (H / 8·W / 8)×B×256; the global feature G is flattened into (H / 8·W / 8)×B×256.

[0037] Four multi-head attention modules process each patch separately:

[0038] O i =MultiHeadAttention(L[:,i:,:,:],G,G)i=0,1,2,3 (9)

[0039] in,

[0040] The output is concatenated along the sequence dimension to form

[0041] The concatenated output is then processed sequentially through layers of normalization and two layers of feedforward neural networks (FFN):

[0042] O=LayerNorm(O+Dropout(ReLU(O)))) (10)

[0043] The final output was reshaped as And reassembled into For use by the decoder.

[0044] Furthermore, the decoder upsamples the bottleneck feature E4 through transposed convolution and concatenates it with the skip connections and fused features E3,fused from the global encoder.

[0045] Each decoder stage applies a double convolutional block:

[0046] D i =DoubleConv(Concat(Up(D i+1 ),E i ),i=3,2,1,0 (11)

[0047] Where Up is a 2×2 transposed convolution, and D4 = DoubleConv(Concat(Up(E4), E3, fused));

[0048] The final output is obtained through a 1×1 convolution: Y = Conv 1×1 (D1)(12).

[0049] Furthermore, the Dice-Focal dynamic weighted loss function combines Dice loss and Focal loss;

[0050] Dice loss ensures the overlap between the prediction and the true value. The definition of Dice loss is:

[0051]

[0052] in, These are the predicted values ​​after sigmoid activation, where y is the true label and ε = 10. -6 These are factors used for smoothing, where i and j represent the spatial indexes of the image;

[0053] Focal loss addresses annotation noise by focusing on hard samples, and its expression is:

[0054]

[0055] in, When y i,j =1, otherwise α = 0.75 represents the weighting factor for positive samples, γ = 2 emphasizes the focusing parameter for hard samples, and BCE represents the binary cross-entropy loss.

[0056] The total loss dynamically balances both, expressed as:

[0057] L = L Dice +0.5·L Focal (15)

[0058] The proposed Dice-Focal dynamic weighted loss function improves the robustness of the model by mitigating the false negative problem caused by noisy annotations and addressing the challenge of low crack pixel ratio.

[0059] The road crack segmentation model based on Transformer enhancement was constructed using the method for constructing a road crack segmentation model based on Transformer enhancement.

[0060] The method of using the Transformer-enhanced road crack segmentation model includes at least the following steps:

[0061] S1: Input preprocessing, input image After normalization, the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] are obtained, and the image is divided into 2×2 non-overlapping blocks for local processing, while the entire image is preserved for global processing.

[0062] S2: Dual-stream encoding. The global stream applies convolutional blocks to capture long-range contextual information, generating feature maps (E1, E2, E3, E4). The local stream processes 2×2 blocks to preserve fine-grained crack details, generating E...3,loc ;

[0063] S3: Multi-scale attention fusion. The MSAF module utilizes a multi-head attention mechanism and two-dimensional sinusoidal position encoding to integrate local features E. 3,loc The feature E is then fused with the global feature E4 after channel dimensionality reduction to generate the fused feature E. 3,fused ;

[0064] S4: Decoding, the decoder upsamples E4 and connects it to E4 via a skip connection. 3,fused Concatenate with global encoder features to generate a segmentation map.

[0065] S5: Loss function. The Dice-Focal dynamic weighted loss function is used to optimize the model. By focusing on hard samples and balancing cracks and background pixels, the annotation noise and class imbalance problems are reduced.

[0066] Compared with the prior art, the beneficial effects of the present invention are:

[0067] 1. This invention proposes a Multi-Scale Cross-Level Module (MSAF), which explicitly fuses global semantics of the entire image with 2×2 local block features through a multi-head attention mechanism that embeds 2D sinusoidal positional encoding. This module not only preserves the edge details of cracks but also captures long-range contextual information, significantly improving the model's ability to detect cracks without adding additional parameters;

[0068] 2. This invention designs a global-local dual-stream coding architecture, namely MSAFNet, which adopts a parallel dual-stream coding architecture. The global stream processes the entire image, and the local branch processes 2×2 blocks. The features of the two are fused and seamlessly recombined through the MSAF module, which not only preserves the fine-grained texture information of the cracks, but also captures the global context information, thus maintaining high-precision crack segmentation capability in different scenarios.

[0069] 3. This invention introduces a dynamic weighted hybrid loss function: To address the issues of annotation noise and class imbalance, we introduce the Dice-Focal dynamic weighted hybrid loss function. This loss function effectively suppresses false negatives caused by label noise by adjusting the weights of positive samples online, while also alleviating the class imbalance between crack pixels and background pixels, significantly improving the robustness of the model. Attached Figure Description

[0070] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a diagram of the MASFNet framework of the present invention. Detailed Implementation

[0072] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0073] Pixel-level accurate segmentation of road cracks is a core component for ensuring road safety inspection, lifespan prediction, and intelligent maintenance decisions. However, the extremely low pixel ratio of cracks, severe annotation noise, and significant differences in lighting and texture across scenes make it difficult for existing methods to balance accuracy and generalization on public Crack500 and cross-domain CFD datasets. To address this, we propose MASFNet, which explicitly fuses global semantics of the entire image with 2×2 local block features by embedding a multi-head attention multi-scale cross-layer module with 2D sinusoidal positional encoding. Parallel dual-stream encoding captures long-range context while preserving crack edge details, and a Dice-Focal dynamic weighted hybrid loss is introduced to suppress label noise and class imbalance online.

[0074] Example 1:

[0075] Please see Figure 1 The method for constructing a road crack segmentation model based on Transformer enhancement includes at least the following steps:

[0076] The MSAFNet framework is built. The MSAFNet framework is a global-local dual-stream coding architecture. It extends the U-Net architecture by introducing a dual-stream encoder, which can process global and local features.

[0077] The global stream encodes the entire input image to capture long-range contextual information, while the local stream processes 2×2 image patches to preserve fine-grained crack details;

[0078] A MSAF module is built to fuse global and local streams. This MSAF module uses a multi-head attention mechanism with 2D sinusoidal position encoding to align and integrate multi-scale features.

[0079] A decoder is built, and the fused features are then passed to the decoder to generate the final segmentation map;

[0080] The Dice-Focal dynamic weighted loss function is introduced to mitigate annotation noise and class imbalance. The input image is processed to output a binary segmentation map, where H and W represent the height and width, respectively.

[0081] A dual-stream encoder consists of two parallel branches: a global encoder and a local encoder.

[0082] Both branches use a series of DoubleConv modules, each containing two 3×3 convolutions with batch normalization (BN) and ReLU activation functions, defined as follows:

[0083] DoubleConv(X)=ReLU(BN(Conv(ReLU(Conc(X))))) (1)

[0084] Where BN is batch normalization and ReLU is the activation function;

[0085] The global encoder processes the input image X∈R3×H×W through four double convolutional modules with 64, 128, 256 and 512 channels respectively, and intersperses 2×2 max pooling (stride 2) between modules;

[0086] The output of the iii-th global encoder is:

[0087]

[0088] Where Ei is the feature map of the i-th global encoder. It is the height and width after i max pooling operations, C i It is the number of channels in the i-th layer;

[0089] Deepest features Reduced to 256 channels via 1×1 convolution, generating a global feature map for MSAF fusion:

[0090]

[0091] The local encoder divides the input image X into 2×2 non-overlapping blocks and reshapes it as follows:

[0092]

[0093] Where B is the batch size;

[0094] These blocks are processed through three double convolutional modules with 64, 128, and 256 channels respectively, interspersed with max pooling, to generate:

[0095]

[0096] Among them, E 3,locIt is a local encoder feature map after passing through three modules, which retains more crack details. This dual-stream coding design ensures that crack edges can be accurately detected while capturing the global context, effectively addressing the challenge of low crack pixel ratio.

[0097] The MSAF module integrates global and local features by employing a multi-head attention mechanism with 2D sinusoidal positional encoding;

[0098] in, and These represent local features and global features, respectively.

[0099] The operation procedure for the MSAF module is as follows:

[0100] To incorporate spatial information, 2D sinusoidal position coding is applied to L and G. For a feature map of size B×H×W, the position coding... The definition method is as follows:

[0101]

[0102] Where x and y are normalized spatial coordinates, θ = 10000, and C = 256 is the channel dimension;

[0103] Encoding features include:

[0104] L=L+PE(B·4,H / 8,W / 8),G=G+PE(B,H / 8,W / 8) (8)

[0105] The local feature L is reshaped into B×4×256×H / 8×W / 8, and each patch L[:,i,:,:,:] (i=0,1,2,3) is flattened into (H / 8··W / 8)×B×256; the global feature G is flattened into (H / 8·W / 8)×B×256.

[0106] Four multi-head attention modules process each patch separately:

[0107] O i =MultiHeadAttention(L[:,i:,:,:],G,G)i=0,1,2,3 (9)

[0108] in,

[0109] The output is concatenated along the sequence dimension to form

[0110] The concatenated output is then processed sequentially through layers of normalization and two layers of feedforward neural networks (FFN):

[0111] O=LayerNorm(O+Dropout(ReLU(O)))) (10)

[0112] The final output was reshaped as And reassembled into For use by the decoder.

[0113] The decoder upsamples the bottleneck feature E4 through transposed convolution and concatenates it with the skip connections and fused features E3,fused from the global encoder.

[0114] Each decoder stage applies a double convolutional block:

[0115] D i =DoubleConv(Concat(Up(D i+1 ),E i ),i=3,2,1,0 (11)

[0116] Where Up is a 2×2 transposed convolution, and D4 = DoubleConv(Concat(Up(E4), E3, fused));

[0117] The final output is obtained through a 1×1 convolution: Y = Conv 1×1 (D1) (12).

[0118] The Dice-Focal dynamic weighted loss function combines Dice loss and Focal loss;

[0119] Dice loss ensures the overlap between the prediction and the true value. The definition of Dice loss is:

[0120]

[0121] in, These are the predicted values ​​after sigmoid activation, where y is the true label and ε = 10. -6 These are factors used for smoothing, where i and j represent the spatial indexes of the image;

[0122] Focal loss addresses annotation noise by focusing on hard samples, and its expression is:

[0123]

[0124] in, When y i,j =1, otherwise α = 0.75 represents the weighting factor for positive samples, γ = 2 emphasizes the focusing parameter for hard samples, and BCE represents the binary cross-entropy loss.

[0125] The total loss dynamically balances both, expressed as:

[0126] L = L Dice +0.5·L Focal (15) The Dice-Focal dynamic weighted loss function improves the robustness of the model by mitigating the false negative problem caused by noise annotation and solving the challenge of low crack pixel ratio.

[0127] Example 2:

[0128] The road crack segmentation model based on Transformer enhancement was constructed using the above-described embodiment one.

[0129] Example 3:

[0130] The method for using the Transformer-enhanced road crack segmentation model proposed in Embodiment 2 above includes at least the following steps:

[0131] S1: Input preprocessing, input image After normalization, the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] are obtained, and the image is divided into 2×2 non-overlapping blocks for local processing, while the entire image is preserved for global processing.

[0132] S2: Dual-stream encoding. The global stream applies convolutional blocks to capture long-range contextual information, generating feature maps (E1, E2, E3, E4). The local stream processes 2×2 blocks to preserve fine-grained crack details, generating E... 3,loc ;

[0133] S3: Multi-scale attention fusion. The MSAF module utilizes a multi-head attention mechanism and two-dimensional sinusoidal position encoding to integrate local features E. 3,loc The feature E is then fused with the global feature E4 after channel dimensionality reduction to generate the fused feature E. 3,fused ;

[0134] S4: Decoding, the decoder upsamples E4 and connects it to E4 via a skip connection. 3,fused Concatenate with global encoder features to generate a segmentation map.

[0135] S5: Loss function. The Dice-Focal dynamic weighted loss function is used to optimize the model. By focusing on hard samples and balancing cracks and background pixels, the annotation noise and class imbalance problems are reduced.

[0136] Based on the above embodiments one to three, the following experiments are proposed:

[0137] Experimental setup

[0138] All experiments were conducted on a single NVIDIA GeForce RTX 3090 (24GB GDDR6X) system running Ubuntu 20.04 LTS with driver version 535.104, and CUDA 11.8 and cuDNN 8.7 installed together. The deep learning framework used was PyTorch 2.0, Python 3.9, and GCC 9.4 compiler to ensure source code compatibility. At the code level, torch.cuda.amp was enabled for automatic mixed-precision training: forward computation used FP16, backward gradients maintained FP32, and with GradScaler dynamic scaling, memory usage was reduced from 19GB to 10GB, resulting in a 1.4x speedup.

[0139] During training, the batch size is fixed at 8, num_workers = 4, and pin_memory = True to avoid CPU-GPU data transfer bottlenecks. The optimizer used is Adam, with an initial learning rate of 1e-4 and weight decay of 1e-5. Training for 200 epochs takes approximately 6 hours. The learning rate is scheduled using ReduceLROnPlateau; the learning rate is halved if the validation loss does not improve within 5 epochs. Gradient calculation and random augmentation are disabled during the inference phase. Forward pass of a single 256×256 image takes 27 milliseconds, corresponding to 37 FPS, meeting the real-time inspection requirements of vehicles or drones.

[0140] Dataset

[0141] This invention selects two publicly available crack segmentation benchmarks: CRACK500 and CrackForest (CFD), covering various scenarios of urban roads and concrete surfaces, to ensure the repeatability and generalization of the experiment.

[0142] CRACK500

[0143] CRACK500 is one of the main datasets used in this invention. Provided by Temple University, this dataset contains 500 high-resolution mobile phone photos showcasing environments primarily featuring urban areas and pedestrians. The CRACK500 images are 2,000 × 1,500 pixels in size, which helps to better utilize this detail for training the segmentation model. Each image was cropped into 16 non-overlapping regions for analysis, resulting in a total of 3,363 images: 2,354 for training, 673 for validation, and 336 for testing. (CrackForest Dataset (CFD))

[0144] The CrackForest Dataset provides 118 road surface images with a resolution of 320×480 pixels for crack detection and segmentation. The detailed annotations in the CrackForest dataset allow for training models to identify cracks under various conditions. The high-resolution images can capture even small cracks, making this dataset extremely useful for more detailed crack detection. By training the model using the CrackForest Dataset, the model will be exposed to more crack patterns, thus improving its generalization ability under different surface and environmental conditions. The dataset consists of 83 training images, 23 validation images, and 12 test images.

[0145] It is important to emphasize that this invention is trained only on the CRACK500 dataset training set, while the CFD dataset is used only for testing and is not involved in the model training process. In this way, we can more rigorously evaluate the model's generalization ability on unseen data.

[0146] Evaluation indicators

[0147] To comprehensively and objectively evaluate the performance of MSAFNet in the crack segmentation task, this invention adopts five universally recognized metrics in the semantic segmentation field. All metrics are calculated at the pixel level and macro-averaged across the test set images to ensure fair comparisons between different samples. The specific definitions and calculation formulas for each metric are as follows:

[0148] 1. Intersection over Union (IoU)

[0149] IoU is defined as the ratio of the intersection area to the union area of ​​the predicted crack and the actual crack. It directly quantifies the degree of overlap of the segmented regions and is the most discriminative indicator in the current crack detection literature.

[0150]

[0151] Wherein, TP (TruePositive) represents the number of pixels correctly predicted as cracks, FP (FalsePositive) represents the number of pixels incorrectly predicted as cracks, and FN (FalseNegative) represents the number of crack pixels that were not correctly predicted.

[0152] 2. Precision

[0153] Precision measures the proportion of pixels predicted as cracks that are actually cracks, and is used to evaluate the model's false positive rate.

[0154]

[0155] 3. Recall

[0156] Recall rate reflects the proportion of all real crack pixels that are correctly detected and is a core indicator for measuring the risk of missed detections.

[0157]

[0158] 4. Accuracy

[0159] Accuracy describes the overall pixel classification accuracy. Since background pixels are the dominant element, this metric is often close to 1 in the crack task, but it can still reflect the model's grasp of the global distribution.

[0160]

[0161] F1 score

[0162] The F1 score is a harmonic average of Precision and Recall, and is suitable for overall performance evaluation when there is a severe class imbalance.

[0163]

[0164] These metrics evaluate the model's performance from different perspectives, ensuring the comprehensiveness and reliability of the results.

[0165] Comparison with other state-of-the-art methods

[0166] Tables 1 and 2 present detailed comparisons of MSAFNet with other state-of-the-art methods on the Crack500 and CFD datasets, respectively.

[0167] On the Crack500 dataset, MSAFNet achieved state-of-the-art performance across all key metrics: precision of 98.291%, recall of 65.884%, accuracy of 98.291%, F1 score of 66.014%, and intersection-over-union (IoU) of 65.368%. In comparison, other methods such as UP-CrackNet achieved an IoU of only 44.726% and an F1 score of 61.808%, while MSAFNet improved these metrics by 20.642% and 4.206%, respectively, demonstrating a significant performance advantage.

[0168] In the evaluation of cross-domain generalization ability, experimental results on the CFD dataset further validated the superiority of MSAFNet. MSAFNet maintained a high level of performance on the unseen CFD dataset: Precision 74.579%, Recall 47.308%, Accuracy 98.890%, F1-Score 57.715%, and IoU 46.967%. Compared to UP-CrackNet, MSAFNet improved the IoU metric by 13.715% and the F1-Score by 7.806%. These results fully demonstrate that MSAFNet not only performs exceptionally well on the Crack500 dataset but also possesses excellent cross-domain generalization ability, enabling it to stably and efficiently complete road crack segmentation tasks in different scenarios.

[0169] Table 1 shows the test results on the Crack500 dataset.

[0170]

[0171] Table 2 shows the test results on the CFD dataset (trained only on Crack500).

[0172]

[0173]

[0174] In summary, we propose a novel crack segmentation network, MSAFNet, which is a cross-layer fusion network based on Transformer enhancement. The core innovation of this network lies in MSAF (Multi-Scale Attention Fusion), a self-designed multi-scale cross-layer module designed to improve the accuracy and robustness of crack segmentation by fusing local and global features. The MSAF module utilizes a multi-head attention mechanism based on 2D sinusoidal positional encoding to explicitly fuse the global semantics of the entire image with 2×2 local block features without requiring additional parameters. Specifically, the MSAF module achieves this by dividing the input image into 2×2 local blocks. Each block extracts features through an independent encoder, then interacts with global features through a multi-head attention mechanism. Finally, the fused features are recombined to form a complete feature map. This design not only preserves the edge details of cracks but also captures long-range contextual information, significantly improving the model's crack detection capability. Experimental results show that MSAFNet achieves an IoU of 65.37% and an F1 score of 66.01% on the Crack500 dataset, which is 25.5% higher than the unsupervised method UP-CrackNet. It also achieves an IoU of 46.97% on the cross-domain CFD dataset, which is significantly better than existing methods, verifying its cross-scenario robustness and practical deployment potential.

[0175] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for constructing a road crack segmentation model based on a Transformer enhancement, characterized in that: At least comprising the following steps: A MSAFNet framework is built, which is a global-local dual-stream encoding architecture, which extends the U-Net architecture by introducing a dual-stream encoder that can handle global and local features; The global stream encodes the entire input image to capture long-range contextual information, while the local stream processes 2x2 image blocks to preserve fine-grained crack details; A MSAF module is built, which integrates the global and local streams, and uses a multi-head attention mechanism with 2D sinusoidal position encoding to align and integrate multi-scale features; A decoder is built, which passes the integrated features to the decoder to generate the final segmentation map; A Dice-Focal dynamic weighted loss function is introduced to alleviate the labeling noise and class imbalance problem, and the input image is processed to output a binary segmentation map, where H and W represent height and width respectively.

2. The method of claim 1, wherein the method is characterized by: The dual-stream encoder consists of two parallel branches, namely the global encoder and the local encoder; Both branches use a series of double convolution modules, each module containing two 3x3 convolutions with batch normalization and ReLU activation function, defined as follows: DoubleConv(X)=ReLU(BN(Conv(ReLU(Conc(X))))) (1) Where BN is batch normalization and ReLU is the activation function; The global encoder processes the input image X∈R3×H×W through four double convolution modules with channel numbers of 64, 128, 256, and 512, and intersperses 2x2 max pooling between modules; The output of the i-th global encoder is: wherein Ei is the feature map of the i-th global encoder, is the height and width after i times max-pooling, C i is the number of channels of the i-th layer; deepest features down to 256 channels by 1x1 convolution, generate global feature maps for MSAF fusion: The local encoder divides the input image X into non-overlapping 2x2 blocks, reshaped as: Where B is the batch size; These blocks are processed through three double convolution modules with channel numbers of 64, 128, and 256, and interspersed with max pooling, generating: where E 3,loc is the local encoder feature map after three modules, which preserves more crack details.

3. The method of claim 2, wherein the method is characterized by: The MSAF module integrates global and local features by using a multi-head attention mechanism with 2D sinusoidal position encoding; wherein, and denote local and global features, respectively; The operation process of the MSAF module is as follows: To incorporate spatial information, 2D sinusoidal position encoding is applied to L and G. For a feature map of size B x H x W, the position encoding is defined as: Where x and y are normalized spatial coordinates, θ=10000, C=256 is the channel dimension; The encoding features include: L=L+PE(B·4,H / 8,W / 8),G=G+PE(B,H / 8,W / 8)(8) Reshape the local feature L to Bx4x256xH / 8xW / 8, and flatten each patch L[:,i,:,:,:](i=0,1,2,3) to (H / 8·W / 8)x Bx256; Flatten the global feature G to (H / 8·W / 8)x Bx256; Four multi-head attention modules process each patch respectively: O i = MultiHeadAttention(L[:, i : : :, : ], G, G) i = 0, 1, 2, 3 (9) wherein concatenating the outputs along the sequence dimension, resulting in The concatenated output is sequentially passed through layer normalization and two layers of feedforward neural networks: O=LayerNorm(O+Dropout(ReLU(O))))(10) The final output is reshaped as and recombined into for use by the decoder.

4. The method of claim 1, wherein the method is characterized by: The decoder upsamples the bottleneck feature E4 by transposed convolution and concatenates it with the skip connection and fused feature E3, fused of the global encoder; Each decoder stage applies a double convolution block: D i = DoubleConv(Concat(Up(D i+1 ), E i )), i = 3, 2, 1, 0 (11) where Up is a 2x2 transposed convolution and D4 = DoubleConv(Concat(Up(E4), E3, fused)); The final output is obtained by a 1x1 convolution: Y = Conv 1×1 (D1) (12).

5. The method of claim 1, wherein the method is characterized by: The Dice-Focal dynamic weighted loss function combines Dice loss and Focal loss; The Dice loss ensures the overlap between the prediction and the ground truth, and the definition of the Dice loss is: where, is the predicted value after sigmoid activation, y is the true label, and ε = 10 -6 is a factor for smoothing, i and j represent the spatial location indices of the image; The Focal loss deals with the label noise by focusing on the hard samples, and its expression is: wherein, when y i,j = 1, otherwise a = 0.75 represents a weight factor for positive samples, g = 2 emphasizes a focusing parameter for difficult samples, and BCE stands for binary cross-entropy loss. The total loss dynamically balances the two, and is expressed as: L = L Dice + 0.5 · L Focal (15) The Dice-Focal dynamic weighted loss function alleviates the false negative problem caused by noisy labels and solves the challenge of low crack pixel proportion, improving the robustness of the model.

6. The road crack segmentation model based on Transformer enhancement, characterized in that: The road crack segmentation model based on the Transformer enhancement is constructed by the construction method of any one of claims 1-5.

7. The use of a road crack segmentation model based on Transformer enhancement according to claim 6, characterized in that: At least comprising the following steps: S1: input preprocessing, input image After normalization processing, the mean is [0.485, 0.456, 0.406], the standard deviation is [0.229, 0.224, 0.225], and it is divided into 2x2 non-overlapping blocks for local processing, while the whole image is reserved for global processing; S2: Bi-flow encoding, global flow applies convolutional block to capture long-range context information, generates feature maps (E1, E2, E3, E4), local flow processes 2x2 block to preserve fine-grained crack details, generates E 3,loc ; S3: Multi-scale attention fusion, the MSAF module uses multi-head attention mechanism and two-dimensional sinusoidal position encoding to fuse the local features E 3,loc with the global features E4 after channel dimension reduction, to generate the fused features E 3,fused ; S4: Decode, the decoder up-samples E4 and concatenates it with E 3,fused and global encoder features to generate a segmentation map S5: loss function, using Dice-Focal dynamic weighted loss function to optimize the model, focusing on hard samples and balancing crack and background pixels, reducing label noise and class imbalance problems.

Images