Crack image segmentation method based on bidirectional structure sensing cascade network

Abstract

The invention discloses a crack image segmentation method based on a bidirectional structure sensing cascade network. The crack image segmentation method comprises the steps of performing preprocessing based on a collected crack image; performing block processing and position coding on a crack image in the preprocessed data to obtain coded image data; obtaining a BiScaNet network model based on a mixed objective function combining binary cross entropy loss and Dice loss; after feature extraction is carried out on the coded image data, fusion and decoding are carried out on features of different scales, and a pixel-level segmentation image is generated; and carrying out an ablation experiment and outputting evaluation indexes, and carrying out visual comparison on the pixel-level segmentation image and segmentation images of other models. According to the invention, the scanning strategy in the Mama architecture is improved, and the BiSS module is provided, so that the modeling capability of crack multi-direction extension and complex topological structures is remarkably enhanced.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a crack image segmentation method based on a bidirectional structured perceptual cascaded network. Background Technology

[0002] Under the combined effects of complex environments and long-term loads, engineering materials such as asphalt pavements, concrete, and metals generally face the generation, accumulation, and escalation of various defects during prolonged use. This seriously threatens the safety of infrastructure and significantly shortens its service life. Cracks, as the most direct and typical visual representation of infrastructure performance degradation, have become a key evaluation indicator in infrastructure health monitoring systems. Achieving accurate and robust identification and quantitative analysis of cracks is of great significance for ensuring the safe operation of infrastructure and guiding scientific maintenance.

[0003] Currently, deep learning has been widely applied in image segmentation and crack detection tasks, and its powerful deep feature extraction capabilities have significantly improved model performance. Convolutional Neural Network (CNN)-based methods can effectively extract local texture features, but due to the limited local receptive field of convolution operations, it is difficult to model the long-distance spatial dependencies between crack regions and the background. This limitation often leads to segmentation results with issues such as fragmentation and jagged edges, and model performance degrades under complex background interference. Because crack structures in images often extend in multiple directions and exist in a distributed manner, CNN methods face significant challenges in effectively capturing their global morphological characteristics. To overcome the limitations of CNNs in capturing long-term dependencies within images, the Transformer-based Visual Model (ViT) can utilize its global self-attention mechanism to process image data, exhibiting significant advantages in crack segmentation tasks. This model effectively captures long-range spatial dependencies using a self-attention mechanism, improving its ability to represent widely distributed and complex crack structures. However, the computational and spatial complexity of the self-attention mechanism increases quadratically with the length of the input sequence, leading to a sharp increase in memory usage and computational latency when processing high-resolution engineering images. This makes it difficult to achieve real-time and efficient crack detection on embedded devices with limited computing resources. In contrast, the Mamba architecture based on the State-Space Model (SSM) has surpassed Transformer to become the mainstream architecture for global modeling of long sequences. Mamba effectively captures long-distance dependencies by introducing a variable parameter mechanism and a selective scanning strategy, leveraging its linear computational cost advantage. Although it shows an advantage in linear computational complexity, its unidirectional scanning mechanism is difficult to adapt to the multi-directional extension characteristics of cracks, performs poorly in modeling complex topologies such as intersecting and mesh cracks, and lacks an effective morphological feature enhancement module.

[0004] In summary, existing methods still have significant shortcomings in terms of global context modeling, computational efficiency, and adaptability to complex crack morphologies. To overcome these limitations, this application proposes a crack image segmentation method based on the bidirectional structure-aware cascaded network BiScaNet. Summary of the Invention

[0005] This application provides a crack image segmentation method based on a bidirectional structure-aware cascaded network to address the technical problems that existing methods still have significant shortcomings in terms of global context modeling, computational efficiency, and adaptability to complex crack morphologies.

[0006] To address the aforementioned technical problems, this application adopts the following technical solution: a crack image segmentation method based on a bidirectional structure-aware cascaded network, comprising:

[0007] S1. Preprocess the collected crack images to obtain preprocessed data;

[0008] S2. Perform block processing and position encoding on the crack images in the preprocessed data to obtain encoded image data;

[0009] S3. Obtain the BiScaNet network model based on a hybrid objective function that combines binary cross-entropy loss and Dice loss;

[0010] S4. Based on the BiScaNet network model, the encoded image data is processed by the BiSVSS module for feature extraction, and then the MFF module is used to fuse and decode features at different scales to generate a pixel-level segmentation map.

[0011] S5. Based on the BiSS module in the BiSVSS module, conduct ablation experiments and output evaluation indicators, and visualize and compare the pixel-level segmentation map with other models.

[0012] Furthermore, the specific method of step S2 includes:

[0013] S21. Extract the crack image X∈R from the preprocessed data. C×H×W The image is divided into n blocks by a convolutional block; where C is the number of channels, H is the height of the image, and W is the width of the image.

[0014] S22. Based on the linear projection layer, each image patch is mapped to a feature vector with a fixed dimension;

[0015] S23. Based on the feature vector, add learnable location embedding information to preserve the spatial positional relationship of each image patch in the original image and obtain encoded image data.

[0016] Furthermore, the method in step S3 includes:

[0017] Based on formula (1), the BCE loss function for a single pixel is obtained; where formula (1) is:

[0018] L BCE =-{y·log(p)+(1-y)·log(1-p)} (1);

[0019] Where y is the true label value, p is the probability value predicted by the model; the BCE loss of the entire image is the average of the losses of all pixels.

[0020] Based on formulas (2)-(3), the Dice loss function is obtained; where formulas (2)-(3) are:

[0021]

[0022] L Dice =1-Dice (3);

[0023] Among them, L Dice Let X be the Dice loss function, where X represents the set of pixels predicted as "positive" by the model; Y represents the set of pixels in the true labels as "positive"; |X∩Y| is the number of "true positives" TP, i.e., the positive pixels correctly predicted by the model; |X| is the total number of positive pixels predicted by the model TP+FP; |Y| is the total number of true positive pixels TP+FN.

[0024] Based on formula (4), a hybrid objective function combining BCE and Dice is obtained; where formula (4) is:

[0025] L=α·L Dice +β·L BCE (4);

[0026] Here, α and β are hyperparameters that control the weights of the two loss components.

[0027] Furthermore, the method in step S4 includes:

[0028] S41. Input the encoded image data into four cascaded BiSVSS modules for sampling and feature extraction to obtain feature maps F1-F4;

[0029] S42. Input feature maps F1-F4 into the MFF module, and after feature transformation by the multilayer perceptron, obtain pixel-level segmentation maps.

[0030] Furthermore, the method for feature extraction by the BiSVSS module in step S41 includes:

[0031] S411. Based on a lightweight gated bottleneck convolution module, detailed features of the encoded image data are extracted and normalized by layers to obtain normalized feature data; wherein, the normalized feature data is processed in two branches.

[0032] S412. The left branch expands the channel dimension through a linear layer, and after group normalization and SiLU activation function processing, the original features are preserved to obtain the left branch output;

[0033] S413. The right branch undergoes the same linear transformation, group normalization, and SiLU activation function processing before entering the BiSS module to model the global morphological structure of the crack along multiple directions and obtain the right branch output.

[0034] S414. The outputs of the left and right branches are multiplied and fused to obtain the fused feature output;

[0035] S415. Based on the fused feature output, the original number of channels is restored by linear projection, and the initial information flow is preserved through residual connection to obtain the feature map.

[0036] Furthermore, the method for modeling the global morphological structure of cracks along multiple directions in step S413 of the BiSS module includes:

[0037] S4131. Perform forward and reverse serpentine scanning on the two-dimensional feature map in the right branch along the horizontal, vertical, main diagonal, and secondary diagonal directions, expand it into a sequence, and generate a total of 8 scanning paths, that is, each direction contains a forward sequence and a reverse sequence;

[0038] S4132. Input the forward and reverse sequences of each direction into the BiGF module for fusion to obtain 4 fused sequences;

[0039] S4133. Subsequently, the four fused sequences are fed into separate S6 blocks for processing. The S6 block performs selective scanning along the sequence direction, dynamically adjusts the weights, and obtains four output sequences. The S6 block is a selective state space module in Mamba.

[0040] S4134. Reshape and merge the four output sequences back into the original two-dimensional spatial structure.

[0041] Furthermore, the method in step S4132 includes:

[0042] The input right branch undergoes the same linear transformation, group normalization, and SiLU activation function to obtain feature z. Then, feature z is recursively used to obtain y. fwd and y bwd , where y fwd For a forward sequence, y bwd It is a reverse sequence.

[0043] Based on formula (5), the original feature z is compressed to the [0,1] interval by the Sigmoid activation function to form the gate signal;

[0044] gate=σ(Linear(z)) (5);

[0045] Where σ is the Sigmoid function and Linear is the linear layer;

[0046] Based on formula (6), the positive and negative features are weighted and summed to obtain y;

[0047]

[0048] in, This indicates element-wise multiplication; if the gate value is close to 1, the output will be more biased towards y. fwd If the gate value is close to 0, the output will be more inclined towards y. bwd If the gate value is close to 0.5, the output will be more biased towards y. fwd and y bwd The average value;

[0049] The forward and reverse sequences generated in the four directions (horizontal, vertical, main diagonal, and secondary diagonal) are weighted and summed to obtain y1-y4. These sequences are then merged through a scan operation to obtain the fused sequence y. sum .

[0050] The beneficial effects of this application are: This invention improves the scanning strategy in the Mamba architecture, proposes the BiSS module, and significantly enhances the modeling ability for multi-directional crack extension and complex topology; This application introduces the BiGF module into the BiSS module to achieve adaptive fusion of features of different scanning paths, thereby improving the discriminativeness and robustness of feature representation. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating an embodiment of the crack image segmentation method based on a bidirectional structure-sensing cascaded network of this application.

[0052] Figure 2 yes Figure 1 A flowchart illustrating step S2 of an embodiment;

[0053] Figure 3 yes Figure 1 A flowchart illustrating step S4 in one embodiment;

[0054] Figure 4 This is a network architecture diagram of BiScaNet, an embodiment of the crack image segmentation method based on a bidirectional structure-aware cascaded network, according to this application.

[0055] Figure 5 This is a schematic diagram of the BiSVSS module of an embodiment of the crack image segmentation method based on a bidirectional structure-sensing cascaded network of this application;

[0056] Figure 6 This is a schematic diagram of the scanning strategy of an embodiment of the crack image segmentation method based on a bidirectional structure-sensing cascaded network of this application;

[0057] Figure 7 This is a schematic diagram of the BiSS module of an embodiment of the crack image segmentation method based on a bidirectional structure-sensing cascaded network of this application;

[0058] Figure 8 This is a schematic diagram of the BiGF module in an embodiment of the crack image segmentation method based on a bidirectional structure-sensing cascaded network of this application.

[0059] Figure 9 This is a comparison diagram of the BiSS scanning strategy of an embodiment of the crack image segmentation method based on a bidirectional structure-sensing cascaded network of this application;

[0060] Figure 10 This is a visual comparison chart of the model in the embodiments of this application with other segmentation models. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0062] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.

[0063] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the crack image segmentation method based on a bidirectional structure-aware cascaded network according to this application. The method includes:

[0064] S1. Based on the collected crack images, preprocessing is performed to obtain preprocessed data; wherein, the preprocessed data is divided into training set, validation set and test set.

[0065] Specifically, the TUT dataset used in this invention contains 1408 RGB crack images, of which 1270 were taken with a mobile phone, and the remaining 138 were supplemented from the internet. This dataset covers a variety of typical scenarios, including 900 images of asphalt and cement with irregular crack patterns but flat backgrounds, 142 images of bricks and tiles with regular crack patterns, 228 images of plastic runways with extremely complex cracks, and 138 images of metal, generator blades, and underground pipes with complex lighting conditions and background interference. All images are displayed at a uniform resolution of 640×640, with crack pixels accounting for 3.16%, a moderate level. This ensures that the model learns useful information without being hindered by too few crack pixels, and prevents the model from over-relying on the crack pixel ratio when there are too many crack pixels.

[0066] In the preprocessing stage, all images are first uniformly adjusted to 512×512 pixels. Then, the dataset is divided into a training set of 985 images, a validation set of 141 images, and a test set of 282 images in a ratio of 7:1:2. This ratio ensures the scientific rigor and reliability of model training and performance evaluation.

[0067] S2. Perform block processing and position encoding on the crack image in the preprocessed data to obtain encoded image data.

[0068] For details, please refer to Figure 2 The method of step S2 includes:

[0069] S21. Extract the crack image X∈R from the preprocessed data. C×H×W The image is divided into n blocks by a convolutional block; where C is the number of channels, H is the height of the image, and W is the width of the image.

[0070] Specifically, convolutional blocks are used to divide the input RGB crack image into blocks. First, the image is divided into n local image blocks through convolution operation.

[0071] S22. Based on the linear projection layer, each image patch is mapped to a feature vector with a fixed dimension.

[0072] Specifically, each image patch is mapped to a feature vector with fixed dimensions through a learnable linear projection layer.

[0073] S23. Based on the feature vector, add learnable location embedding information to preserve the spatial positional relationship of each image patch in the original image and obtain encoded image data.

[0074] Specifically, based on the aforementioned feature vectors, learnable location embedding information is added to preserve the spatial relationship of each image patch in the original image, forming a sequence {B1, B2, ..., B...}. n}, which means encoding image data.

[0075] The core function of convolutional blocks is to simultaneously perform two tasks: image segmentation and local feature extraction through convolution operations, which has higher computational efficiency compared to traditional explicit segmentation methods.

[0076] S3. Obtain the BiScaNet network model based on a hybrid objective function that combines binary cross-entropy loss (BCE) and Dice loss.

[0077] Specifically, the BCE function calculates the error between the predicted value and the true label pixel by pixel, enabling the model to learn the details of crack boundaries more accurately. The Dice loss function calculates the overlap between the predicted and true regions, guiding the model to focus on the overall shape and structural integrity of the crack region, effectively mitigating the class imbalance problem caused by the large proportion of background pixels. These two functions complement each other and work synergistically to significantly improve the segmentation accuracy and robustness for small, discontinuous cracks while maintaining model training stability.

[0078] The formula for calculating the BCE loss of a single pixel is as follows:

[0079] L BCE =-{y·log(p)+(1-y)·log(1-p)} (1);

[0080] Where y is the true label value, p is the probability value predicted by the model; the BCE loss of the entire image is the average of the losses of all pixels.

[0081] Dice Loss originates from the Dice coefficient, a metric used to measure the similarity between two sets, commonly employed in image segmentation tasks. Its value ranges from 0 to 1, with higher values ​​indicating greater similarity. Dice Loss is calculated by subtracting the Dice coefficient from 1, meaning that the closer the predicted result is to the true label, the smaller the loss.

[0082] The formula for calculating Dice is:

[0083]

[0084] L Dice =1-Dice (3);

[0085] Among them, L DiceLet X be the Dice loss function, where X represents the set of pixels predicted as "positive" by the model; Y represents the set of pixels in the true labels as "positive"; |X∩Y| is the number of "true positives" TP, i.e., the positive pixels correctly predicted by the model; |X| is the total number of positive pixels predicted by the model TP+FP; |Y| is the total number of true positive pixels TP+FN.

[0086] The hybrid objective function combining BCE and Dice is as follows:

[0087] L=α·L Dice +β·L BCE (4);

[0088] Here, α and β are hyperparameters that control the weights of the two loss components.

[0089] S4. Based on the BiScaNet network model, the encoded image data is processed by the Bidirectional Structured Visual State Space (BiSVSS) module for feature extraction, and then the multi-scale feature fusion (MFF) module is used to fuse and decode features at different scales to generate a pixel-level segmentation map.

[0090] For details, please refer to Figure 3-4 The method of step S4 includes:

[0091] S41. Input the encoded image data into four cascaded BiSVSS modules for sampling and feature extraction to obtain feature maps F1-F4.

[0092] Specifically, the preprocessed RGB crack image is divided into blocks and then positionally encoded to obtain encoded image blocks. These encoded image blocks are then input into four cascaded BiSVSS modules for depth processing. The original feature image (C, H, W) is input into the four cascaded BiSVSS modules and downsampled.

[0093] Each block and downsampling produces feature outputs at different scales: the shape of the first BiSVSS module and the downsampled output feature map F1 is (C, H, W); the shape of the second BiSVSS module and the downsampled output feature map F2 is (2C, H / 2, W / 2); the shape of the third BiSVSS module and the downsampled output feature map F3 is (4C, H / 4, W / 4); and the shape of the fourth BiSVSS module and the downsampled output feature map F4 is (8C, H / 8, W / 8). Here, C is the number of channels, H is the image height, and W is the image width.

[0094] Referring to 5, the method for feature extraction using the BiSVSS module in step S41 includes:

[0095] S411. Based on a lightweight gated bottleneck convolution module, detailed features of the encoded image data are extracted and normalized by layers to obtain normalized feature data; the normalized feature data is processed in two branches.

[0096] S412. The left branch expands the channel dimension through a linear layer, and after group normalization and SiLU activation function processing, the original features are preserved to obtain the left branch output.

[0097] S413. The right branch undergoes the same linear transformation, group normalization, and SiLU activation function processing before entering the Bidirectional Structural Scan (BiSS) module to model the global morphological structure of the crack along multiple directions and obtain the right branch output.

[0098] For details, please refer to Figure 6-7 The method for modeling the global morphological structure of cracks along multiple directions using the BiSS module in step S413 includes:

[0099] S4131. Perform forward and reverse serpentine scans on the two-dimensional feature map in the right branch along the horizontal, vertical, main diagonal, and secondary diagonal directions to expand it into a sequence, generating a total of 8 scan paths, that is, each direction contains a forward sequence and a reverse sequence.

[0100] Specifically, the two-dimensional feature map is scanned in a forward serpentine pattern along the horizontal, vertical, main diagonal, and secondary diagonal directions, and then expanded into a sequence, generating a total of 8 scanning paths, i.e., one forward and one reverse sequence in each direction.

[0101] S4132. Input the forward and reverse sequences of each direction into the BiGF module for fusion to obtain 4 fused sequences.

[0102] Specifically, the BiGF module, such as Figure 8 As shown, the specific steps for fusing the forward and reverse sequences in four directions are as follows:

[0103] The input right branch undergoes the same linear transformation, group normalization, and SiLU activation function to obtain feature z. Then, feature z is recursively used to obtain y. fwd and y bwd , where y fwd For a forward sequence, y bwd It is a reverse sequence.

[0104] The original feature z is compressed to the [0,1] interval by using the Sigmoid activation function to form the gate signal;

[0105] gate=σ(Linear(z)) (5);

[0106] Where σ is the Sigmoid function and Linear is the linear layer;

[0107] Then, using the generated gating signal, the forward and reverse features are weighted and summed to obtain y;

[0108]

[0109] in, This indicates element-wise multiplication; if the gate value is close to 1, the output will be more biased towards y. fwd If the gate value is close to 0, the output will be more inclined towards y. bwd If the gate value is close to 0.5, the output will be more biased towards y. fwd and y bwd The average value;

[0110] The forward and reverse sequences generated in the four directions (horizontal, vertical, main diagonal, and secondary diagonal) are weighted and summed to obtain y1-y4. These sequences are then merged through a scan operation to obtain the fused sequence y. sum .

[0111] S4133. Subsequently, the four fused sequences are sent to separate S6 blocks for processing. The S6 block performs selective scanning along the sequence direction to capture long-range dependencies and dynamically adjust weights to obtain four output sequences. The S6 block is a selective state space module in Mamba.

[0112] S4134. Reshape and merge the four output sequences back into the original two-dimensional spatial structure.

[0113] S414. The outputs of the left and right branches are interacted through product fusion to obtain the fused feature output.

[0114] Specifically, the original features processed from the left branch output and the features obtained by mixing the bidirectional structural scan from the right branch output are multiplied and fused. That is, the features obtained by mixing the bidirectional structural scan from the left and right branches are multiplied element-wise to obtain the fused feature output.

[0115] S415. Based on the fusion feature output, the original number of channels is restored by linear projection, and the initial information flow is preserved through residual connection to obtain feature extraction data.

[0116] Specifically, the fused feature output after product fusion is restored to the original number of channels by linear projection, and the initial information flow is preserved by residual connection to obtain feature maps F1-F4.

[0117] S42. Input feature maps F1-F4 into the MFF module, and after feature transformation by the multilayer perceptron, obtain pixel-level segmentation maps.

[0118] Specifically, the BiSVSS module and the downsampled output feature maps F1, F2, F3, and F4 are input into the MFF module. After feature transformation by a multilayer perceptron, dynamic upsampling is used to gradually restore the spatial resolution of the feature maps to the original image size. All upsampled multi-scale feature maps are concatenated and fused along the channel dimension, and the final element-level segmentation map is obtained through 1×1 convolution and a multilayer perceptron.

[0119] S5. Based on the BiSS module in the BiSVSS module, conduct ablation experiments and output evaluation indicators, and visualize and compare the pixel-level segmentation map with other models.

[0120] The specific steps are as follows:

[0121] The model in this application was trained using PyTorch 2.0.1 on an Intel(R) Xeon(R) CPU E5-2680v4@2.40GHz and an NVIDIA RTX 3090 GPU. The AdamW optimizer was used during training, with an initial learning rate of 5e-4, weight decay of 0.01, and a PolyLR learning rate scheduling strategy. The random seed was fixed at 42. The network was trained for 50 epochs, and the model that performed best on the validation set was selected for testing.

[0122] The model has seven evaluation indicators: ODS, OIS, The formulas for calculating Recall(R), F1, H_95, and mIoU are as follows:

[0123] ODS uses a fixed threshold m to measure the model's adaptability to datasets of different scales, and the calculation formula is as follows:

[0124]

[0125] OIS measures the model's adaptability across image scales using the optimal threshold n, calculated as follows:

[0126]

[0127] The formulas for calculating Precision, Recall, and F1 are as follows:

[0128]

[0129] The formula for calculating H_95 is as follows:

[0130]

[0131] Where A is the prediction boundary, B is the truth boundary, and q 0.95It is the 95th percentile of experience.

[0132] The formula for calculating mIoU is as follows:

[0133]

[0134] Where k is the number of classes, here we set k=1, i represents the predicted value, j represents the actual value, and p ji This represents the number of pixels that belong to category i but are also in category j.

[0135] To verify the effectiveness of the multi-directional scanning strategy in the BiSS module proposed in this application, a systematic ablation experiment was conducted. The performance evaluation metrics of different scanning strategies on the standard test set are shown in Table 1 below:

[0136] Table 1 shows the performance evaluation metrics of different scanning strategies on the standard test set.

[0137]

[0138] The experiments in this application first evaluated the performance of a single scanning path: using only parallel serpentine scanning (horizontal and vertical directions) or only diagonal serpentine scanning (main and secondary diagonal directions). The results showed that diagonal serpentine scanning slightly outperformed parallel serpentine scanning on most metrics, indicating that diagonal serpentine scanning has a better ability to capture complex boundaries and directional features. Combining the two scanning paths, i.e., parallel serpentine + diagonal serpentine, significantly improved all metrics, especially mIoU, which improved by 2% compared to using only parallel serpentine, demonstrating the effectiveness of multi-directional information complementarity. Furthermore, based on the fusion of the two scanning paths, a bidirectional scanning mechanism was introduced, i.e., scanning in both directions, constructing a richer flow of contextual information. This strategy further improved ODS, OIS, P, F1 score, and mIoU, while significantly reducing H_95, indicating enhanced model recovery of detailed structures and reduced prediction error.

[0139] Based on bidirectional scanning, the proposed BiGF module is introduced to adaptively integrate sequence information from different directions. Experimental results show that it achieves optimal performance across all evaluation metrics: compared to bidirectional scanning without BiGF, ODS is significantly improved by 1.21%, and OIS is improved by 0.78%.

[0140] The performance improved by 1.29%, R by 1.06%, F1 score by 1.18%, and mIoU by 0.56%, while H_95 decreased by 1.34 units. Compared to the initial single parallel snake scan baseline, the final model showed a 2.46% improvement in ODS, a 3.56% improvement in mIoU, and a significant 9.14-unit reduction in H_95. This demonstrates that the BiGF module effectively balances the feature contributions of different scan paths, enhances the transmission of useful information, and suppresses noise, thereby comprehensively improving the accuracy and continuity of segmentation boundaries.

[0141] Experimental results clearly demonstrate that the proposed multi-directional scanning strategy and bidirectional gating fusion mechanism both play a crucial role in improving segmentation performance, verifying the rationality and effectiveness of the BiSS module design.

[0142] like Figure 9 The figure shows a comparison of BiSS scanning strategies, illustrating the effective receptive field (ERF) results of different scanning strategies. Figure 9 (a) Two-dimensional scanning can capture long-range dependencies in both the horizontal and vertical directions; Figure 9 (b) diagonal scanning excels at capturing global information along diagonal paths; Figure 9 (c) Hilbert scans are insufficient in capturing global information. These methods can only capture specific spatial information. For example... Figure 9 (d) shows the ERF results of the BiSS scanning strategy. This method generates eight different scanning paths by integrating forward and reverse serpentine scans in four directions: horizontal, vertical, main diagonal, and secondary diagonal. Compared with traditional single-scanning strategies, BiSS scanning can simultaneously capture spatial dependencies in multiple directions, effectively combining the advantages of different scanning paths. Its receptive field presents more uniform and comprehensive spatial coverage, preserving the long-range dependencies in the longitudinal and transverse directions of two-dimensional scanning while incorporating the oblique global information capture capability of diagonal scanning. Furthermore, the complementarity of forward and reverse scanning eliminates the directional bias that may exist in traditional scanning methods. This multi-path fusion mechanism significantly enhances the model's ability to represent the global structure and local details of the image, providing richer spatial contextual information for image restoration tasks.

[0143] like Figure 10The image shows a visual comparison between the model of this invention and other segmentation models. The red boxes highlight easily missed crack pixel areas, the yellow boxes represent falsely detected non-crack areas, and the ground truth (GT) is the labeled image. The comparison results show that all models can basically identify the main crack structure. However, the method proposed in this invention (Ours) outperforms the comparison methods (HED, Unet++, DTrCNet, Crackmer, SimCrack) in both accuracy and completeness of crack segmentation, especially in preserving the fine crack structure and suppressing false background detections. For example, in the red box area, the DTrCNet and Crackmer models both missed crack pixel segments. Compared to other models, the method of this invention can better restore the continuity and structural details of the crack, avoiding breakage and missed detections. In the yellow box area, the HED and Unet++ models both falsely detected non-crack areas, while the method of this application effectively suppressed false detections in non-crack areas, significantly reducing the generation of false cracks. In contrast, other methods may exhibit breakage, loss of detail, or incorrect detection of cracks in smooth areas, indicating that the method of this invention has stronger generalization ability and robustness in complex scenarios.

[0144] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A crack image segmentation method based on a bidirectional structure-sensing cascaded network, characterized in that, include: S1. Preprocess the collected crack images to obtain preprocessed data; S2. Perform block processing and position encoding on the crack image in the preprocessed data to obtain encoded image data; S3. Obtain the BiScaNet network model based on a hybrid objective function that combines binary cross-entropy loss and Dice loss; S4. Based on the BiScaNet network model, the encoded image data is processed by the BiSVSS module for feature extraction, and then the MFF module is used to fuse and decode features at different scales to generate a pixel-level segmentation map. S5. Based on the BiSS module in the BiSVSS module, conduct ablation experiments and output evaluation indicators, and visually compare the pixel-level segmentation map with the segmentation maps of other models.

2. The method according to claim 1, characterized in that, The specific method of step S2 includes: S21. The crack image X∈R in the preprocessed data... C×H×W The image is divided into n blocks by a convolutional block; where C is the number of channels, H is the height of the image, and W is the width of the image. S22. Based on the linear projection layer, each image patch is mapped to a feature vector with a fixed dimension; S23. Based on the feature vector, add learnable position embedding information to preserve the spatial positional relationship of each image block in the original image and obtain encoded image data.

3. The method according to claim 1, characterized in that, The method of step S3 includes: Based on formula (1), the BCE loss function for a single pixel is obtained; wherein, formula (1) is: L BCE =-(y·log(p)+(1-y)·log(1-p)} (1); Where y is the true label value, p is the probability value predicted by the model; the BCE loss of the entire image is the average of the losses of all pixels. Based on formulas (2)-(3), the Dice loss function is obtained; wherein, formulas (2)-(3) are: THE Dice =1-Dice (3); Among them, L Dice Let X be the Dice loss function, where X represents the set of pixels predicted as "positive" in the model; Y represents the set of pixels in the true labels as "positive"; |X∩Y| is the number of "true positives" TP, i.e., the positive pixels correctly predicted by the model; |X| is the total number of positive pixels predicted by the model TP+FP; |Y| is the total number of true positive pixels TP+FN. Based on formula (4), a hybrid objective function combining BCE and Dice is obtained; wherein, formula (4) is: L=α·L Dice +β·L BCE (4); Here, α and β are hyperparameters used to control the weights of the two loss components.

4. The method according to claim 1, characterized in that, The method of step S4 includes: S41. Input the encoded image data into four cascaded BiSVSS modules for sampling and feature extraction to obtain feature maps F1-F4; S42. Input the feature maps F1-F4 into the MFF module, and after feature transformation by the multilayer perceptron, obtain the pixel-level segmentation map.

5. The method according to claim 4, characterized in that, The method for feature extraction by the BiSVSS module in step S41 includes: S411. Based on a lightweight gated bottleneck convolution module, extract detailed features of the encoded image data and obtain normalized feature data through layer normalization; wherein, the normalized feature data is processed in two branches. S412. The left branch expands the channel dimension through a linear layer, and after group normalization and SiLU activation function processing, the original features are preserved to obtain the left branch output; S413. The right branch undergoes the same linear transformation, group normalization, and SiLU activation function processing before entering the BiSS module to model the global morphological structure of the crack along multiple directions and obtain the right branch output. S414. The left branch output and the right branch output are interacted through product fusion to obtain fused feature output; S415. Based on the fusion feature output, the original number of channels is recovered by linear projection, and the initial information flow is preserved through residual connection to obtain the feature map.

6. The method according to claim 5, characterized in that, The method for modeling the global morphological structure of cracks along multiple directions using the BiSS module in step S413 includes: S4131. Perform forward and reverse serpentine scanning on the two-dimensional feature map in the right branch along the horizontal, vertical, main diagonal, and secondary diagonal directions to expand it into a sequence, generating a total of 8 scanning paths, that is, each direction contains a forward sequence and a reverse sequence; S4132. Input the forward sequence and the reverse sequence in each direction into the BiGF module for fusion to obtain 4 fused sequences; S4133. Subsequently, the four fused sequences are sent to independent S6 blocks for processing. The S6 blocks selectively scan along the sequence direction, dynamically adjust the weights, and obtain four output sequences. The S6 block is a selective state space module in Mamba. S4134. Reshape and merge the four output sequences back into the original two-dimensional spatial structure.

7. The method according to claim 6, characterized in that, The method of step S4132 includes: The input right branch undergoes the same linear transformation, group normalization, and SiLU activation function to obtain feature z. Then, feature z is recursively used to obtain y. fwd and y bwd , where y fwd For a forward sequence, y bwd It is a reverse sequence. Based on formula (5), the original feature z is compressed to the [0,1] interval by the Sigmoid activation function to form the gate signal; gate=σ(Linear(z)) (5); Where σ is the Sigmoid function and Linear is the linear layer; Based on formula (6), the weighted sum of the positive and negative features is used to obtain y; in, This indicates element-wise multiplication; if the gate value is close to 1, the output will be more biased towards y. fwd If the gate value is close to 0, the output will be more inclined towards y. bwd If the gate value is close to 0.5, the output will be more biased towards y. fwd and y bwd The average value; The forward and reverse sequences generated in the four directions (horizontal, vertical, main diagonal, and secondary diagonal) are weighted and summed to obtain y1-y4. These sequences are then merged through a scan operation to obtain the fused sequence y. sum .

Images