A Real-Time Concrete Crack Segmentation Method and System Based on Dynamic State Space Reinforcement Network

Abstract

This invention discloses a real-time concrete crack segmentation method and system based on a dynamic state-space augmentation network, belonging to the field of image recognition technology. The method includes: acquiring concrete crack images; performing data augmentation on the concrete crack images to obtain a concrete crack dataset; training a dynamic state-space augmentation network using the concrete crack dataset; and segmenting cracks in the concrete images using the trained dynamic state-space augmentation network. This invention achieves advanced performance in both accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, specifically to a real-time concrete crack segmentation method and system based on a dynamic state space augmentation network. Background Technology

[0002] Cracks are a common defect in concrete structures, often exhibiting complex structural patterns such as irregular linear, network, or branching forms. These cracks not only impair the mechanical properties of infrastructure but can also cause further deterioration. Therefore, real-time crack segmentation is crucial for preventative maintenance. However, in real-world scenarios, uneven lighting, debris obstructions, and background interference hinder accurate crack segmentation. Furthermore, the limited memory and computational resources of mobile devices pose challenges to real-time segmentation.

[0003] To address the aforementioned challenges, traditional methods utilize crack morphology, contrast with background pixels, and spatial clustering algorithms to extract cracks. For example, Reference 1 (Landstrom, A.; Thurley, MJ. Morphology-based crack detection for steel slabs. IEEE Journal of selected topics in signal processing 2012, 6, 866–875.) developed an automatic crack detection system based on crack morphology and logistic regression statistical classification, successfully extracting 80% of the crack lengths. Reference 2 (Akagic, A.; Buza, E.; Omanovic, S.; Karabegovic, A. Pavement crack detection using Otsu thresholding for imagesegmentation. In Proceedings of the 2018 41st international convention on information and communication technology, electronics and microelectronics (MIPRO). IEEE, 2018, pp. 1092–1097.) proposed a pavement crack segmentation method based on grayscale histograms and Otsu's thresholding method. This method searches for cracks based on the ratio between the Otsu threshold and the maximum histogram value of the image, and can effectively extract various types of cracks from different pavement images. These traditional methods can effectively extract prominent cracks. However, for crack images with complex backgrounds, traditional methods struggle to accurately identify fine-grained cracks. Furthermore, the repetitive parameter tuning process can easily lead the algorithm into local optima.

[0004] With the emergence of deep learning technology, methods based on convolutional neural networks (CNNs) have alleviated the limitations of traditional algorithms in real-time crack segmentation and significantly improved accuracy. For example, Reference 3 (Xu, G.; Zhang, Y.; Yue, Q.; Liu, X. A deep learning framework for real-time multi-task recognition and measurement of concrete cracks. Advanced Engineering Informatics 2025, 65, 103127.) proposes an innovative deep learning framework, YOLO-DL, for detecting cracks in concrete. This method is based on DeepLabv3+ and introduces an attention mechanism and calibration module to achieve accurate crack segmentation. Reference 4 (Guo, L.; Xiong, F.; Cao, Y.; Xue, H.; Cui, L.; Han, X. Focusing on Cracks with InstanceNormalization Wavelet Layer. Sensors 2024, 25, 146.) incorporates the proposed instance normalization wavelet (INW) layer into the deep model for crack segmentation. This model utilizes prior knowledge from wavelets to capture crack features while filtering out high-frequency noise. Reference 5 (Liu, L.; Shen, B.; Huang, S.; Liu, R.; Liao, W.; Wang, B.; Diao, S. Binocular Video-Based Automatic Pixel-Level Crack Detection and Quantification Using Deep Convolutional Neural Networks for ConcreteStructures. Buildings 2025, 15, 258.) proposes a lightweight deep learning model based on stereo vision for real-time crack segmentation. This method effectively segments crack structures by integrating a semi-global block matching (SGBM) algorithm, laying the foundation for subsequent quantitative analysis of crack length, width, and azimuth.To improve the efficiency of CNNs, reference 6 (Howard, A.; Sandler, M.; Chu, G.; Chen, LC; Chen, B.; Tan, M.; Wang, W.; Zhu, Y.; Pang, R.; Vasudevan, V.; et al. Searching for mobilenetv3. In Proceedings of the Proceedings of the IEEE / CVF international conference on computer vision, 2019, pp. 1314–1324.) proposes MobileNetV3, which builds upon MobileNetV1 and MobileNetV2, introducing a squeeze-excitation module and neural architecture search to achieve efficient and accurate semantic segmentation. These CNN-based methods have significant advantages in extracting cracks with local variations. However, the limited receptive field of CNNs cannot accurately simulate the global dependencies of complex and long-span cracks, which are prone to breakage.

[0005] With the introduction of Vision Transformer (ViT) into the field of computer vision (CV), a series of Transformer-based crack segmentation models have been proposed. Reference 7 (Liu, H.; Miao, X.; Mertz, C.; Xu, C.; Kong, H. Crackformer: Transformer network for fine-grained crack detection. In Proceedings of the Proceedings of the IEEE / CVF international conference on computer vision, 2021, pp. 3783–3792.) utilizes the advantages of Transformers in global relation modeling to design a Crackformer for accurate crack detection. Reference 8 (Shan, J.; Huang, Y.; Jiang, W. DCUFormer: Enhancing pavement crack segmentation in complex environments with dual-cross / upsampling attention. Expert Systems with Applications 2025, 264, 125891.) designs a novel DCUFormer to address the bottlenecks of existing Transformer-based boundary delineation methods. By introducing a dual-cross attention module and an upsampling attention module, DCUFormer effectively integrates low-level and high-level features, enabling it to refine boundary pixels. To improve accuracy, a novel tunnel crack segmentation network, CGV-Net, was proposed in reference 9 (Liu, K.; Ren, T.; Lan, Z.; Yang, Y.; Liu, R.; Xu, Y. CGV-Net: Tunnel Lining Crack Segmentation Method Based on GraphConvolution Guided Transformer. Buildings 2025, 15, 197.). This method combines the advantages of CNN, Graph Neural Network (GNN), and ViT. By exchanging information among local features, CGV-Net effectively models the global structural patterns of cracks and achieves state-of-the-art performance. However, as a core component of transformer-based methods, the self-attention mechanism requires significant computational resources, posing a challenge to real-time segmentation tasks.

[0006] Existing crack segmentation algorithms can be divided into traditional methods and deep learning-based methods. Traditional methods mainly rely on morphological operations, edge detection, and adaptive thresholding to achieve crack segmentation. Although traditional methods can efficiently extract obvious cracks, they struggle to capture the key textures of cracks in complex backgrounds.

[0007] Deep learning-based crack segmentation methods are mainly divided into models based on convolutional neural network (CNN) architecture and models based on Transformer architecture. CNN-based methods are mainly variations and extensions of the fully convolutional network (FCN) framework.

[0008] These convolutional neural network (CNN)-based methods are more accurate in extracting local crack structures, but they may break down during the segmentation of long-span and slender cracks. The computational complexity of Transformers limits their deployment on real-time devices.

[0009] MobileNetV3 is widely used in various segmentation tasks due to its high segmentation performance. In the field of medical imaging, reference 10 (Alsenan, A.; Ben Youssef, B.; Alhichri, H. Mobileunetv3—a combined unetand mobilenetv3 architecture for spinal cord gray matter segmentation. Electronics 2022, 11, 2388.) combines the advantages of the U-Net architecture and MobileNetV3 to propose MobileUNetV3 for spinal cord gray matter segmentation. In agricultural engineering, Reference 11 (Jia, L.; Wang, T.; Chen, Y.; Zang, Y.; Li, X.; Shi, H.; Gao, L. MobileNet-CA-YOLO: An improved YOLOv7 based on the MobileNetV3 and attention mechanism for Rice pests and diseases detection. Agriculture 2023, 13, 1285.) uses MobileNetV3 to efficiently encode deep features and combines it with an attention mechanism to improve the accuracy of rice disease detection. For lane lines in traffic scenarios, Reference 12 (Deng, T.; Wu, Y. Simultaneous vehicle and lane detection via MobileNetV3 in carfollowing scene. PLOS ONE 2022, 17, e0264551.) uses MobileNetV3 to enhance the encoding capability and real-time performance of vehicle and lane line features. These MobileNetV3-based variants have achieved state-of-the-art (SOTA) performance in various downstream tasks. However, due to the inherent limitations of convolutional neural networks (CNNs), they have difficulty in modeling the global features of various cracks. Summary of the Invention

[0010] This invention addresses the shortcomings of existing technologies by providing a real-time concrete crack segmentation method and system based on a dynamic state space augmentation network. The dynamic state space augmentation network in this invention employs a U-shaped encoder-decoder architecture. For the encoder, a dynamic state space (DSS) module is introduced into the MobileNetV3 bottleneck module, and a state space-based MobileNetV3 bottleneck module is proposed to extract hierarchical features. For the decoder, upsampling operations and depthwise separable convolutions are primarily used to progressively recover and integrate features. Considering the complex backgrounds and uneven lighting in images captured by mobile devices, a strip pooling module is embedded between the encoder and decoder to highlight cracks and suppress background-related noise. Furthermore, cross-entropy loss and boundary loss are jointly used as objective functions to optimize the classification of crack pixels and the accuracy of boundary pixels.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] A real-time concrete crack segmentation method based on dynamic state-space augmented network includes the following steps:

[0013] Collect images of concrete cracks;

[0014] Data augmentation was performed on concrete crack images to obtain a concrete crack dataset.

[0015] A dynamic state-space augmentation network was trained using a concrete crack dataset.

[0016] A trained dynamic state space augmentation network is used to segment cracks in concrete images.

[0017] To optimize the above technical solution, the specific measures also include:

[0018] Furthermore, the data augmentation of the concrete crack images specifically includes: random flipping, random cropping, random deformation, rotation, color jittering, and scaling operations.

[0019] Furthermore, the dynamic state space enhancement network is specifically DSS-MobileNetV3;

[0020] The structure of DSS-MobileNetV3 is a U-shaped encoder-decoder architecture. DSS-MobileNetV3 includes an encoder, a decoder, and a strip pooling module, which is used for skip connections between the encoder and the decoder.

[0021] The encoder includes a 3×3 convolutional layer and a state-space-based MobileNetV3 bottleneck module;

[0022] The decoder comprises sequentially connected upsampling layers and depthwise separable convolutions.

[0023] Furthermore, the structure of the state-space-based MobileNetV3 bottleneck module specifically includes: a 1×1 convolution, a depthwise separable convolution, a dynamic state-space module, and a 1×1 convolution connected in sequence, with residual connections between the input and output of the state-space-based MobileNetV3 bottleneck module; the implementation process of the state-space-based MobileNetV3 bottleneck module is expressed by the following formula:

[0024] x dw =DSConv(Expand(x) in ))

[0025] In the formula, x in For input features of the state-space-based MobileNetV3 bottleneck module, Expand(·) denotes a 1×1 convolution used to expand the number of channels, and DSConv(·) denotes a depthwise separable convolution used to extract local features. dw This represents the features of the depthwise separable convolution output;

[0026] x out =Conv(DSS(x) dw ))+x in

[0027] In the formula, x out For the final output of the state-space based MobileNetV3 bottleneck module, Conv(·) represents the 1×1 convolution used to compress the channels, and DSS(·) represents the dynamic state-space module.

[0028] Furthermore, the dynamic state space module includes a linear layer for feature expansion, dynamic serpentine convolutions in the horizontal and vertical directions, a SiLU function for local representation enhancement, a two-dimensional selective scan module for modeling two-dimensional image context relations with linear complexity, and layer normalization for standardizing deep features.

[0029] The workflow of the dynamic state space module can be expressed by the following formula:

[0030] x DSS =(Linear*DSC*SiLu*SS2D*LN)(x dw )

[0031] In the formula, x DSS The output of the dynamic state space module is represented by Linear, DSC is a dynamic snake convolution, SiLU is the SiLU activation function, SS2D is the two-dimensional selection scan module, LN is the layer normalization, and "*" is the cascade operation.

[0032] Furthermore, the two-dimensional selective scanning module includes the processes of cross scanning, selective scanning, and cross merging;

[0033] The cross-scanning specifically involves scanning slices of a two-dimensional image in four different directions to obtain sequences in four scanning directions; the selective scanning specifically involves applying the S6 block in Mamba to process the sequences from the four scanning directions simultaneously; the cross-merging specifically involves integrating and reshaping the sequences processed by the S6 block to generate the output.

[0034] Furthermore, the accumulation process of the dynamic serpentine convolution in the horizontal direction is specifically as follows:

[0035] The kernel center coordinates of dynamic serpentine convolution are defined as K. i =(x i ,y i In a 3×3 receptive field, starting from the center point K i K i±1 By K i Given the offset Δ, where Δ∈[-1,1], the accumulation process of dynamic serpentine convolution in the horizontal direction is expressed by the following formula:

[0036]

[0037] In the formula, K i±c =(x i±c ,y i±c ) represents the coordinates of all pixels within the horizontally offset convolution kernel, x i The x-coordinate of the center of the convolution kernel, y i Δy represents the ordinate of the convolution kernel center, c represents the number of pixels from the kernel center (c∈[0,4]), i represents the horizontal pixel index, and Δy represents the learnable vertical offset.

[0038] The accumulation process of dynamic serpentine convolution in the vertical direction is as follows:

[0039]

[0040] In the formula, K j±c =(x j±c ,y j±c ) represents the coordinates of all pixels within the convolution kernel after vertical offset, x j The x-coordinate of the center of the convolution kernel, y j Δx represents the ordinate of the center of the convolution kernel, c represents the number of pixels from the center of the convolution kernel, c∈[0,4], j represents the index of the vertical pixel, and Δx represents the learnable offset in the horizontal direction.

[0041] Furthermore, the strip pooling module includes strip pooling branches and skip connection branches, given a feature x from the encoder. e ,

[0042] The strip pooling branch first performs horizontal and vertical strip pooling to extract strip features in both directions in parallel. One-dimensional convolution and interpolation are then used to integrate the strip features and restore spatial resolution. Finally, a 1×1 convolution and the sigmoid function are used to fuse strip features from different directions, outputting x′. e The formula is expressed as follows:

[0043] x′ e =Sigmoid(Conv(F(Conv)) 1D (SP h (x e )))))+F(Conv 1D (SP v (x e )))

[0044] Where Sigmoid(·) represents the Sigmoid function, SP h and SP v These represent strip pooling operations in the horizontal and vertical directions, respectively; F(·) is the interpolation operation; Conv(·) represents a 1×1 convolution. 1D (·) represents 1D convolution.

[0045] The skip connection branch obtains the final output of the bar pooling module by multiplying the output by the original input, as expressed by the formula:

[0046] x sp =x′ e ⊙x e

[0047] Where ⊙ represents the Hadamard product, x sp This represents the final output of the bar pooling module.

[0048] Furthermore, the loss functions used in training the dynamic state space augmentation network are cross-entropy loss and boundary loss;

[0049] The total loss expression is as follows:

[0050] Loss total =λ·Loss CE +(1-λ)·Loss Bo

[0051] Loss CE=-[GT·log(Pred)+(1-GT)·log(1-Pred)]

[0052] Loss Bo =|DT(Pred)-DT(GT)|

[0053] Among them, Loss total It is the total loss, Loss CE It's cross-entropy loss, Loss Bo λ is the boundary loss, λ is the hyperparameter that balances the contribution of each loss term, Pred and GT are the predicted and true values, respectively, and DT(·) is the distance transformation operation.

[0054] This invention also proposes a real-time concrete crack segmentation system based on a dynamic state-space augmentation network, comprising:

[0055] The data acquisition module is used to acquire images of concrete cracks.

[0056] The data augmentation module is used to augment concrete crack images to obtain a concrete crack dataset.

[0057] The training module is used to train a dynamic state-space augmentation network using a concrete crack dataset;

[0058] The segmentation module is used to segment cracks in concrete images using a trained dynamic state space augmentation network.

[0059] The beneficial effects of this invention are:

[0060] This invention proposes a real-time concrete crack segmentation method and system based on a dynamic state space augmentation network (DSS) for accurate and efficient concrete crack segmentation. The invention embeds a dynamic state space (DSS) module into the encoder to enhance the MobileNetV3 bottleneck module's ability to model global dependencies. Specifically, the SE module in the original MobileNetV3 bottleneck module is replaced with the proposed DSS module to enhance its expressive power. During the decoding stage, upsampling and depthwise separable convolutions are used to progressively restore spatial resolution. Furthermore, to highlight crack texture and suppress background-related noise, a strip pooling module is embedded in the skip connections between the encoder and decoder. Extensive experiments on three public datasets demonstrate that the proposed method outperforms the proposed method in terms of both accuracy and efficiency. Attached Figure Description

[0061] Figure 1 This is a network structure diagram of the DSS-MobileNetV3 proposed in this invention;

[0062] Figure 2This is a schematic diagram of a DSS block;

[0063] Figure 3 This is a schematic diagram of the dynamic accumulation process of dynamic serpentine convolution in the x-axis direction;

[0064] Figure 4 This is a schematic diagram of the strip pooling module;

[0065] Figure 5 Example images are provided for three public datasets. The images in the first, second, and third rows belong to the DeepCrack, Crack500, and CFD datasets, respectively.

[0066] Figure 6 For a visual comparison of the best-performing methods on the DeepCrack dataset, Figure 6 (a) The actual situation Figure 6 (b) Unet segmentation results, Figure 6 (c) Segmentation results of RHACrackNet Figure 6 (d) CarNet segmentation results, Figure 6 (e) The segmentation results of the real-time concrete crack segmentation method based on dynamic state space enhancement network proposed in this invention. The red rectangle highlights the differences between the prior art method and the method proposed in this invention.

[0067] Figure 7 A visual comparison of the best-performing methods on the Crack500 dataset. Figure 7 (a) is the truth image. Figure 7 (b) Unet segmentation results, Figure 7 (c) Segmentation results of RHACrackNet Figure 7 (d) CarNet segmentation results, Figure 7 (e) Segmentation results of the method proposed in this invention; the red rectangle highlights the differences between the prior art method and the method proposed in this invention;

[0068] Figure 8 This is a visual comparison of the best-performing methods on the CFD dataset. Figure 8 (a) Truth image, Figure 8 (b) Segmentation results of U-Net, Figure 8 (c) Segmentation results of RHACrackNet Figure 8 (d) CarNet segmentation results, Figure 8 (e) The segmentation result of the method proposed in this invention, with the red rectangle highlighting the difference between the prior art method and the method proposed in this invention;

[0069] Figure 9 These are the IoU and loss curves for three public datasets. Figure 9(a) and Figure 9 (b) is the curve corresponding to the DeepCrack dataset. Figure 9 (c) and Figure 9 (d) is the curve corresponding to the Crack500 dataset. Figure 9 (e) and Figure 9 (f) is the curve corresponding to the CFD dataset;

[0070] Figure 10 This is a comparison of the parameters and efficiency of each method. The size of the circle represents the scale of the model parameters, and FPS (frames per second) and FLOPs (floating-point operations) represent the efficiency of the model.

[0071] Figure 11 It is a study on the ablation of the hyperparameter λ of the loss function. Figure 11 (a) Figure 11 (b) and Figure 11 (c) shows the parameter changes for DeepCrack, Crack500 and CFD datasets respectively. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0073] Example 1

[0074] This invention proposes a real-time concrete crack segmentation method based on a dynamic state-space augmentation network, comprising the following steps:

[0075] Collect images of concrete cracks;

[0076] Data augmentation is performed on concrete crack images to obtain a concrete crack dataset. The data augmentation includes random flipping, random cropping, random deformation, rotation, color jittering, and scaling operations.

[0077] A dynamic state space augmentation network is trained using a concrete crack dataset; in this embodiment, the dynamic state space augmentation network is specifically DSS-MobileNetV3.

[0078] The structure of DSS-MobileNetV3 is a U-shaped encoder-decoder architecture, such as... Figure 1As shown, DSS-MobileNetV3 includes an encoder, a decoder, and a bar pooling module, which is used for skip connections between the encoder and decoder. Directly using skip connections to fuse deep and shallow features may introduce background-related noise and disrupt the structured representation of cracks. Images captured by mobile devices contain various types of objects, and uneven lighting may reduce the distinguishability between crack pixels and background pixels. To highlight crack features and suppress noise, this invention embeds the bar pooling module into the skip connections between the encoder and decoder.

[0079] The encoder includes 3×3 convolutional layers and a state-space-based MobileNetV3 bottleneck module;

[0080] The bottleneck module in traditional MobileNetV3 mainly consists of depthwise separable convolutions, inverted residual structures, Squeeze-and-Excitation (SE) attention modules, and Hard-Swish activation functions. It can efficiently model local dependencies and encode features. However, it still cannot escape the inherent limitations of convolutions because it cannot model global dependencies for various types of cracks. To address this issue, we propose a state-space-based bottleneck module for MobileNetV3. Unlike the bottleneck module in traditional MobileNetV3, we replace the SE attention module with a Dynamic State Space (DSS) block to bridge the gap between the state-space model and two-dimensional crack images.

[0081] The structure of the state-space-based MobileNetV3 bottleneck module specifically includes: a cascaded 1×1 convolution, a depthwise separable convolution, a dynamic state-space module, and another 1×1 convolution; a residual connection between the input and output of the state-space-based MobileNetV3 bottleneck module; the implementation process of the state-space-based MobileNetV3 bottleneck module is expressed by the following formula:

[0082] x dw =DSConv(Expand(x) in ))

[0083] In the formula, x in For input features of the state-space-based MobileNetV3 bottleneck module, Expand(·) denotes a 1×1 convolution used to expand the number of channels, and DSConv(·) denotes a depthwise separable convolution used to extract local features. dw This represents the features of the depthwise separable convolution output;

[0084] x out=Conv(DSS(x) dw ))+x in

[0085] In the formula, x out For the final output of the state-space based MobileNetV3 bottleneck module, Conv(·) represents the 1×1 convolution used to compress the channels, and DSS(·) represents the dynamic state-space module.

[0086] Dynamic state space module such as Figure 2 As shown, it includes linear layers for feature expansion, dynamic snake convolutions (DSC) in the horizontal and vertical directions, a SiLU function for local representation enhancement, a two-dimensional selection scan module for modeling two-dimensional image context relationships with linear complexity, and layer normalization for standardizing deep features.

[0087] The workflow of the dynamic state space module can be expressed by the following formula:

[0088] x DSS =(Linear*DSC*SiLu*SS2D*LN)(x dw )

[0089] In the formula, x DSS The output of the dynamic state space module is represented by Linear, DSC is a dynamic snake convolution, SiLU is the SiLU activation function, SS2D is the two-dimensional selection scan module, LN is the layer normalization, and "*" is the cascade operation.

[0090] The two-dimensional selective scanning module includes the processes of cross scanning, selective scanning, and cross merging;

[0091] The cross-scanning specifically involves scanning slices of a two-dimensional image in four different directions to obtain sequences in four scanning directions; the selective scanning specifically involves applying the S6 block (Selective Scan Space State Sequential Model, S6) in Mamba to process the sequences from the four scanning directions simultaneously; the cross-merging specifically involves integrating and reshaping the sequences processed by the S6 block to generate the output.

[0092] The accumulation process of dynamic serpentine convolution in the horizontal direction (x-axis) is as follows: Figure 3 As shown, specifically:

[0093] The kernel center coordinates of dynamic serpentine convolution are defined as K. i =(x i ,y i In a 3×3 receptive field, starting from the center point Ki K i±1 By K i Given the offset Δ, where Δ∈[-1,1], the accumulation process of dynamic serpentine convolution in the horizontal direction is expressed by the following formula:

[0094]

[0095] In the formula, K i±c =(x i±c ,y i±c ) represents the coordinates of all pixels within the horizontally offset convolution kernel, x i The x-coordinate of the center of the convolution kernel, y i Δy represents the ordinate of the convolution kernel center, c represents the number of pixels from the kernel center (c∈[0,4]), i represents the horizontal pixel index, and Δy represents the learnable vertical offset.

[0096] The accumulation process of dynamic serpentine convolution in the vertical direction is as follows:

[0097]

[0098] In the formula, K j±c =(x j±c ,y j±c ) represents the coordinates of all pixels within the convolution kernel after vertical offset, x j The x-coordinate of the center of the convolution kernel, y j Δx represents the ordinate of the center of the convolution kernel, c represents the number of pixels from the center of the convolution kernel, c∈[0,4], j represents the index of the vertical pixel, and Δx represents the learnable offset in the horizontal direction.

[0099] The decoder consists of sequentially connected upsampling layers and depthwise separable convolutions.

[0100] Strip pooling module such as Figure 4 As shown, it includes strip pooling branches and skip connection branches, given features x from the encoder. e ,

[0101] The strip pooling branch first performs horizontal and vertical strip pooling to extract strip features in both directions in parallel. One-dimensional convolution and interpolation are then used to integrate the strip features and restore spatial resolution. Finally, a 1×1 convolution and the sigmoid function are used to fuse strip features from different directions, outputting x′. e The formula is expressed as follows:

[0102] x′ e =Sigmoid(Conv(F(Conv)) 1D (SP h (xe )))))+F(Conv 1D (SP v (x e )))

[0103] Where Sigmoid(·) represents the Sigmoid function, SP h and SP v These represent bar pooling operations in the horizontal and vertical directions, respectively; F(·) is the interpolation operation; Conv() represents a 1×1 convolution. 1D () denotes 1D convolution.

[0104] The skip connection branch obtains the final output of the bar pooling module by multiplying the output by the original input, as expressed by the formula:

[0105] x sp =x′ e ⊙x e

[0106] Where ⊙ represents the Hadamard product, x sp This represents the final output of the bar pooling module.

[0107] The loss functions used to train dynamic state-space augmentation networks are cross-entropy loss and boundary loss.

[0108] The total loss expression is as follows:

[0109] Loss total =λ·Loss CE +(1-λ)·Loss Bo

[0110] Loss CE =-[GT·log(Pred)+(1-GT)·log(1-Pred)]

[0111] Loss Bo =|DT(Pred)-DT(GT)|

[0112] Among them, Loss total It is the total loss, Loss CE It's cross-entropy loss, Loss Bo λ is the boundary loss, λ is the hyperparameter that balances the contribution of each loss term, Pred and GT are the predicted and true values, respectively, and DT(·) is the distance transformation operation.

[0113] A trained dynamic state space augmentation network is used to segment cracks in concrete images.

[0114] Example 2

[0115] This invention proposes a real-time concrete crack segmentation system based on a dynamic state-space augmentation network, corresponding to the method in Embodiment 1, comprising:

[0116] The data acquisition module is used to acquire images of concrete cracks.

[0117] The data augmentation module is used to augment concrete crack images to obtain a concrete crack dataset.

[0118] The training module is used to train a dynamic state-space augmentation network using a concrete crack dataset;

[0119] The segmentation module is used to segment cracks in concrete images using a trained dynamic state space augmentation network.

[0120] The implementation methods of each module and its function in the system are completely consistent with the steps of the method in Implementation Example 1, so they will not be repeated here.

[0121] Example 3

[0122] This invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the real-time concrete crack segmentation method based on a dynamic state-space augmented network as described in Embodiment 1.

[0123] Example 4

[0124] This invention proposes a computer-readable storage medium storing a computer program that enables a computer to execute the real-time concrete crack segmentation method based on a dynamic state-space augmented network as described in Embodiment 1.

[0125] The effectiveness of this invention will be demonstrated through experiments below.

[0126] (1) Experimental Dataset

[0127] The DeepCrack dataset is a publicly available dataset specifically designed for crack segmentation tasks, containing 537 RGB images and their manually annotated mask images. Figure 5 As shown in the first row, each image and its corresponding mask image have a resolution of 544×284 pixels. In this invention, we follow the original settings and divide the DeepCrack dataset into a training set and a test set, containing 300 and 237 image-mask pairs, respectively.

[0128] The Crack500 dataset. The publicly available Crack500 dataset

[32] was photographed using a mobile phone at Temple University's main campus in the United States, such as Figure 5The second row shows the image. Each image and its corresponding labeled mask contain road surface cracks, with a resolution of approximately 2000×1500 pixels. In the original dataset, the training, validation, and test sets contain 250, 50, and 200 images, respectively. To balance the high resolution of the images with limited computational resources, these high-resolution images are cropped into 16 non-overlapping sub-regions, and only slices containing more than 1000 crack pixels are retained. Based on this method, the training, validation, and test sets contain 1896, 348, and 1124 image patches, respectively.

[0129] CFD dataset. The CFD dataset was collected using an iPhone 5 on urban roads in Beijing. For example... Figure 5 As shown in the last line, the challenges of the CFD dataset include complex backgrounds, uneven lighting, and interference from various occlusions such as water stains, oil stains, and lane lines. The dataset contains 118 images with a resolution of 480×320 pixels and their manually annotated mask images. In our experiments, the dataset was split into training and test sets in a 6:4 ratio.

[0130] (2) Performance evaluation indicators

[0131] To quantitatively evaluate the segmentation performance of the model, we selected six evaluation metrics: Intersection over Union (IoU), Dice Similarity Coefficient (DSC), Precision (Pre), Accuracy (Acc), and Centerline Dice (clDice). The first five metrics are mainly used to evaluate pixel-level accuracy, while clDice evaluates the difference between predicted and true values ​​based on the connectivity of the curve. The calculation principles of these metrics are as follows:

[0132]

[0133] TP, FP, FN, and TN represent true positive, false positive, false negative, and true negative, respectively.

[0134] (3) Experimental details

[0135] All experiments in this invention were conducted on a server equipped with an Ubuntu 18.04 operating system and a GeForce RTX 3070 8GB GPU. For the software environment, we chose the PyTorch 2.0 framework and configured the necessary dependencies using Anaconda version 23.5.2. During model training, we carefully configured the hyperparameters: the batch size was set to 4, the initial learning rate was 0.0001, and the number of training iterations was 80,000. We used the AdamW optimizer to optimize the deep learning model. Furthermore, to effectively prevent overfitting, we employed data augmentation techniques, including random flipping, random cropping, and random deformation, as well as rotation, color jittering, and scaling operations. All images were resized to 256×256 pixels. In the loss function, the hyperparameter λ was set to 0.6.

[0136] (4) Comparative Experiment

[0137] This invention compares the proposed method with nine different segmentation methods, including FCN, U-Net, MobileNetV1, MobileNetV2, MobileNetV3, CrackSegNet, CrackW-Net, TEED, CrackFormer, DECSNet, RHACrackNet, and CarNet, and tests them on three public datasets. These methods include those specifically designed for real-time crack segmentation and general segmentation models. All methods are loaded with weights pre-trained on ImageNet-1K.

[0138] This invention quantifies the performance of all methods on the DeepCrack dataset, and the results are recorded in Table 1. Our observations show that the proposed model achieves the highest scores across all metrics. Compared to the general-purpose U-Net, our method improves IoU, DSC, recall, accuracy, and clDice scores by 5.18%, 2.53%, 9.97%, 0.47%, and 2.28%, respectively. Furthermore, the proposed method outperforms the CarNet model specifically designed for crack segmentation by 0.48% in IoU, 1.09% in DSC, 1.07% in recall, 0.16% in accuracy, and 0.1% in clDice. For a more intuitive comparison, we... Figure 6The article visualizes the best-performing models, including U-Net, RHACrackNet, and CarNet. We observe that U-Net and RHACrackNet segmentation results contain more false-detection pixels, while CarNet exhibits discontinuities in its segmentation of fine cracks. In contrast, the proposed method demonstrates higher accuracy in crack detection from DeepCrack images, especially in regions of crack discontinuity. Therefore, our method is competitive on the publicly available DeepCrack dataset compared to its competitors.

[0139] As shown in Table 2, the model proposed in this invention consistently outperforms all other models across all evaluation metrics. Specifically, in terms of IoU, DSC, recall, accuracy, and clDice, this model improves upon the Unet model by 3.43%, 1.81%, 9.53%, 0.59%, and 1.81%, respectively. Furthermore, the method proposed in this invention also outperforms the CarNet model specifically designed for crack segmentation, improving IoU by 0.52%, DSC by 0.33%, recall by 0.39%, accuracy by 0.54%, and clDice by 1.77%. To more clearly compare segmentation performance, we... Figure 7 The results for UNet and CarNet are visualized in the diagram. We observe that for Crack500 images, our proposed model exhibits more accurate segmentation performance, significantly reducing false positives and breakpoints. For mesh structures formed by interconnected annular cracks, the method demonstrates a stronger ability to capture pixel-level correlations. Furthermore, it successfully identifies crack textures that are ignored in ground truth labeled images. Therefore, the proposed method achieves state-of-the-art performance on the publicly available Crack500 dataset.

[0140] Table 1 shows the performance (%) of each method on the DeepCrack dataset

[31] .

[0141]

[0142] Table 2 shows the performance (%) of each method on the Crack500 dataset

[32] .

[0143]

[0144] This experiment quantified the performance of all methods on the CFD dataset, and the results are recorded in Table 3. We found that the proposed method achieved the highest scores on all metrics. Compared with the Unet model, the proposed method outperforms it by 6.84%, 1.52%, 5.97%, 1.82%, and 1.58% in IoU, DSC, recall, accuracy, and clDice, respectively. Furthermore, the proposed model also outperforms the CarNet model, which is specifically designed for crack segmentation, by 1.14%, 0.27%, 0.48%, 0.23%, and 0.33% in IoU, DSC, recall, accuracy, and clDice, respectively. To clearly compare the segmentation performance, we... Figure 8 The U-Net and CarNet methods are visualized in the diagram. We find that U-Net and RHACrackNet perform poorly in segmenting complex annular crack networks, failing to accurately capture pixel-level connectivity relationships. In contrast, the proposed model demonstrates higher accuracy and coherence in segmenting long-span and complex mesh-like crack structures. Furthermore, the proposed method achieves more accurate identification of crack intersections.

[0145] Table 3 shows the performance (%) of each method on the CFD dataset

[33] .

[0146]

[0147] This experiment demonstrates the training process of the proposed method and recent state-of-the-art (SOTA) models on three publicly available datasets. Figure 9 As shown in the first row, compared to the three state-of-the-art (SOTA) methods RHACrackNet, DECSNet, and CarNet, the proposed method converges faster and with lower loss on the DeepCrack dataset. On the challenging Crack500 dataset, DECSNet exhibits severe oscillations and fails to converge stably. In contrast, our proposed method demonstrates stable convergence. On the CFD dataset, the IoU and loss curves of the proposed method also converge quickly and achieve higher accuracy, effectively validating the superiority of the proposed method. Therefore, the proposed method demonstrates stable segmentation performance on three public datasets and can quickly converge to the optimum, proving its superiority.

[0148] To evaluate the real-time performance of the models, we report the number of parameters, floating-point operations (FLOPs), and frame rate (FPS) values ​​for all methods. As shown in Table 4, the model proposed in this invention has significantly fewer parameters compared to general methods. Furthermore, compared to real-time segmentation models, the method proposed in this invention has lower FLOPs and faster FPS. To visually evaluate the efficiency of all models, their performance has been visualized based on the number of parameters, FLOPs, and FPS. Figure 10 As shown, the method proposed in this invention is located in the lower right corner and has the smallest radius, indicating fewer parameters, fewer FLOPs, and faster FPS. Therefore, the method of this invention is competitive in real-time crack segmentation tasks.

[0149] Table 4 shows the number of parameters and floating-point operations (FLOPs) for each method.

[0150]

[0151] (5) Ablation test

[0152] To evaluate the contribution of each module in the proposed model, we conducted corresponding ablation experiments. As shown in Table 5, using MobileNetV3 (minor version) as the backbone network and introducing a progressive decoder, we found that the proposed method improved the IoU score by 0.48%. Furthermore, we observed that the DSS module contributed the most to the model, improving the IoU score by 0.75%, indicating that the proposed DSS module has great potential in crack segmentation tasks. Finally, we conducted ablation experiments on the strip pooling module, finding that it improved the IoU score by 0.62%. Therefore, based on the above analysis, we have demonstrated the effectiveness of each module in the proposed method.

[0153] Table 5 shows the ablation study of the segmentation method proposed in this invention.

[0154]

[0155] To investigate the impact of different input image resolutions on the proposed method, we conducted corresponding ablation experiments. As shown in Table 6, the proposed method performed poorly when using input images with resolutions of 128×128 and 224×224. We believe this is because the images were scaled too small, leading to the loss of fine-grained features. When we increased the input image resolution to 448×448, we found that the proposed method scored lower in recall, accuracy, and clDice metrics. This indicates that increasing the input image resolution may introduce noise interference, thereby destroying the structural features of the cracks. In contrast, by scaling the input image to 256×256, the proposed method achieved state-of-the-art (SOTA) performance.

[0156] Table 6 shows the ablation study on input image resolution on the DeepCrack dataset

[31] .

[0157]

[0158] In Table 6, “★” indicates the method proposed in this invention.

[0159] Furthermore, we evaluated the hyperparameters controlling the fusion of cross-entropy loss and boundary loss on three public datasets, such as Figure 11 As shown, we found that when λ = 0.6, the proposed method achieved the highest Intersection over Union (IoU) score on all three datasets.

[0160] (6) Experimental Conclusion

[0161] In this invention, we propose a lightweight network with dynamic state space enhancement for accurate and efficient segmentation of concrete cracks. We embed a dynamic state space (DSS) module into the encoder to enhance the MobilenetV3 bottleneck module's ability to model global dependencies. Specifically, we replace the SE module in the original MobilenetV3 bottleneck module with our proposed DSS module to enhance its expressive power. In the decoding stage, we utilize upsampling and depthwise separable convolutions to progressively restore spatial resolution. Furthermore, to highlight crack texture and suppress background-related noise, we embed a strip pooling module into skip connections between the encoder and decoder. We conduct extensive experiments on three public datasets, demonstrating that our proposed method outperforms others in both accuracy and efficiency.

[0162] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0163] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0164] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A real-time concrete crack segmentation method based on a dynamic state-space augmented network, characterized in that, Includes the following steps: Collect images of concrete cracks; Data augmentation was performed on concrete crack images to obtain a concrete crack dataset. A dynamic state-space augmentation network was trained using a concrete crack dataset; specifically, the dynamic state-space augmentation network is DSS-MobileNetV3. The structure of DSS-MobileNetV3 is a U-shaped encoder-decoder architecture. DSS-MobileNetV3 includes an encoder, a decoder, and a strip pooling module, which is used for skip connections between the encoder and the decoder. The encoder includes a 3×3 convolutional layer and a state-space-based MobileNetV3 bottleneck module; The decoder includes sequentially connected upsampling layers and depthwise separable convolutions; The structure of the state-space-based MobileNetV3 bottleneck module specifically includes: a 1×1 convolution, a depthwise separable convolution, a dynamic state-space module, and a 1×1 convolution connected in sequence, with residual connections between the input and output of the state-space-based MobileNetV3 bottleneck module; the implementation process of the state-space-based MobileNetV3 bottleneck module is expressed by the following formula: In the formula, x in For inputting features of the state-space-based MobileNetV3 bottleneck module, Expand(·) denotes a 1×1 convolution used to expand the number of channels, and DSConv(·) denotes a depthwise separable convolution used to extract local features. x dw This represents the features of the depthwise separable convolution output; In the formula, x out For the final output of the state-space based MobileNetV3 bottleneck module, Conv(·) represents the 1×1 convolution used to compress the channel, and DSS(·) represents the dynamic state-space module; The dynamic state space module includes a linear layer for feature expansion, dynamic serpentine convolutions in the horizontal and vertical directions, a SiLU function for local representation enhancement, a two-dimensional selective scan module for modeling two-dimensional image context relations with linear complexity, and layer normalization for standardizing deep features. The workflow of the dynamic state space module can be expressed by the following formula: In the formula, x DSS The output of the dynamic state-space module is represented by "Linear", "DSC", "SiLU", "SS2D", and "LN". "It is a cascading operation; A trained dynamic state space augmentation network is used to segment cracks in concrete images.

2. The real-time concrete crack segmentation method based on dynamic state-space augmented network as described in claim 1, characterized in that, The data augmentation of concrete crack images specifically includes: random flipping, random cropping, random deformation, rotation, color jittering, and scaling operations.

3. The real-time concrete crack segmentation method based on dynamic state-space augmented network as described in claim 1, characterized in that, The two-dimensional selective scanning module includes the processes of cross scanning, selective scanning, and cross merging; The cross-scanning specifically involves scanning slices of a two-dimensional image in four different directions to obtain sequences in four scanning directions; the selective scanning specifically involves applying the S6 block in Mamba to process the sequences from the four scanning directions simultaneously; the cross-merging specifically involves integrating and reshaping the sequences processed by the S6 block to generate the output.

4. The real-time concrete crack segmentation method based on dynamic state-space augmented network as described in claim 1, characterized in that, The accumulation process of the dynamic serpentine convolution in the horizontal direction is specifically as follows: The kernel center coordinates of dynamic serpentine convolution are defined as K. i =( x i , y i In a 3×3 receptive field, starting from the center point K i K i±1 By K i And the offset Δ is determined, Δ∈ [ [1, 1], the accumulation process of dynamic serpentine convolution in the horizontal direction is expressed by the following formula: In the formula, K i±c =( x i±c , y i±c ) represents the coordinates of all pixels within the horizontally offset convolution kernel, x i The x-coordinate of the center of the convolution kernel, y i Δy represents the ordinate of the convolution kernel center, c represents the number of pixels from the kernel center (c∈ [0, 4]), i represents the horizontal pixel index, and Δy represents the learnable vertical offset. The accumulation process of dynamic serpentine convolution in the vertical direction is as follows: In the formula, M j±c =( u j±c , v j±c ) represents the coordinates of all pixels within the convolution kernel after vertical offset, u j v represents the x-coordinate of the center of the convolution kernel. j Δu represents the ordinate of the center of the convolution kernel, c represents the number of pixels from the center of the convolution kernel, c∈ [0, 4], j represents the index of the vertical pixel, and Δu represents the learnable offset in the horizontal direction.

5. The real-time concrete crack segmentation method based on dynamic state-space augmented network as described in claim 1, characterized in that, The strip pooling module includes strip pooling branches and skip connection branches, given features from the encoder. x e , The strip pooling branch first performs horizontal and vertical strip pooling to extract strip features in both directions in parallel. One-dimensional convolution and interpolation are then used to integrate the strip features and restore spatial resolution. Finally, a 1×1 convolution and the sigmoid function are used to fuse strip features from different directions, resulting in the output. x ′ e The formula is expressed as follows: Where Sigmoid(·) represents the Sigmoid function. SP h and SP v These represent the horizontal and vertical bar pooling operations, respectively, and F(·) is the interpolation operation; Represents a 1×1 convolution. Represents 1D convolution The skip connection branch obtains the final output of the bar pooling module by multiplying the output by the original input, as expressed by the formula: Where ⊙ represents the Hadamard product. This represents the final output of the bar pooling module.

6. The real-time concrete crack segmentation method based on dynamic state-space augmented network as described in claim 1, characterized in that, The loss functions used in training the dynamic state space augmentation network are cross-entropy loss and boundary loss. The total loss expression is as follows: in, Loss total It is the total loss. Loss CE It is cross-entropy loss. Loss Bo It is boundary loss. λ It is a hyperparameter that balances the contribution of each loss term, Pred and GT are the predicted value and the true value, respectively, and DT(·) is the distance transformation operation.

7. A real-time concrete crack segmentation system based on a dynamic state-space augmented network, characterized in that, include: The data acquisition module is used to acquire images of concrete cracks. The data augmentation module is used to augment concrete crack images to obtain a concrete crack dataset. The training module is used to train a dynamic state-space augmentation network using a concrete crack dataset; the dynamic state-space augmentation network is specifically DSS-MobileNetV3. The structure of DSS-MobileNetV3 is a U-shaped encoder-decoder architecture. DSS-MobileNetV3 includes an encoder, a decoder, and a strip pooling module, which is used for skip connections between the encoder and the decoder. The encoder includes a 3×3 convolutional layer and a state-space-based MobileNetV3 bottleneck module; The decoder includes sequentially connected upsampling layers and depthwise separable convolutions; The structure of the state-space-based MobileNetV3 bottleneck module specifically includes: a 1×1 convolution, a depthwise separable convolution, a dynamic state-space module, and a 1×1 convolution connected in sequence, with residual connections between the input and output of the state-space-based MobileNetV3 bottleneck module; the implementation process of the state-space-based MobileNetV3 bottleneck module is expressed by the following formula: In the formula, x in For inputting features of the state-space-based MobileNetV3 bottleneck module, Expand(·) denotes a 1×1 convolution used to expand the number of channels, and DSConv(·) denotes a depthwise separable convolution used to extract local features. x dw This represents the features of the depthwise separable convolution output; In the formula, x out For the final output of the state-space based MobileNetV3 bottleneck module, Conv(·) represents the 1×1 convolution used to compress the channel, and DSS(·) represents the dynamic state-space module; The dynamic state space module includes a linear layer for feature expansion, dynamic serpentine convolutions in the horizontal and vertical directions, a SiLU function for local representation enhancement, a two-dimensional selective scan module for modeling two-dimensional image context relations with linear complexity, and layer normalization for standardizing deep features. The workflow of the dynamic state space module can be expressed by the following formula: In the formula, x DSS The output of the dynamic state-space module is represented by "Linear", "DSC", "SiLU", "SS2D", and "LN". "It is a cascading operation; The segmentation module is used to segment cracks in concrete images using a trained dynamic state space augmentation network.

Images