Road crack segmentation method based on SRA-Net
By employing the SRA-Net model in pavement crack segmentation, combined with void space pyramid pooling and spatial channel modules, the problem of insufficient fine-grained feature extraction is solved, thereby improving the accuracy and effectiveness of pavement crack segmentation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-08
- Publication Date
- 2026-04-07
AI Technical Summary
Existing methods for pavement crack segmentation suffer from poor fine-grained feature extraction and insensitivity to small cracks and inconspicuous features, resulting in suboptimal segmentation performance.
A road surface crack segmentation method based on SRA-Net is adopted. By adding a void space pyramid pooling module and a spatial channel module to the encoder, a neural network model with an encoder-decoder structure is constructed. The model is trained by adding cross-entropy loss and Dice loss to enhance the feature extraction capability of small cracks and low-resolution images.
It improves the accuracy of road surface crack segmentation, enhances the model's ability to extract features from small cracks and low-resolution images, and improves the segmentation effect.
Smart Images
Figure CN116630617B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image segmentation, and more particularly to a method for road surface crack segmentation based on SRA-Net. Background Technology
[0002] Traditional manual crack segmentation methods, while simple, often suffer from drawbacks such as high subjectivity and labor costs, leading to inconsistent results and low reliability. Manual segmentation methods are no longer adequate for the demands of highway development. Since the 1980s, advancements in artificial intelligence and computer vision have gradually shifted the collection and evaluation of pavement condition data from manual to automated processes. Compared to traditional manual segmentation, automated methods overcome the shortcomings of manual detection and are more sensitive to crack features, which is crucial for later maintenance decisions. While traditional machine learning methods offer some automation, they often still require manual feature extraction. Furthermore, factors such as complex crack backgrounds, random crack locations, and poor image quality due to lighting conditions during acquisition contribute to the poor robustness and limited generalization ability of traditional machine learning methods. Since deep learning was first applied to pavement crack segmentation in 2016, continuous development has largely overcome the shortcomings of traditional machine learning. However, existing models still suffer from susceptibility to noise, insufficient feature extraction for small cracks, and low robustness to complex backgrounds.
[0003] Therefore, in order to solve the above-mentioned technical problems, it is urgent to propose a new technical approach. Summary of the Invention
[0004] In view of this, the present invention provides a pavement crack segmentation method based on SRA-Net, which aims to solve the problems of poor performance of classic pavement crack semantic segmentation models in extracting fine-grained features and insensitivity to small cracks and inconspicuous features, resulting in poor segmentation performance, and improve the accuracy, IoU and other indicators of existing pavement crack semantic segmentation models.
[0005] This invention provides a pavement crack segmentation method based on SRA-Net, comprising the following steps:
[0006] S1. Obtain a dataset of road surface crack image samples;
[0007] S2. Construct an SRA-Net model based on an encoder-decoder structure, and add a void space pyramid pooling module and a space channel module to the encoder;
[0008] S3. Input the dataset into the SRA-Net model for training;
[0009] S4. Determine whether the SRA-Net model has been trained. If yes, proceed to step S5. If no, update the parameters of the SRA-Net model and return to step S3.
[0010] S5. Input the test data into the trained SRA-Net model and output the segmentation result image.
[0011] Furthermore, in step S2, the encoder module in the SRA-Net model is constructed and encoded using the following method:
[0012] The encoder and decoder are constructed by means of convolutional kernels, residual blocks and hollow spatial pyramid modules, wherein the spatial channel module is used for downsampling in the residual blocks;
[0013] Encode the image:
[0014] S211. Construct a 1×1 convolution kernel for the input image, input the RGB image into the residual block, change the number of channels in the residual block, and output feature map M1;
[0015] S212. Add a dilated spatial pyramid pooling (ASPP) module after the residual block to perform dilated convolution sampling on the input feature map;
[0016] S213. After the hollow spatial pyramid pooling ASPP module, a residual block SPD-Res Block is added again. The residual block uses the spatial channel module for downsampling. Depending on the input channel, different numbers of residual blocks are superimposed on the input feature map, and the number of channels is changed to output feature map M2.
[0017] S214. Repeat steps S212 and S213 three times each, and output feature map M3, feature map M4 and feature map M5 respectively;
[0018] S215. Input feature map M5 into the hole space pyramid pooling module ASPP, change the number of channels, and output feature map M6.
[0019] Furthermore, the dilated spatial pyramid pooling uses dilated convolutions with different sampling rates to sample in parallel, adds the sampling results to obtain a more accurate feature map, and changes the number of channels.
[0020] Furthermore, in the residual block, spatial channel modules are used for downsampling, and the nonlinear activation function ReLU6 is used as the activation function:
[0021] The spatial channel module segments the feature layers of the input residual block feature map along the channel direction and then stitches them together along the spatial direction to obtain the segmented and stitched feature map. The segmentation formula is as follows:
[0022] f 0，0=X[0: S: scale, 0: S: scale], f 1，0 =X[1:S:scale,0:S:scale],...,f scale-1，0
[0023] =X[scale-1:S:scale,0:S:scale];
[0024] f 0，1 =X[0: S: scale, 1: S: scale], f 1，1 , ..., f scale-1，1
[0025] =X[scale-1:S:scale,1:S:scale];
[0026] f 0，scale-1 =X[0: S: scale, scale-1: S: scale], f 1，scale-1 , ..., f scale-1，scale-1
[0027] =X[scale-1: S: scale, scale-1: S: scale].
[0028] Where X represents the feature map of the input residual block, S represents the length or width of the feature map of the input residual block, and f x，y This represents the subplot after downsampling, and scale represents the downsampling scaling factor.
[0029] The formula for the nonlinear activation function ReLU6 is:
[0030] f(c) = min{max(0, c), 6}
[0031] Where c is the input from the previous layer of the neural network.
[0032] Furthermore, in step S2, the decoder module in the SRA-Net model is constructed and decoded using the following method:
[0033] Construct a decoder, which includes convolutional kernels, a concat concatenation module, and a bilinear interpolation upsampling module;
[0034] Decode the feature map:
[0035] S221. Perform bilinear interpolation upsampling on the feature map M6 output by the encoder to obtain the feature map M7 in the decoder;
[0036] S222. By using skip connections, the feature map output by the decoder is concat-joined with the M1-M4 feature maps of the encoder, which have the same number of channels, to achieve feature fusion;
[0037] S223. Pass the image after Concat stitching through two 3×3 convolution kernels to change the number of channels to one-quarter of the number of channels after stitching;
[0038] S224. Perform bilinear interpolation upsampling on the image obtained in step S223 and output feature map M8;
[0039] S225. Repeat steps S222-S224 twice in sequence, and output feature map M9 and feature map M10 respectively; wherein, during the repetition, the decoder feature map concatenated with the encoder is derived from the feature map output by the previous layer after bilinear interpolation.
[0040] S226. Concat the feature map M1 and feature map M10, and pass the concatenated image through two 3×3 convolution kernels to obtain an image with 64 channels. Pass the image with 64 channels through a 1×1 convolution kernel and change the number of channels to 2. Output an image that achieves the segmentation of crack pixels and background pixels.
[0041] Furthermore, in step S3, the SRA-Net model is trained using the following method:
[0042] The dataset collected in step S1 is input into the SRA-Net model for training. The initial learning rate is set to 0.01 and the impulse is 0.9. Weight decay is used to prevent overfitting. The SDG optimizer is used to iteratively update the weights. The loss function is the sum of cross-entropy loss and Dice loss.
[0043] The loss function formula is:
[0044]
[0045] Where Loss represents the loss function, and i represents a pixel in the image. y represents the predicted value of a pixel. i The label value represents the pixel, and n represents the total number of pixels.
[0046] Furthermore, in step S4, the following method is used to determine whether the SRA-Net model has completed training:
[0047] For each dataset, the model is validated and the Dice function is calculated. If the model has the highest Dice function, the training is complete and the process proceeds to step S5. If not, the parameters of the SRA-Net model are updated, and the process returns to step S3 to continue training until the model has the highest Dice function.
[0048] The beneficial effects of this invention: This invention discloses a road surface crack segmentation method based on SRA-Net. It is a semantic segmentation method for road surface crack images that combines the advantages of residual structure and skip connection structure. By adding spatial channel module and void spatial pyramid pooling module to the residual structure, it avoids the loss of fine-grained features in traditional convolution and pooling downsampling, enhances the model's feature extraction ability for small cracks and low-resolution images, improves the connection between various feature layers, and enables the SRA-Net model to achieve better segmentation results. Attached Figure Description
[0049] The present invention will be further described below with reference to the accompanying drawings and embodiments:
[0050] Figure 1 This is a flowchart of the present invention;
[0051] Figure 2 This is a network structure diagram of the present invention;
[0052] Figure 3 This is a schematic diagram of the SPD-Res Block in this invention. Detailed Implementation
[0053] The present invention will be further described in detail below:
[0054] This invention provides a pavement crack segmentation method based on SRA-Net, comprising the following steps:
[0055] S1. Obtain a dataset of road surface crack image samples;
[0056] S2. Construct an SRA-Net model based on an encoder-decoder structure, and add a void space pyramid pooling module and a space channel module to the encoder;
[0057] S3. Input the dataset into the SRA-Net model for training;
[0058] S4. Determine whether the SRA-Net model has been trained. If yes, proceed to step S5. If no, update the parameters of the SRA-Net model and return to step S3.
[0059] S5. Input the test data into the trained SRA-Net model and output the segmentation result image. This method avoids the loss of fine-grained features in traditional convolution and pooling downsampling, enhances the model's feature extraction capabilities for small cracks and low-resolution images, and improves the connection between various feature layers.
[0060] In this embodiment, in step S1, 500 images of road surface cracks with a resolution of 3264×2448 are taken using a smartphone on the Temple University campus as a dataset.
[0061] In this embodiment, in step S2, the encoder module in the SRA-Net model is constructed and encoded using the following method:
[0062] The encoder and decoder are constructed by means of convolutional kernels, residual blocks and hollow spatial pyramid modules, wherein the spatial channel module is used for downsampling in the residual blocks;
[0063] Encode the image:
[0064] S211. Construct a 1×1 convolution kernel for the input image, add 3 residual blocks, change the number of channels from 3 to 64, and output a 480×480×64 feature map M1;
[0065] S212. Input feature map M1 Figure 2 The ASPP module in the middle inputs the feature map M1 with 64 channels into the dilated spatial pyramid pooling module, performs dilated convolution parallel sampling on the feature map M1 with different sampling rates, adds the results, and outputs the feature map M1′.
[0066] S213. Input the feature map M1′ into the residual blocks. Depending on the number of channels in the input feature map, stack different numbers of residual blocks. The number of channels and the number of stacked residual blocks can be adjusted according to... Figure 3 The process of stacking residual blocks is existing technology and will not be elaborated here. In the residual block, downsampling is performed using a spatial channel module with ReLU6 as the activation function. The spatial channel module cuts the feature map's feature layers along the channel direction and then stitches them together along the spatial direction to obtain the cut and stitched feature map. The cutting formula is:
[0067] f 0，0 =X[0: S: scale, 0: S: scale], f 1，0 =X[1:S:scale,0:S:scale],...,f scale-1，0
[0068] =X[scale-1:S:scale,0:S:scale];
[0069] f0，1 =X[0: S: scale, 1: S: scale], f 1，1 , ..., f scale-1，1
[0070] =X[scale-1:S:scale,1:S:scale];
[0071] f 0，scale-1 =X[0: S: scale, scale-1: S: scale], f 1，scale-1 , ..., f scale-1，scale-1
[0072] =X[scale-1: S: scale, scale-1: S: scale].
[0073] =X[scale-1: S: scale, scale-1: S: scale].
[0074] Where X represents the feature map of the input residual block, S represents the length or width of the feature map of the input residual block, and f x，y This represents the subplot after downsampling, and scale represents the downsampling scaling factor.
[0075] The formula for the nonlinear activation function ReLU6 is given below, which is existing technology and will not be elaborated upon here:
[0076] f(c) = min{max(0, c), 6}
[0077] Where c is the input from the previous layer of the neural network;
[0078] The input channel is 64, the number of superimposed residual blocks is 1, and the number of channels is changed to 128, resulting in an output feature map M2 with 128 channels and 240×240 pixels;
[0079] S214. Input feature map M2 into the hollow spatial pyramid pooling module to obtain feature map M2′. Input feature map M2′ into the residual block, use the spatial channel module to downsample, stack 2 residual blocks, and change the number of channels to 256. Output feature map M3 with a pixel size of 120×120 and a number of channels of 256.
[0080] The feature map M3 is input into the hollow spatial pyramid pooling module to obtain the feature map M3′. The feature map M3 is then input into the residual block, and the spatial channel module is used for downsampling. Four residual blocks are superimposed, and the number of channels is changed to 512. The output feature map M4 has a pixel size of 60×60 and a number of channels of 512.
[0081] The feature map M4 is input into the hollow spatial pyramid pooling module to obtain the feature map M4′. The feature map M4 is then input into the residual block, downsampled using the spatial channel module, and the number of channels is changed to 1024. The output feature map M5 has a pixel size of 30×30 and a number of channels of 1024.
[0082] S215. Input feature map M5 into the hollow spatial pyramid pooling module and change the number of channels to 512, outputting feature map M6 with a pixel size of 30×30 and a number of channels of 512. This method avoids the loss of fine-grained features, enhances the feature extraction capability of small cracks and low-resolution images, increases the receptive field of the convolution kernel, and strengthens the connection between crack feature maps at various scales.
[0083] In this embodiment, in step S2, the decoder module in the SRA-Net model is constructed and decoded using the following method:
[0084] Construct a decoder, which includes convolutional kernels, a concat concatenation module, and a bilinear interpolation upsampling module;
[0085] Decode the feature map:
[0086] S221. Perform bilinear interpolation upsampling on the feature map M6 output by the encoder to obtain a feature map M7 with 512 channels;
[0087] S222. By using a skip connection method, the feature map M4 with 512 channels in the encoder and the feature map M7 with 512 channels in the decoder are concat-joined to achieve feature fusion;
[0088] S223. Input the image after Concat stitching into two 3×3 convolution kernels for convolution, change the number of channels to 256, and output feature map M7′;
[0089] S224. Perform bilinear interpolation upsampling on feature map M7′ to obtain feature map M8 with 256 channels;
[0090] S225. Concat the feature map M3 with 256 channels and the feature map M8 with 256 channels to achieve feature fusion. Then, input the fused image into two 3×3 convolution kernels for convolution, change the number of channels to 128, and output the feature map M8′. Perform bilinear interpolation upsampling on the feature map M8′ to obtain the feature map M9 with 128 channels.
[0091] Concat the feature map M2 with 128 channels and the feature map M9 with 128 channels to achieve feature fusion. Then, input the fused image into two 3×3 convolution kernels for convolution. Change the number of channels to 64 and output the feature map M9′. Perform bilinear interpolation upsampling on the feature map M9′ to obtain the feature map M10 with 64 channels.
[0092] S226. Concat the 64-channel feature map M1 and the 64-channel feature map M10 to achieve feature fusion. Then, input the fused image into two 3×3 convolution kernels for convolution to obtain a 64-channel feature map M10′. Pass the feature map M10′ through a 1×1 convolution kernel and change the number of channels to 2. Output an image that achieves the segmentation of crack pixels from background pixels. This method ensures the continuity of grayscale values after sampling, resulting in a higher-quality scaled image.
[0093] In this embodiment, in step S3, the SRA-Net model is trained using the following method:
[0094] The dataset collected in step S1 is input into the SRA-Net model for training. The initial learning rate is set to 0.01 and the impulse is 0.9. An optimizer and loss function are built based on the PyTorch deep learning library. Weight decay is used to prevent overfitting. The SDG optimizer is used to iteratively update the weights. The optimization process of the SDG optimizer is an existing technology and will not be described in detail here. The loss function is the sum of cross-entropy loss and dice loss.
[0095] The loss function formula is:
[0096]
[0097] Where Loss represents the loss function, and i represents a pixel in the image. y represents the predicted value of a pixel. i The label value represents the pixel, and n represents the total number of pixels. Using the method described above, the model can be trained quickly and the weights optimized.
[0098] In this embodiment, in step S4, the dataset is used to validate each epoch, and the DICE function is calculated to determine if it is the model with the highest DICE function. If yes, training is complete, and the process proceeds to step S5. If not, the parameters of the SRA-Net model are updated, and the process returns to step S3 to continue training until the model has the highest DICE function. This method ensures that the trained model has the best performance and the smallest error.
[0099] In this embodiment, in step S5, the test data is input into the SRA-Net model with the highest dice function, and the segmentation result image is output. This method avoids the loss of fine-grained features in traditional convolution and pooling downsampling, enhances the model's feature extraction capabilities for small cracks and low-resolution images, improves the connection between various feature layers, and enables SRA-Net to achieve better segmentation results.
[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A pavement crack segmentation method based on SRA-Net, characterized in that: Includes the following steps: S1. Obtain a dataset of road surface crack image samples; S2. Construct an SRA-Net model based on an encoder-decoder structure, and add a void space pyramid pooling module and a space channel module to the encoder; The encoder module in the SRA-Net model is constructed and encoded using the following method: An encoder is constructed, which includes a convolutional kernel, residual blocks, and a holed spatial pyramid module. The residual blocks use a spatial channel module for downsampling. Encode the image: S211. Construct a 1×1 convolution kernel for the input image, input the RGB image into the residual block, change the number of channels in the residual block, and output feature map M1; S212. Add a dilated spatial pyramid pooling (ASPP) module after the residual block to perform dilated convolution sampling on the input feature map; S213. After the hollow spatial pyramid pooling ASPP module, a residual block SPD-Res Block is added again. The residual block uses the spatial channel module for downsampling. Depending on the input channel, different numbers of residual blocks are superimposed on the input feature map, and the number of channels is changed to output feature map M2. S214. Repeat steps S212 and S213 three times each, and output feature map M3, feature map M4 and feature map M5 respectively; S215. Input feature map M5 into the hole space pyramid pooling module ASPP, change the number of channels, and output feature map M6; The decoder module in the SRA-Net model is constructed and decoded using the following method: Construct a decoder, which includes convolutional kernels, a concat concatenation module, and a bilinear interpolation upsampling module; Decode the feature map: S221. Perform bilinear interpolation upsampling on the feature map M6 output by the encoder to obtain the feature map M7 in the decoder; S222. By using skip connections, the feature map output by the decoder is concat-joined with the M1-M4 feature maps of the encoder, which have the same number of channels, to achieve feature fusion; S223. Pass the image after Concat stitching through two 3×3 convolution kernels to change the number of channels to one-quarter of the number of channels after stitching; S224. Perform bilinear interpolation upsampling on the image obtained in step S223 and output feature map M8; S225. Repeat steps S222-S224 twice in sequence, and output feature map M9 and feature map M10 respectively; wherein, during the repetition, the decoder feature map concatenated with the encoder is derived from the feature map output by the previous layer after bilinear interpolation. S226. Concat the feature map M1 and the feature map M10, and pass the concatenated image through two 3×3 convolution kernels to obtain an image with 64 channels. Pass the image with 64 channels through a 1×1 convolution kernel and change the number of channels to 2. Output the image that realizes the segmentation of crack pixels and background pixels. S3. Input the dataset into the SRA-Net model for training; train the SRA-Net model using the following method: The dataset collected in step S1 is input into the SRA-Net model for training. The initial learning rate is set to 0.01 and the impulse is 0.
9. Weight decay is used to prevent overfitting. The SDG optimizer is used to iteratively update the weights. The loss function is the sum of cross-entropy loss and Dice loss. The loss function formula is: Where Loss represents the loss function, and i represents a pixel in the image. y represents the predicted value of a pixel. i The label value represents the pixel, and n represents the total number of pixels; S4. Determine whether the SRA-Net model has been trained. If yes, proceed to step S5. If no, update the parameters of the SRA-Net model and return to step S3. S5. Input the test data into the trained SRA-Net model and output the segmentation result image.
2. The pavement crack segmentation method based on SRA-Net according to claim 1, characterized in that: Hollow spatial pyramid pooling uses dilated convolutions with different sampling rates to sample in parallel, and then adds the sampling results to obtain a more accurate feature map.
3. The pavement crack segmentation method based on SRA-Net according to claim 1, characterized in that: In the residual block, spatial channel modules are used for downsampling, and the nonlinear activation function ReLU6 is used as the activation function. The spatial channel module segments the feature layers of the input residual block's feature map along the channel direction and then stitches them together along the spatial direction to obtain the segmented and stitched feature map. The segmentation formula is as follows: f 0,0 =X[0:S:scale,0:S:scale],f 1,0 =X[1:S:scale,0:S:scale],…,f scale-1,0 =X[scale-1:S:scale,0:S:scale]; f 0,1 =X[0:S:scale,1:S:scale],f 1,1 ,…,f scale-1,1 =X[scale-1:S:scale,1:S:scale]; ... f 0,scale-1 =X[0:S:scale,scale-1:S:scale],f 1,scale-1 ,…,f scale-1,scale-1 =X[scale-1:S:scale,scale-1:S:scale]. Where X represents the feature map of the input residual block, S represents the length or width of the feature map of the input residual block, and f x,y This represents the subplot after downsampling, and scale represents the downsampling scaling factor. The formula for the nonlinear activation function ReLU6 is: f(c) = min{max(0,c), 6} Where c is the input from the previous layer of the neural network.
4. The pavement crack segmentation method based on SRA-Net according to claim 1, characterized in that: In step S4, the following method is used to determine whether the SRA-Net model has completed training: For each epoch, the dataset is used for validation, and the DICE function is calculated to determine if it is the model with the highest DICE function. If it is, the training is complete, and the process proceeds to step S5. If not, the parameters of the SRA-Net model are updated, and the process returns to step S3 to continue training until the model has the highest DICE function.
Citation Information
Patent Citations
Building change detection method in urban video shooting
CN113033454A
Device, process and system for diagnosing and tracking of the development of the spinal alignment of a person
US20220254018A1