An image crack detection method based on group normalization and shift

Abstract

The present invention discloses a method for detecting cracks in an image based on group normalization and shifting. The method is based on an attention mechanism of channel group normalization to enhance crack features. Each layer of a symmetrical U-shaped network uses a group normalization method to normalize features in the entire network architecture. A new group normalization attention is designed using learnable parameters of group normalization to better adapt to the entire network architecture. Group normalization compresses features into vectors from a spatial dimension and normalizes them by channel grouping. After the group normalization operation is completed, linear changes are used to compensate for the loss in the normalization and compression process, and linearly changing learnable parameters are used to weight the original feature map from the channel level, thereby achieving the effect of suppressing meaningless features and highlighting meaningful features. In addition to being able to cope with the interference of various complex situations in natural scenes, the method of the present invention can also better refine the detection of small cracks, and the model has high robustness and generalization.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and more particularly, relates to an image crack detection method based on group normalization and shifting. Background Art

[0002] During construction and operation, infrastructure such as bridges and highways inevitably suffer structural or non-structural damage due to factors such as construction materials, construction techniques, extreme weather, and vehicle overload. This damage can lead to surface defects such as honeycombs, missing reinforcement, holes, and cracks. Therefore, regular inspections and condition assessments of these infrastructures are essential to ensure their safety and practicality. Traditional manual inspection methods are subject to high subjectivity, time consumption, and cost. While many algorithms have achieved promising results in recent years, they still suffer from certain flaws. For example, the presence of numerous distracting information such as lane arrows on road surfaces can reduce model performance due to environmental interference, resulting in a lack of generalization and robustness. Furthermore, segmentation of fine-texture features can be discontinuous and noisy.

[0003] Currently, the mainstream crack detection method uses deep learning algorithms, which can achieve higher accuracy when computing resources are sufficient. Crack segmentation algorithms based on deep convolutional neural networks primarily improve crack segmentation performance from two perspectives: one approach is to design a network structure specifically for crack detection, and the other is to design a module specifically tailored to crack characteristics and embed it into the existing network structure.

[0004] While it's easy to improve crack segmentation performance by designing the network structure, such a design often results in an extremely bloated model. While performance improves, the speed is reduced, and training the model requires significant computational resources. Due to its simplicity and efficiency, the attention mechanism based on convolutional neural networks has become increasingly popular in the crack detection field. The SE network first compresses the feature map along the spatial dimension, then applies two linear transformations to the resulting feature vector to produce a feature weight vector representing the importance of each feature channel, with larger values ​​indicating the lowest importance. Subsequently, various variants of attention mechanisms have emerged, such as CBAM and BAM, that combine channel-wise and spatial attention. These attention mechanisms have also been applied to crack detection networks, further improving their performance without incurring excessive computational overhead.

[0005] While many excellent crack segmentation algorithms have been mentioned above, they still have many shortcomings. For example, when designing an efficient new network, many factors must be considered, such as efficiency, accuracy, and applicability. Self-designed crack segmentation algorithms are often not pre-trained on ImageNet, resulting in slightly lower performance. Furthermore, as the network depth and number of channels increase, the network speed decreases. For example, DeepCrack, while supervised training of deep networks is an excellent approach, it can also significantly slow network training and inference.

[0006] Furthermore, previously designed networks often incorporate attention simply by combining existing networks or modules, without considering the applicability to crack segmentation tasks, such as background interference, light interference, or noise interference. If the designed model simply combines existing structures and modules, not only will it be unsuitable for crack segmentation tasks, resulting in low accuracy, but also poor robustness and generalization due to the diverse surface features of civil structures.

[0007] Based on this, it is urgent to design an image crack detection method based on group normalization and shift. Summary of the Invention

[0008] (1) Technical issues

[0009] Based on the above-mentioned technical defects, the present invention provides an image crack detection method based on group normalization and shifting. In addition to being able to cope with the interference of various complex situations in natural scenes, this method can also better refine the detection of small cracks, and the model has high robustness and generalization.

[0010] (2) Technical solution

[0011] The present invention provides an image crack detection method based on group normalization and shifting, which improves the original SegNet network and specifically includes the following steps:

[0012] Step 1: Convolution is used to extract features from cracks in the image. The encoder in the network is divided into five parts. The feature outputs of each convolution layer in each stage are used for splicing, and then convolution is used to reduce the dimension as the output of the entire stage. Downsampling uses maximum pooling. During maximum pooling, the index information of the downsampled pixels is stored. The decoder stage uses the saved index in the encoder stage, and the pixels are restored to the position indicated by the index during upsampling decoding. The residual structure is used in the encoder stage, but not in the decoder stage.

[0013] Step 2: Divide the C-dimensional channel into N groups denoted as {g1, g2, ... g N}, the channel of each group is function Indicates that x is rounded down; each group calculates the average value μ of the weights in C' channels i and variance σ i , x j Represents the j-th channel in the feature map x:

[0014]

[0015] For each group g i , whose group normalization operation GN i (t) is shown in formula (2). All weight vectors are normalized in groups. After the weights on each channel are normalized within the group, a linear transformation is performed. The coefficient of each channel is γ t and β t , where subscript t∈{1,2,3...C'}, function GN i (t) represents the vector g for the i-th group i Perform group planning operations:

[0016]

[0017] Step 3: The group normalized attention module is defined as formula (3), where x represents the input feature map, the functions GAP() and GMP() represent global average pooling and global maximum pooling, respectively, which are used to compress the spatial dimension features into a vector, and ⊙ represents the channel-by-channel multiplication operation:

[0018] f(x)=Sigmoid(γ)⊙(GN(GAP(x))+GN(GMP(x)))⊙x (9)

[0019] Among them, f(x) is the characteristic function after weighting attention, and γ is the channel coefficient γ t Sigmoid() is the S-type growth curve function.

[0020] Preferably, the following steps 4 to 6 are also included:

[0021] Step 4: First, divide the feature map into S groups according to the channel, and take S as an odd number; each group of feature maps is moved d pixels along the x-axis, where d∈[-pad,pad], After the movement, point convolution is performed to map the features that are not aligned in spatial dimensions to the space of the same dimension. After the point convolution is performed, the feature map is reversely moved to align the feature map to avoid misleading the model.

[0022] Step 5: Use deep convolution to perceive crack features at longer distances and extract features of small cracks;

[0023] Step 6: Repeat the operation of step 4 along the y-axis direction. First, group the channels, then move d pixels along the y-axis after grouping the feature maps, perform point convolution mapping to the space of the same dimension, and finally restore the feature map by inverse shifting operation.

[0024] Preferably, the method further comprises the following steps 7:

[0025] Step 7: In the decoder, each layer uses bilinear interpolation to upsample the predicted feature map. The upsampled feature map is concatenated with the feature map of the same size output by the encoder stage. The upsampling uses the maximum unpooling operation and reuses the downsampling index saved in the encoder stage to ensure that the features are effectively restored to a specific position. Finally, a single-channel probability map is output.

[0026] Preferably, the method further comprises the following steps 8:

[0027] Step 8: The loss calculation between the predicted probability map y and the true value mask y is shown in Equations (4)-(6). The model finally outputs a single-channel probability map:

[0028]

[0029]

[0030] Loss total (y,y)= Loss BCE (y,y)+Loss Dice (y,y) (12).

[0031] Preferably, step 1 further includes:

[0032] In the encoder, the basic block of each stage is encoding block-k, which is a structure composed of k basic block convolution-group normalization-GELU activation function CGG. Encoding block-2 represents a module formed by 2 CGG blocks in series, and encoding block-3 is composed of 3 CGG blocks. The encoder uses 2 encoding blocks-2 and 3 encoding blocks-3 in series. Before the CGG block, a CBG is used to double the number of channels. CBG is composed of basic block convolution-batch normalization-GELU activation function. After completing the feature extraction of the CGG block, the CBG block is used to reduce the feature dimension to the required dimension.

[0033] Preferably, step 1 further includes:

[0034] In the decoder, the basic block of the decoder stage is "decoding block-k", and the core extraction part is also composed of CGG blocks. "Decoding block-2" and "decoding block-3" are composed of 2 and 3 CGG blocks respectively. In the decoder, 2 decoding blocks-2 and 3 decoding blocks-3 are used in series; in "decoding block-k", the inverse maximum pooling operation is first performed, and the index information stored in the encoder stage is used to restore the position of the pixels during upsampling; also after the inverse maximum pooling operation, CBG is used to reduce the number of channels of the feature map, and then CGG is used to extract features. When outputting, the CBG block is used to adjust the feature dimension to the dimension required by the present invention. Unlike the "encoding block-k" block, the "decoding block-k" block does not use residual connection.

[0035] In another aspect, the present invention further discloses an image crack detection system based on group normalization and shifting, comprising:

[0036] at least one processor; and

[0037] at least one memory in communication with the processor, wherein:

[0038] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute any of the above-mentioned image crack detection methods based on group normalization and shifting.

[0039] In another aspect, the present invention further discloses a non-transitory computer-readable storage medium storing computer instructions, which enable the computer to execute the image crack detection method based on group normalization and shifting as described in any one of the above items.

[0040] (3) Beneficial effects

[0041] Compared with existing technologies, the image crack detection method based on group normalization and shifting has the following advantages:

[0042] (1) Compared with the original SegNet network model, the present invention proposes an image crack detection method based on group normalization attention of the improved SegNet semantic segmentation model, which has the following advantages: First, in order to cope with the interference of various complex situations in natural scenes, the present invention proposes an attention mechanism based on group normalization. Group normalization compresses features into vectors from the spatial dimension and normalizes them by channel grouping; and uses linearly changing learnable parameters in group normalization to weight them from the channel level, thereby achieving the effect of suppressing meaningless features and highlighting meaningful features. Secondly, in order to better detect small cracks, the present invention proposes a "fill-move-cut" convolution module. By shifting all feature maps by different pixel values ​​in the horizontal and vertical directions, the information of each part of the feature map can pay attention to each other after the pixel shift, and the model can obtain crack information at a longer distance during learning.

[0043] (2) In addition, the present invention proposes a group-normalized attention method and a "fill-shift-cut" convolution, and the two can complement each other. The group-normalized attention method GNAM improves the original SE module to make it suitable for crack characteristics. First, the feature is compressed from the spatial dimension through maximum pooling and average pooling to become a vector with only channel dimensions. Then, the vector is group-normalized in the channel dimension. In order to compensate for the information loss in the feature compression and normalization process, a linear change is performed after the group normalization, and the learnable parameters of the linear change are multiplied channel by channel with the normalized feature vector as the weight vector of the attention mechanism, which acts on the original feature map. Considering the problem of compensating for the small receptive field of the model under limited depth, based on the designed "fill-shift-cut" convolution, by filling and shifting the feature map, the information at each position in the feature map can be noticed by each other, which can greatly improve the receptive field of the model and overcome the problem of small cracks lost due to the large expansion rate of the dilated convolution. In addition, the Dice loss slightly modified by the BCE loss based on the loss function can better mine the foreground object. The present invention can achieve good results on the CrackForestDataset, DeepCrack, CrackTree260 and CrackLS315 datasets. In particular, on the CrackTree260 dataset, the present invention achieves the best performance in various indicators, which proves the superior performance of the method proposed in the present invention in image crack detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is an overview diagram of the encoder-decoder model in the image crack detection method of the present invention;

[0045] Figure 2 This is the flow chart of the attention module based on group normalization;

[0046] Figure 3 This is the flow chart of the "fill-shift-cut" convolution;

[0047] Figure 4 It is a flow chart of feature changes in the "fill-shift-cut" convolution;

[0048] Figure 5 This is a performance comparison chart of the method of the present invention and other methods on CrackForestDataset;

[0049] Figure 6 The performance comparison of the method of the present invention and other methods on CrackTree260 is shown;

[0050] Figure 7 This is a performance comparison chart of the method of the present invention and other methods on CrackLS315;

[0051] Figure 8 This is a performance comparison chart of the method of the present invention and other methods on DeepCrack;

[0052] Figure 9 This is a comparison chart of the prediction effects of the method of the present invention and other methods on the CrackForestDataset dataset and other models; from top to bottom, they are the RAW original image, the GT true value mask image, the effect diagram predicted by the HED model, the effect diagram predicted by the RCF model, the effect diagram predicted by the SegNet model, the effect diagram predicted by the DeepCrack model, and the effect diagram predicted by the present invention.

[0053] Figure 10 This is a comparison chart of the prediction effects of the method of the present invention and other methods on the CrackTree260 dataset and other models, from top to bottom: the RAW original image, the GT true value mask image, the effect chart predicted by the HED model, the effect chart predicted by the RCF model, the effect chart predicted by the SegNet model, the effect chart predicted by the DeepCrack model, and the effect chart predicted by the present invention.

[0054] Figure 11 This is a comparison chart of the prediction effects of the method of the present invention and other methods on the CrackLS315 dataset and other models, from top to bottom: the RAW original image, the GT true value mask image, the effect graph predicted by the HED model, the effect graph predicted by the RCF model, the effect graph predicted by the SegNet model, the effect graph predicted by the DeepCrack model, and the effect graph predicted by the present invention.

[0055] Figure 12This is a comparison chart of the prediction effects of the method of the present invention and other methods on the DeepCrack dataset, including the RAW original image, the GT true value mask image, the effect chart predicted by the HED model, the effect chart predicted by the RCF model, the effect chart predicted by the SegNet model, the effect chart predicted by the DeepCrack model, and the effect chart predicted by the present invention.

[0056] Figure 13 (a) is the PR curve of this method and other methods on CrackForestDataset; (b) is the PR curve of this method and other methods on CrackTree260; (c) is the PR curve of this method and other methods on CrackLS315; (d) is the PR curve of this method and other methods on DeepCrack. DETAILED DESCRIPTION

[0057] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0058] In order to solve the problems of poor anti-interference ability, insufficient crack detection accuracy, low robustness and generalization of the model, the network structure of the present invention is improved on the SegNet network in the prior art. Figure 1 As shown in FIG, the encoder-decoder model in the new SegNet network of the present invention specifically includes the following improvements:

[0059] (a) In the encoder, the basic block at each stage is "Encoding Block-k," a CGG structure consisting of k basic blocks: convolution (Conv), group normalization (GN), and GELU activation. "Encoding Block-2" represents a module consisting of two CGG blocks connected in series, while Encoding Block-3 consists of three CGG blocks. Before the CGG block, a CBG is used to double the number of channels. CBG consists of the basic blocks: convolution (Conv), batch normalization (BN), and GELU activation. This is because as the network deepens, the present invention needs to increase the number of channels to obtain richer semantic information. After completing the feature extraction of the CGG block, the present invention uses the CBG block to reduce the feature dimension to the dimension required by the present invention. The present invention uses the maximum pooling with a step size of 2 to reduce the resolution, and retains the pixel position index information during downsampling, and restores the pixel information to the index position during upsampling. The present invention connects the feature maps output by all CGG blocks together to obtain richer information, and in order to further enrich the obtained feature maps, the residual structure proposed in the ResNet network is adopted in the "coding block-k" block, which can prevent the gradient from disappearing during the network training process.

[0060] (b) In the decoder, the present invention adopts a structure similar to that of the "Encoding Block-k". The basic block of the decoder stage is the "Decoding Block-k", and the core extraction part is also composed of CGG blocks. The "Decoding Block-2" and "Decoding Block-3" are composed of 2 and 3 CGG blocks, respectively. In the "Decoding Block-k", the present invention first performs an inverse maximum pooling operation, and uses the index information stored in the encoder stage to restore the position of the pixels during upsampling. Also after the inverse maximum pooling operation, the present invention uses CBG to reduce the number of channels of the feature map, and then extracts features through CGG. At output, the CBG block is used to adjust the feature dimension to the dimension required by the present invention. Unlike the "Encoding Block-k" block, the "Decoding Block-k" block does not use residual connections because, in the decoder stage, the element-by-element addition of feature maps from different stages will introduce noise and reduce the performance of the model.

[0061] From this we can see that Figure 1 The network results are designed with a special embedding module for problems such as crack characteristics and anti-interference. The network is divided into several parts: encoder, decoder, and residual connection. In the encoder part, the present invention adopts a pyramid structure similar to the residual network for feature extraction, and the entire network implements five downsamplings. The features of each stage will fuse the outputs of all the convolutional layers of that stage, so that the richest semantic information can be obtained in the last layer. The use of the residual structure in the encoder stage can also make up for the semantic information lost during the maximum pooling downsampling to the greatest extent. The residual structure is not used in the decoder stage because the residual structure will introduce some noise, resulting in a large amount of noise in the output mask image. In order to eliminate the noise introduced by the residual structure in the encoder stage and the various noise interferences that may exist in the crack image itself, the present invention also proposes an attention module based on group normalization. Group normalization first compresses the spatial dimension of the input image into a vector with only the channel dimension left, and then normalizes these vectors in groups. In order to compensate for the information loss during normalization and spatial compression, a linear change is performed on the normalized vector. The final vector obtained is the attention weight. The weight represents the importance of the information of each channel in the feature map. Furthermore, at the end of the encoder feature extraction phase, to further expand the receptive field and enhance the ability to capture fine cracks, this paper proposes a convolution based on "padding-shifting-cutting." First, a certain number of pixels are padded around the feature map. Then, feature maps from different channels are shifted vertically and horizontally to allow information at different locations in the feature maps to be mutually noticeable.

[0062] In another embodiment, the present invention also proposes an image crack detection method based on group normalization and shifting, which improves the existing SegNet network and specifically includes the following steps:

[0063] 1. Feature Extraction Downsampling

[0064] Step 1: For crack information in the image, convolution is used to extract features. The encoder in the network is divided into five parts. The feature output of each convolution layer in each stage is used for splicing, and then the dimensionality is reduced through convolution as the output of the entire stage to carry richer semantic information when the final encoding is completed; downsampling adopts maximum pooling. During maximum pooling, the index information of the downsampled pixels is stored. The saved index of the encoder stage is used in the decoder stage. During upsampling decoding, the pixels are restored to the position indicated by the index, retaining more semantic information; in addition, the residual structure is used in the encoder stage, but not in the decoder stage.

[0065] Specifically, in step 1, the network structure of the present invention may be preferably Figure 1 The network structure shown in the figure can also be a modified SegNet network that only uses a residual structure in the encoder stage. This is because the features at each stage in this network fuse the outputs of all convolutional layers in that stage, thus obtaining the richest semantic information in the final layer. The use of a residual structure in the encoder stage can also maximize the semantic information lost during max pooling downsampling. The residual structure is not used in the decoder stage because it introduces some noise, resulting in a noisy mask output.

[0066] 2. Attention calculation based on group normalization

[0067] Step 2: Divide the C-dimensional channel into N groups denoted as {g1, g2, ... g N}(N=32 can be set as the default value), the channels of each group are function Indicates that x is rounded down; each group calculates the average value μ of the weights in C' channels i and variance σ i , x j Represents the j-th channel in the feature map x:

[0068]

[0069] For each group g i , whose group normalization operation GN i (t) is shown in formula (2). All weight vectors are normalized in groups. After the weights on each channel are normalized within the group, a linear transformation is performed to compensate for the possible information loss during the network learning process. The coefficient of each channel is γ t and β t , where subscript t∈{1,2,3...C'}, function GN i (t) represents the vector g for the i-th group iPerform group planning operations:

[0070]

[0071] Step 3: The group normalized attention module is defined as formula (3), where x represents the input feature map, the functions GAP() and GMP() represent global average pooling and global maximum pooling, respectively, which are used to compress the spatial dimension features into a vector, and ⊙ represents the channel-by-channel multiplication operation, which is used to multiply the calculated weights by the corresponding feature map and feature vector.

[0072] f(x)=Sigmoid(γ)⊙(GN(GAP(x))+GN(GMP(x)))⊙x (15)

[0073] Among them, f(x) is the characteristic function after weighting attention, and γ is the channel coefficient γ t Sigmoid() is an S-shaped growth curve function. In information science, due to its monotonic and inverse monotonic properties, the Sigmoid function is often used as an activation function of neural networks to map variables to between 0 and 1.

[0074] like Figure 2 As shown above, the channel-based group normalization attention mechanism used in steps 1-3 above is used to enhance crack features. Each layer of the symmetrical U-shaped network uses group normalization to normalize features throughout the entire network architecture. A new group normalization attention mechanism is designed using learnable parameters for group normalization to better adapt to the entire network architecture. Group normalization compresses features into vectors from the spatial dimension and normalizes them by channel group. After group normalization, linear transformations are used to compensate for losses during normalization and compression, and linearly varying learnable parameters are used to weight the original feature maps at the channel level, thereby suppressing meaningless features and highlighting meaningful ones. This eliminates the noise introduced by the residual structure used in the encoder stage and various noise interferences inherent in the crack image itself, enhancing the anti-interference performance of crack detection.

[0075] 3. “Pad-Shift-Cut” Convolution

[0076] Step 4: First, divide the feature map into S groups according to the channel. Considering the symmetry during translation, take S as an odd number (preferably S = 5); each group of feature maps is moved along the x-axis by d pixels, where d∈[-pad,pad], After the movement, since the feature maps are not aligned in space, point convolution is performed to map the features that are not aligned in spatial dimensions to the space of the same dimension. After the point convolution is performed, the feature maps are reversely moved to align the feature maps to avoid misleading the model.

[0077] Step 5: Use deep convolution to perceive crack features at longer distances and extract features of small cracks.

[0078] Step 6: Repeat the operation of step 4 along the y-axis. First, group the channels, then move d pixels along the y-axis after grouping the feature maps, perform point convolution to map them to the space of the same dimension, and finally restore the feature map by inverse shifting.

[0079] like Figure 3-4 As shown, through the resource-friendly "fill-shift-cut" convolution operation of steps 4 to 6 above, in semantic segmentation, a simple and effective dilated convolution is often used to increase the receptive field. However, for relatively fine crack features, when the dilation rate is large, the intervals between each learnable parameter in the convolution kernel are large, and the neighboring information of the pixels around a specific pixel will be lost, resulting in the loss of the fine pixel information. The "fill-shift-cut" convolution of the present invention shifts the feature map channels by different pixel values ​​in the horizontal and vertical directions, so that the pixel information at different positions in the feature map can pay attention to each other to obtain a larger receptive field. The "fill-shift-cut" convolution operation not only does not lose the information of fine cracks like the dilated convolution, but also because the pixels at each position in each channel of the shifted feature map are not aligned, the convolution can also extract the cracks that the model really needs to pay attention to in the non-aligned feature map, thereby improving the robustness of the model, and ultimately overcoming the problem of dilated convolution losing fine cracks due to the large dilation rate and improving the detection accuracy of cracks.

[0080] 4. Decoder-side predicted image mask

[0081] Step 7: In the decoder, each layer uses bilinear interpolation to upsample the predicted feature map. The upsampled feature map is concatenated with the feature map of the same size output by the encoder stage. The upsampling uses the maximum unpooling operation and reuses the downsampling index saved in the encoder stage to ensure that the features are effectively restored to a specific position. Finally, a single-channel probability map is output.

[0082] 5. Loss function calculation

[0083] Step 8: The loss calculation between the predicted probability map y and the true value mask y is shown in Equations (4)-(6), and the model finally outputs a single-channel probability map.

[0084]

[0085]

[0086] Loss total (y,y)= Loss BCE (y,y)+Loss Dice(y,y) (18)

[0087] From step 7, it can be seen that formulas (4)-(5) correspond to the calculation of BCEWithLogits loss and Dice loss respectively. BCE calculates the loss between the output probability result and the binary mask, guiding the model to learn the semantic features of the crack, and Dice guides the model to learn the edge features of the crack more finely. Usually, Dice loss only calculates the loss of the foreground, but due to the extreme imbalance of the foreground and background information of the crack feature, in order to solve this problem, the present invention slightly modifies the loss of the Dice model, adds a penalty term for the background, and simultaneously trains and learns from the foreground and background constraint model. In this way, the problem of uneven distribution of the foreground and background of the model can be alleviated. The calculation of formula (6) that comprehensively considers the BCEWithLogits loss and Dice loss can more accurately calculate the probability of the crack category.

[0088] It should be pointed out that step 8 is obviously not a necessary step in the method of the present invention. It is only for making the trained network more robust.

[0089] In addition, in order to verify the performance of the proposed method based on group normalized attention and the “fill-shift-cut” convolutional crack detection network, the proposed method is verified on four public datasets: CrackForest, DeepCrack, CrackTree260 and CrackLS315. Figure 5 The performance comparison of the method of the present invention and other methods on CrackForest is shown; Figure 6 The performance comparison between the proposed method and other methods on the CrackTree260 dataset is shown; Figure 7 The performance comparison of the proposed method with other methods on CrackLS315 is shown; Figure 8 The performance comparison of the method of the present invention and other methods on DeepCrack is demonstrated. Figure 9 The comparison of segmentation results of different methods on CFD datasets is shown; Figure 10 Shows the comparison of segmentation results of different methods on the CrackTree260 dataset; Figure 11 Shows the comparison of segmentation results of different methods on the CrackLS315 dataset; Figure 12 The comparison of segmentation results of different methods on the DeepCrack dataset is shown. Figure 13 The PR curve effects of the method of the present invention and other methods on four data sets are demonstrated.

[0090] This demonstrates that the detection method and network of the present invention represent a novel U-shaped crack segmentation network. The improved SegNet network achieves superior Precision, Recall, F1-score, mIoU, and IoU results on the public datasets CrackTree260, DeepCrack, CFD, and CrackLS315. Notably, the present model achieves 1.27% and 0.94% higher mean intersection over union (mIoU) and F1-score, respectively, than the DeepCrack model on the CrackTree260 dataset.

[0091] The above-mentioned image crack detection method based on group normalization and shifting of the present invention can be converted into software program instructions, which can be implemented by running a software analysis system including a processor and a memory, or can be implemented by computer instructions stored in a non-transitory computer-readable storage medium.

[0092] Finally, the method of the present invention is only a preferred embodiment and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image crack detection method based on group normalization and shifting, characterized in that: This method improves the original SegNet network and includes the following steps: Step 1: Convolution is used to extract features from cracks in the image. The encoder in the network is divided into five parts. The feature outputs of each convolution layer in each stage are used for splicing, and then convolution is used to reduce the dimension as the output of the entire stage. Downsampling uses maximum pooling. During maximum pooling, the index information of the downsampled pixels is stored. The decoder stage uses the saved index in the encoder stage, and the pixels are restored to the position indicated by the index during upsampling decoding. The residual structure is used in the encoder stage, but not in the decoder stage. Step 2: Divide the C-dimensional channel into N groups denoted as {g1, g2, ... g N }, the channel of each group is function Indicates that x is rounded down; each group calculates the average value μ of the weights in C' channels i and variance σ i , x j Represents the j-th channel in the feature map x: For each group g i , whose group normalization operation GN i (t) is shown in formula (2). All weight vectors are normalized in groups. After the weights on each channel are normalized within the group, a linear transformation is performed. The coefficient of each channel is γ t and β t , where subscript t∈{1,2,3...C'}, function GN i (t) represents the vector g for the i-th group i Perform group planning operations: Step 3: The group normalized attention module is defined as formula (3), where x represents the input feature map, the functions GAP() and GMP() represent global average pooling and global maximum pooling, respectively, which are used to compress the spatial dimension features into a vector, and ⊙ represents the channel-by-channel multiplication operation: f(x)=Sigmoid(γ)⊙(GN(GAP(x))+GN(GMP(x)))⊙x (3) Among them, f(x) is the characteristic function after weighting attention, and γ is the channel coefficient γ t Sigmoid() is the S-type growth curve function.

2. The image crack detection method based on group normalization and shifting according to claim 1, characterized in that: Also includes the following steps 4-6: Step 4: First, divide the feature map into S groups according to the channel, and take S as an odd number; each group of feature maps is moved d pixels along the x-axis, where d∈[-pad,pad], After the movement, point convolution is performed to map the features that are not aligned in spatial dimensions to the space of the same dimension. After the point convolution is performed, the feature map is reversely moved to align the feature map to avoid misleading the model. Step 5: Use deep convolution to perceive crack features at longer distances and extract features of small cracks; Step 6: Repeat the operation of step 4 along the y-axis direction. First, group the channels, then move d pixels along the y-axis after grouping the feature maps, perform point convolution mapping to the space of the same dimension, and finally restore the feature map by inverse shifting operation.

3. The image crack detection method based on group normalization and shifting according to claim 2, characterized in that: Also includes the following step 7: Step 7: In the decoder, each layer uses bilinear interpolation to upsample the predicted feature map. The upsampled feature map is concatenated with the feature map of the same size output by the encoder stage. The upsampling uses the maximum unpooling operation and reuses the downsampling index saved in the encoder stage to ensure that the features are effectively restored to a specific position. Finally, a single-channel probability map is output.

4. The image crack detection method based on group normalization and shifting according to claim 3, characterized in that: Also includes the following step 8: Step 8: The loss calculation between the predicted probability map y and the true value mask y is shown in Equations (4)-(6). The model finally outputs a single-channel probability map; Loss total (y,y)= Loss BCE (y,y)+Loss Dice (y,y) (6)。 5. The image crack detection method based on group normalization and shifting according to claim 4, characterized in that: Also included in step 1: In the encoder, the basic block of each stage is encoding block-k, which is a structure composed of k basic block convolution-group normalization-GELU activation function CGG. Encoding block-2 represents a module formed by 2 CGG blocks in series, and encoding block-3 is composed of 3 CGG blocks. The encoder uses 2 encoding blocks-2 and 3 encoding blocks-3 in series. Before the CGG block, a CBG is used to double the number of channels. CBG is composed of basic block convolution-batch normalization-GELU activation function. After completing the feature extraction of the CGG block, the CBG block is used to reduce the feature dimension to the required dimension.

6. The image crack detection method based on group normalization and shifting according to claim 5, characterized in that: Also included in step 1: In the decoder, the basic block of the decoder stage is "Decoding Block-k", and the core extraction part is also composed of CGG blocks. "Decoding Block-2" and "Decoding Block-3" are composed of 2 and 3 CGG blocks, respectively. In the decoder, 2 Decoding Block-2 and 3 Decoding Block-3 are connected in series. In "Decoding Block-k", the inverse maximum pooling operation is first performed, and the index information stored in the encoder stage is used to restore the pixel positions during upsampling. After the inverse maximum pooling operation, CBG is used to reduce the number of channels of the feature map. CGG is then used to extract features, and the CBG block is used at output to adjust the feature dimension to the required dimension. Unlike the "Encoding Block-k" block, the "Decoding Block-k" block does not use residual connections.

7. An image crack detection system based on group normalization and shifting, characterized in that: include: at least one processor; as well as at least one memory in communication with the processor, wherein: The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the image crack detection method based on group normalization and shifting according to any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the image crack detection method based on group normalization and shifting according to any one of claims 1 to 6.

Images