Concrete crack identification method and device based on dynamic convolution

Abstract

The invention discloses a concrete crack identification method and device based on dynamic convolution, and relates to the technical field of defect detection, the method better adapts to a slender crack segmentation task by introducing the dynamic convolution, adopts a residual error structure design, prevents the information loss problem in the dynamic convolution process, and improves the detection accuracy. Further improvement of target feature extraction by the model is realized, a space channel double-attention module is introduced, the problem that low-order information is not fully utilized by the model is solved, different channels or dimensions of input features are weighted to highlight importance of some channels, and different positions in an input feature space are weighted to highlight importance of some channels; according to the method, the concrete crack recognition model is introduced to highlight the importance of some positions, a cavity space convolution module is introduced, a convolution kernel receptive field is expanded, crack features are extracted from a global angle, the recognition precision of the concrete crack recognition model is improved, and the accuracy of concrete crack recognition is further improved. The problem that concrete crack identification is inaccurate in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a concrete crack recognition method and device based on dynamic convolution. Background Art

[0002] Concrete structures may age, crack, deform or even be damaged under the influence of factors such as environment, load and construction level. Cracks are a very common structural damage defect in concrete structures. Regular inspection and maintenance of engineering structures are of great significance to improving the safety and durability of the structures.

[0003] Traditional crack detection usually relies on manual on-site detection. Although on-site detection has high accuracy and a simple process, the detection efficiency is low, and some locations that cannot be reached by personnel may be missed, leaving corresponding safety hazards.

[0004] With the advancement of science and technology, digital image detection technology and artificial intelligence (deep learning models) have been applied to crack detection tasks. However, in order to meet the needs of upstream and downstream work, it is not enough to simply detect the existence of cracks. Therefore, using a semantic segmentation model based on pixel classification to obtain information such as crack morphology has become a new research. However, the new technology also brings corresponding problems: cracks are different from ordinary target features. Cracks occupy fewer pixels in the image, and the shape is mostly slender structures, and the crack area in the image is uncertain. In order to improve the accuracy of segmentation, researchers began to focus on the segmentation task of small targets; the actual crack image acquisition environment is complex and easily affected by factors such as posture, lighting and background, which puts higher requirements on the segmentation model.

[0005] Existing crack segmentation models are all improved based on mainstream segmentation models, such as UNet, PSPNet, Deeplabv3+, and HRNet. These models have good performance in general target segmentation tasks, but the segmentation accuracy is not ideal when it comes to segmenting small targets such as cracks. These models use traditional convolution kernels to extract target feature information, but cracks may have different shapes, widths, directions, and textures. Traditional convolution kernels may not be able to adapt well to this diversity, which affects the feature extraction of cracks and causes inaccurate crack identification. Summary of the invention

[0006] In view of this, an object of the present invention is to provide a concrete crack identification method and device based on dynamic convolution, aiming to solve the problem of inaccurate concrete crack identification in the prior art.

[0007] The embodiment of the present invention is implemented as follows: A concrete crack recognition method based on dynamic convolution, which recognizes concrete cracks based on a concrete crack recognition model obtained by training a preset neural network, wherein the preset neural network includes an encoder, a decoder, a dynamic residual convolution module, a spatial channel dual attention module, and a void space convolution module, and the method includes: Performing a unified data preprocessing operation on the images in the original data set including concrete crack images and normal images; Input the training images used for training in the original data set to the first layer of the dynamic residual convolution module in the encoder, and input the feature map output by the dynamic residual convolution module of the previous layer to the dynamic residual convolution module of the next layer after downsampling, wherein each of the dynamic residual convolution modules outputs the feature map extracted and merged by its own dynamic convolution kernel; Inputting the feature maps output by the dynamic residual convolution modules of the first three layers in the encoder into the spatial channel dual attention modules respectively to obtain respective attention weight feature maps; Input the feature map output by the dynamic residual convolution module of the last layer into the hole space convolution module to obtain a hole space feature map; Input the hole space feature map into the dynamic residual convolution module of the first layer in the decoder, and upsample the target feature map output from the dynamic residual convolution module of the previous layer in the decoder, merge it with the corresponding attention weight feature map, and then input it into the dynamic residual convolution module of the current layer for output; The final feature map output by the dynamic residual convolution module of the last layer of the decoder is used to calculate the overall loss with the original label map corresponding to the original data set, and the concrete crack recognition model is obtained by back propagation training until the model converges or reaches a preset training round; A concrete image to be identified is acquired, and the concrete image to be identified is input into the concrete crack identification model for identification to obtain an identification result of the concrete image to be identified.

[0008] Furthermore, in the above concrete crack recognition method based on dynamic convolution, the step of performing a unified data preprocessing operation on the images in the original data set containing the concrete crack images and the normal images comprises: The images in the original data set are scaled, randomly cropped, and randomly horizontally flipped.

[0009] Furthermore, in the above-mentioned concrete crack recognition method based on dynamic convolution, the step of each of the dynamic residual convolution modules outputting their own dynamic convolution kernels to extract the merged feature map comprises: Step 1: After the image is input into the dynamic residual convolution module, three convolution operations are performed to extract features and obtain feature maps. , feature map , feature map ; Among them, 3×3 convolution kernel is used for feature extraction and then passes through normalization layer and activation function, and dynamic convolution kernel is used for feature extraction in both x-axis and y-axis directions respectively; Step 2: The feature map in step 1 , feature map , feature map Perform concatenation to obtain feature maps , the feature map After reducing the number of channels through convolution, the residual convolution is realized by adding the feature map to the input image, and then the feature map output by the dynamic residual convolution module is obtained through an activation function.

[0010] Furthermore, in the above-mentioned concrete crack recognition method based on dynamic convolution, the step of inputting the feature maps output by the dynamic residual convolution modules of the first three layers in the encoder into the spatial channel dual attention modules respectively to obtain respective attention weight feature maps includes: The spatial channel dual attention module performs maximum pooling and average pooling on the feature map output from the dynamic residual convolution module in the spatial dimension, and then passes through a multi-layer perceptron with shared weights to obtain two output results; The output results are added element by element, and then passed through an activation function to obtain a channel attention weight, and the channel attention weight and the feature map are multiplied element by element to obtain a channel attention feature map; The spatial channel dual attention module performs maximum pooling and average pooling on the feature map output from the dynamic residual convolution module in the channel dimension, concatenates the two output results based on the channel, performs convolution operation and activation function on them, obtains the spatial attention weight, and performs element-by-element multiplication of the spatial attention weight and the feature map to obtain the spatial attention feature map: The channel attention feature map and the spatial attention feature map are added element by element to obtain the attention weight feature map.

[0011] Furthermore, in the above-mentioned concrete crack recognition method based on dynamic convolution, the step of inputting the feature map output by the dynamic residual convolution module of the last layer into the void space convolution module to obtain the void space feature map includes: Extracting features from the feature map through three atrous convolution blocks with dilation rates of 1, 2, and 5 respectively connected in a densely connected manner in the atrous spatial convolution module to obtain a first feature map, wherein each of the atrous convolution blocks includes a 3×3 convolution layer, a normalization layer, and an activation function; The feature map is subjected to an average pooling layer, then to a 1×1 convolution layer, and finally to an upsampling operation to return the feature map to its original size to obtain a second feature map; The first feature map and the second feature map are connected and then a convolution operation is performed to obtain the hole space feature map.

[0012] Furthermore, in the above-mentioned concrete crack recognition method based on dynamic convolution, the step of calculating the overall loss between the final feature map output by the dynamic residual convolution module of the last layer of the decoder and the corresponding original label map in the original data set comprises: Get the sum of the bidirectional Hausdorff distances in all dimensions between the point set of the original label map and the point set of the final feature map; A loss function is obtained by adding the sum of the bidirectional Hausdorff distances to the cross entropy loss function, and the overall loss is calculated based on the loss function.

[0013] Furthermore, in the above concrete crack recognition method based on dynamic convolution, the calculation formula of the bidirectional Hausdorff distance is: ; ; ; in, Represent the point set of the final feature map and the point set of the original label map respectively, Represent the points in different feature maps respectively. represents the bidirectional Hausdorff distance.

[0014] Another object of the present invention is to provide a concrete crack recognition device based on dynamic convolution, which recognizes concrete cracks based on a concrete crack recognition model obtained by training a preset neural network, wherein the preset neural network includes an encoder, a decoder, a dynamic residual convolution module, a spatial channel dual attention module, and a void space convolution module, and the device includes: A preprocessing module, used for performing a unified data preprocessing operation on images in an original data set including concrete crack images and normal images; A convolution module, used for inputting the training images used for training in the original data set into the first layer of the dynamic residual convolution module in the encoder, and inputting the feature map output by the dynamic residual convolution module of the previous layer into the dynamic residual convolution module of the next layer after downsampling, wherein each of the dynamic residual convolution modules outputs the feature map extracted and merged by its own dynamic convolution kernel; An attention module, used for inputting the feature maps output by the dynamic residual convolution modules of the first three layers in the encoder into the spatial channel dual attention modules respectively to obtain respective attention weight feature maps; A hole module, used for inputting the feature map output by the dynamic residual convolution module of the last layer into the hole space convolution module to obtain a hole space feature map; A merging module, used to input the hole space feature map into the dynamic residual convolution module of the first layer in the decoder, and merge the target feature map output from the dynamic residual convolution module of the previous layer in the decoder with the corresponding attention weight feature map after upsampling, and then input it into the dynamic residual convolution module of the current layer for output; A training module, used for calculating the overall loss between the final feature map output by the dynamic residual convolution module of the last layer of the decoder and the original label map corresponding to the original data set, and obtaining the concrete crack recognition model through back propagation training until the model converges or reaches a preset training round; The recognition module is used to obtain the concrete image to be recognized, and input the concrete image to be recognized into the concrete crack recognition model for recognition to obtain the recognition result of the concrete image to be recognized.

[0015] Another object of the present invention is to provide a readable storage medium having a computer program stored thereon, wherein the program implements the steps of the above method when executed by a processor.

[0016] Another object of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0017] The present invention can better adapt to the segmentation task of slender crack targets by introducing dynamic convolution. The residual structure design can prevent the problem of information loss in the dynamic convolution process, realize the further improvement of the model in extracting target features, introduce the spatial channel dual attention module, make up for the problem of insufficient utilization of low-order information by the model, on the one hand, weight the different channels or dimensions of the input features to highlight the importance of some channels, on the other hand, weight the different positions in the input feature space to highlight the importance of some positions, introduce the hollow space convolution module, expand the convolution kernel receptive field, extract crack features from a global perspective, improve the recognition accuracy of the concrete crack recognition model, and then improve the accuracy of concrete crack recognition. The problem of inaccuracy in concrete crack recognition in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG1 is a schematic diagram of the structure of a preset neural network in a concrete crack recognition method based on dynamic convolution in one embodiment of the present invention; Figure 2 It is a schematic diagram of the flow of a concrete crack identification method based on dynamic convolution in the first embodiment of the present invention; Figure 3 Schematic diagram of the structure of a dynamic residual convolution model in a concrete crack identification method based on dynamic convolution in one embodiment of the present invention.

[0019] Figure 4 Schematic diagram of the structure of a spatial channel dual attention module in a concrete crack recognition method based on dynamic convolution in one embodiment of the present invention; Figure 5 Schematic diagram of the structure of a void space convolution module in a concrete crack recognition method based on dynamic convolution in one embodiment of the present invention; Figure 6 It is a structural block diagram of a concrete crack identification device based on dynamic convolution in the second embodiment of the present invention.

[0020] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0021] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0022] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0024] The following embodiments can be applied to Figure 1 Among the preset neural networks shown, a concrete crack recognition model is obtained by training the preset neural network, and the concrete crack recognition model is used to identify concrete cracks. Specifically, the preset neural network includes an encoder, a decoder, a dynamic residual convolution module, a spatial channel dual attention module, and a void space convolution module.

[0025] How to improve the accuracy of concrete crack identification will be described in detail below with reference to specific embodiments and drawings.

[0026] Embodiment 1 See also Figure 2 , which shows a concrete crack identification method based on dynamic convolution in the first embodiment of the present invention, and the method includes steps S10 to S16.

[0027] Step S10, performing a unified data preprocessing operation on the images in the original data set including the concrete crack images and the normal images.

[0028] The original data set is used for model training. Specifically, the original data set includes images of concrete with cracks, i.e., concrete crack images, and images of concrete without cracks, i.e., concrete normal images. The images in the original data set are uniformly preprocessed to ensure that the model obtains more diverse and uniformly sized input images. Specifically, the preprocessing operations include: image scaling, random cropping, and random horizontal flipping operations; image scaling ensures that the image size input to the model is consistent, and other operations are used to ensure the diversity of the input images.

[0029] Step S11, input the training images used for training in the original data set into the first layer of the dynamic residual convolution module in the encoder, and input the feature map output by the dynamic residual convolution module of the previous layer into the dynamic residual convolution module of the next layer after downsampling, wherein each of the dynamic residual convolution modules outputs the feature map extracted and merged by its own dynamic convolution kernel.

[0030] Among them, there are multiple dynamic residual convolution modules in the encoder. First, the training image is input to the first dynamic residual convolution module and then output, and then the output of the previous dynamic residual convolution module is used as the input of the next dynamic residual convolution module. In each dynamic residual convolution module, after receiving the input image, the image will be feature extracted, and finally the corresponding dynamic convolution kernel will be output to extract the merged feature map.

[0031] Specifically, Figure 3 As shown in the figure, after the image is input into the dynamic residual convolution module, three convolution operations are performed to extract features and obtain feature maps. , feature map , feature map ; The top layer is a 3×3 convolution kernel for feature extraction and passes through a normalization layer and an activation function. The next two layers are dynamic convolution kernels for feature extraction in the x-axis and y-axis directions respectively; the dynamic convolution kernel stretches the traditional convolution kernel in the x-axis and y-axis directions respectively. Specifically, given a standard 2D convolution coordinate , the center coordinates are A 3×3 convolution kernel It is expressed as: ; by Taking the axis direction as an example, The specific position of each grid in is expressed as: , where c represents the horizontal distance from the center grid, c = 0,1,2,3,4..., n, the convolution kernel Each grid position The selection is a cumulative process. Initially, the position of the grid away from the center depends on the position of the previous grid: Relative to The offset ∆={δ|δ ∈[-1,1]} is added. Therefore, the offset needs to be accumulated Σ to ensure that the convolution kernel conforms to the linear morphology structure.

[0032] The formula for the x-axis direction is:

[0033] The formula for the y-axis direction is:

[0034] Finally, the above feature map , feature map , feature map Perform concatenation operation to obtain feature map , feature map After reducing the number of channels through convolution, the residual convolution is realized by adding the feature map to the input image, and then the feature map output by the dynamic residual convolution module is obtained through the activation function. Specifically: ; ; in, W for n×n The weights of the convolution, 、 Respectively x and y Axis direction convolution weights, σ represents the activation function, Represents a splicing operation, is the feature map output by the dynamic residual convolution module, F is the feature map of the input image.

[0035] Among them, a dynamic convolution kernel is used to achieve more accurate extraction of crack features, and a residual structure is used to prevent the loss of some feature information caused by dynamic convolution.

[0036] Step S12, inputting the feature maps output by the dynamic residual convolution modules of the first three layers in the encoder into the spatial channel dual attention modules respectively to obtain respective attention weight feature maps.

[0037] Among them, the spatial channel dual attention module improves the model's perception ability in space and channel of feature maps respectively. Figure 4 As shown in the figure, the spatial channel dual attention module performs maximum pooling and average pooling on the feature map output from the dynamic residual convolution module in the spatial dimension, and then passes through a multi-layer perceptron with shared weights to obtain two output results. The output results are added element by element and then passed through an activation function, for example sigmoid Activation function, get the channel attention weight, multiply the channel attention weight and feature map element by element to get the channel attention feature map; specifically, the formula of channel attention weight is as follows:

[0038] More specifically, MLP It consists of two layers of convolution operations. After the first layer of convolution, ReLUActivation function, used after the second convolution layer sigmoid The activation function obtains the output weight, and then multiplies each channel by its corresponding weight. The formula for this step is as follows: The formula for obtaining the channel attention feature map by performing element-by-element multiplication of the channel attention weight and the feature map is: ; in, represents the channel attention weight, Represents a feature map.

[0039] More specifically, the spatial channel dual attention module performs maximum pooling and average pooling on the feature maps output from the dynamic residual convolution module in the channel dimension, concatenates the two output results based on the channel, and then performs convolution operations and sigmoid The activation function is used to obtain a spatial attention weight with a channel number of 1 and a feature map size that is the same as the input. The spatial attention weight and the feature map are element-wise multiplied to obtain the spatial attention feature map.

[0040] The formula for this step is as follows: ; ; in, σ express sigmoid Activation function, Indicates that the convolution kernel size is 3×3, [] indicates the concatenation operation, Represents a feature map.

[0041] Finally, the channel attention feature map and the spatial attention feature map are added element by element to obtain the attention weight feature map. Specifically, .

[0042] Step S13, inputting the feature map output by the dynamic residual convolution module of the last layer into the hole space convolution module to obtain a hole space feature map.

[0043] Among them, the dilated spatial convolution module can extract crack features by expanding the receptive field of the convolution kernel without increasing the number of parameters through unperturbed convolution. Specifically, Figure 5 As shown in the figure, the first feature map is obtained by extracting the feature map using three dilation rates of 1, 2, and 5 connected in a dense connection mode in the dilated spatial convolution module. Each convolution block contains a 3×3 convolution layer, a normalization layer (BN), and an activation function (Relu). The formula for this step is as follows:

[0044]

[0045]

[0046] in, , , They represent three convolutions with dilation rates of 1, 2, and 5 respectively. represents the relu activation function, BN represents normalization, [] represents the concatenation operation, represents the feature map input to the dilated spatial convolution module, ， ， Represents the output features of three convolutional blocks.

[0047] Next, the feature map is ASPP Pooling Specifically, the strategy is to perform an average pooling layer, then a 1×1 convolution layer, and finally an upsampling operation to return the feature map to its original size to obtain the second feature map. The formula for this step is as follows: ; in, express ASPP Pooling Output features, represents upsampling, W represents 1×1 convolution, is average pooling.

[0048] Finally, the first feature map and the second feature map are connected and convolved to obtain the hole space feature map; it is expressed as:

[0049] Among them, W represents a 3×3 convolution.

[0050] Step S14, input the hole space feature map into the dynamic residual convolution module of the first layer in the decoder, and upsample the target feature map output from the dynamic residual convolution module of the previous layer in the decoder, merge it with the corresponding attention weight feature map, and then input it into the dynamic residual convolution module of the current layer for output.

[0051] Among them, the obtained hole space feature map is input into the dynamic residual convolution module of the first layer in the decoder, and the target feature map output from the dynamic residual convolution module of the previous layer in the decoder is upsampled and merged with the corresponding attention weight feature map and then input into the dynamic residual convolution module of the current layer for output, until the decoder finally outputs the final feature map.

[0052] Step S15, calculating the overall loss between the final feature map output by the dynamic residual convolution module of the last layer of the decoder and the corresponding original label map in the original data set, and obtaining the concrete crack recognition model through back propagation training until the model converges or reaches a preset training round.

[0053] Among them, the sum of the bidirectional Hausdorff distances between the point set of the original label map and the point set of the final feature map in all dimensions is obtained; The loss function is obtained by adding the sum of the bidirectional Hausdorff distance and the cross entropy loss function, and the overall loss is calculated based on the loss function. Specifically, the topological continuity constraint loss is used. Hausdorff Distance is used to measure the similarity between the label feature map point set and the predicted feature map point set. The formula is:

[0054]

[0055]

[0056] in, Represent the point set of the final feature map and the point set of the original label map respectively, Represent the points in different feature maps respectively. represents the bidirectional Hausdorff distance.

[0057] Summing over all dimensions (n=0,1,2,···,N) yields , With cross entropy loss The final loss function is integrated and can be expressed as: .

[0058] Step S16, obtaining a concrete image to be identified, and inputting the concrete image to be identified into the concrete crack identification model for identification to obtain an identification result of the concrete image to be identified.

[0059] Among them, the concrete crack recognition model masters the recognition logic of concrete cracks, so it can accurately identify whether there are cracks in the concrete reflected by the concrete image and the shape of the cracks.

[0060] In summary, the concrete crack recognition method based on dynamic convolution in the above embodiment of the present invention can better adapt to the segmentation task of slender crack targets by introducing dynamic convolution. The residual structure design can prevent the problem of information loss in the dynamic convolution process, and further improve the model's target feature extraction. The spatial channel dual attention module is introduced to make up for the problem of insufficient use of low-order information by the model. On the one hand, different channels or dimensions of the input features are weighted to highlight the importance of some channels. On the other hand, different positions in the input feature space are weighted to highlight the importance of some positions. The hollow space convolution module is introduced to expand the receptive field of the convolution kernel, extract crack features from a global perspective, and improve the recognition accuracy of the concrete crack recognition model, thereby improving the accuracy of concrete crack recognition. The problem of inaccuracy in concrete crack recognition in the prior art is solved.

[0061] Embodiment 2 See also Figure 6 , shown is a concrete crack recognition device based on dynamic convolution proposed in the second embodiment of the present invention, which recognizes concrete cracks based on a concrete crack recognition model obtained by training a preset neural network, wherein the preset neural network includes an encoder, a decoder, a dynamic residual convolution module, a spatial channel dual attention module, and a void space convolution module, and the device includes: The device comprises: A preprocessing module 100 is used to perform a unified data preprocessing operation on the images in the original data set including the concrete crack image and the normal image; The convolution module 200 is used to input the training images used for training in the original data set into the first layer of the dynamic residual convolution module in the encoder, and input the feature map output by the dynamic residual convolution module of the previous layer into the dynamic residual convolution module of the next layer after downsampling, wherein each of the dynamic residual convolution modules outputs the feature map extracted and merged by its own dynamic convolution kernel; An attention module 300, configured to input the feature maps output by the dynamic residual convolution modules of the first three layers of the encoder into the spatial channel dual attention modules respectively to obtain respective attention weight feature maps; The hole module 400 is used to input the feature map output by the dynamic residual convolution module of the last layer into the hole space convolution module to obtain a hole space feature map; A merging module 500 is used to input the hole space feature map into the dynamic residual convolution module of the first layer in the decoder, and merge the target feature map output from the dynamic residual convolution module of the previous layer in the decoder with the corresponding attention weight feature map after upsampling, and then input it into the dynamic residual convolution module of the current layer for output; A training module 600 is used to calculate the overall loss between the final feature map output by the dynamic residual convolution module of the last layer of the decoder and the original label map corresponding to the original data set, and obtain the concrete crack recognition model through back propagation training until the model converges or reaches a preset training round; The recognition module 700 is used to obtain a concrete image to be recognized, and input the concrete image to be recognized into the concrete crack recognition model for recognition to obtain a recognition result of the concrete image to be recognized.

[0062] Furthermore, in the above-mentioned concrete crack recognition device based on dynamic convolution, the step of performing a unified data preprocessing operation on the images in the original data set containing the concrete crack images and the normal images comprises: The images in the original data set are scaled, randomly cropped, and randomly horizontally flipped.

[0063] Furthermore, in the above-mentioned concrete crack identification device based on dynamic convolution, the step of each of the dynamic residual convolution modules outputting their own dynamic convolution kernels to extract the merged feature map comprises: Step 1: After the image is input into the dynamic residual convolution module, three convolution operations are performed to extract features and obtain feature maps. , feature map , feature map ; Among them, 3×3 convolution kernel is used for feature extraction and then passes through normalization layer and activation function, and dynamic convolution kernel is used for feature extraction in both x-axis and y-axis directions respectively; Step 2: The feature map in step 1 , feature map , feature map Perform concatenation operation to obtain feature map , the feature map After reducing the number of channels through convolution, the residual convolution is realized by adding the feature map to the input image, and then the feature map output by the dynamic residual convolution module is obtained through an activation function.

[0064] Furthermore, in the above-mentioned concrete crack recognition device based on dynamic convolution, the step of inputting the feature maps output by the dynamic residual convolution modules of the first three layers in the encoder into the spatial channel dual attention modules respectively to obtain respective attention weight feature maps includes: The spatial channel dual attention module performs maximum pooling and average pooling on the feature map output from the dynamic residual convolution module in the spatial dimension, and then passes through a multi-layer perceptron with shared weights to obtain two output results; The output results are added element by element, and then passed through an activation function to obtain a channel attention weight, and the channel attention weight and the feature map are multiplied element by element to obtain a channel attention feature map; The spatial channel dual attention module performs maximum pooling and average pooling on the feature map output from the dynamic residual convolution module in the channel dimension, concatenates the two output results based on the channel, performs convolution operation and activation function on them, obtains the spatial attention weight, and performs element-by-element multiplication of the spatial attention weight and the feature map to obtain the spatial attention feature map: The channel attention feature map and the spatial attention feature map are added element by element to obtain the attention weight feature map.

[0065] Furthermore, in the above-mentioned concrete crack identification device based on dynamic convolution, the step of inputting the feature map output by the dynamic residual convolution module of the last layer into the void space convolution module to obtain the void space feature map comprises: Extracting features from the feature map through three atrous convolution blocks with dilation rates of 1, 2, and 5 respectively connected in a densely connected manner in the atrous spatial convolution module to obtain a first feature map, wherein each of the atrous convolution blocks includes a 3×3 convolution layer, a normalization layer, and an activation function; The feature map is subjected to an average pooling layer, then to a 1×1 convolution layer, and finally to an upsampling operation to return the feature map to its original size to obtain a second feature map; The first feature map and the second feature map are connected and then a convolution operation is performed to obtain the hole space feature map.

[0066] Furthermore, in the above-mentioned concrete crack recognition device based on dynamic convolution, the step of calculating the overall loss between the final feature map output by the dynamic residual convolution module of the last layer of the decoder and the corresponding original label map in the original data set comprises: Get the sum of the bidirectional Hausdorff distances in all dimensions between the point set of the original label map and the point set of the final feature map; A loss function is obtained by adding the sum of the bidirectional Hausdorff distances to the cross entropy loss function, and the overall loss is calculated based on the loss function.

[0067] Furthermore, in the above-mentioned concrete crack identification device based on dynamic convolution, the calculation formula of the bidirectional Hausdorff distance is: ; ; ; in, Represent the point set of the final feature map and the point set of the original label map respectively, Represent the points in different feature maps respectively. represents the bidirectional Hausdorff distance.

[0068] The functions or operation steps implemented when the above modules are executed are substantially the same as those in the above method embodiments, and will not be repeated here.

[0069] Embodiment 3 Another aspect of the present invention further provides a readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method described in the first embodiment above.

[0070] Embodiment 4 Another aspect of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first embodiment when executing the program.

[0071] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0073] More specific examples (a non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0074] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0076] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A concrete crack recognition method based on dynamic convolution, characterized in that: Concrete cracks are identified based on a concrete crack recognition model obtained by training a preset neural network, wherein the preset neural network includes an encoder, a decoder, a dynamic residual convolution module, a spatial channel dual attention module, and a void space convolution module. The method includes: Performing a unified data preprocessing operation on the images in the original data set including concrete crack images and normal images; Input the training images used for training in the original data set to the first layer of the dynamic residual convolution module in the encoder, and input the feature map output by the dynamic residual convolution module of the previous layer to the dynamic residual convolution module of the next layer after downsampling, wherein each of the dynamic residual convolution modules outputs the feature map extracted and merged by its own dynamic convolution kernel; Inputting the feature maps output by the dynamic residual convolution modules of the first three layers in the encoder into the spatial channel dual attention modules respectively to obtain respective attention weight feature maps; Input the feature map output by the dynamic residual convolution module of the last layer into the hole space convolution module to obtain a hole space feature map; Input the hole space feature map into the dynamic residual convolution module of the first layer in the decoder, and upsample the target feature map output from the dynamic residual convolution module of the previous layer in the decoder, merge it with the corresponding attention weight feature map, and then input it into the dynamic residual convolution module of the current layer for output; The final feature map output by the dynamic residual convolution module of the last layer of the decoder is used to calculate the overall loss with the original label map corresponding to the original data set, and the concrete crack recognition model is obtained by back propagation training until the model converges or reaches a preset training round; A concrete image to be identified is acquired, and the concrete image to be identified is input into the concrete crack identification model for identification to obtain an identification result of the concrete image to be identified.

2. The concrete crack identification method based on dynamic convolution according to claim 1 is characterized in that: The step of performing a unified data preprocessing operation on the images in the original data set including the concrete crack images and the normal images comprises: The images in the original data set are scaled, randomly cropped, and randomly horizontally flipped.

3. The concrete crack identification method based on dynamic convolution according to claim 1 is characterized in that: The step of each of the dynamic residual convolution modules outputting their own dynamic convolution kernels to extract the merged feature maps comprises: Step 1: After the image is input into the dynamic residual convolution module, three convolution operations are performed to extract features and obtain feature maps. , feature map , feature map ; Among them, 3×3 convolution kernel is used for feature extraction and then passes through normalization layer and activation function, and dynamic convolution kernel is used for feature extraction in both x-axis and y-axis directions respectively; Step 2: The feature map in step 1 , feature map , feature map Perform concatenation operation to obtain feature map , the feature map After reducing the number of channels through convolution, the residual convolution is realized by adding the feature map to the input image, and then the feature map output by the dynamic residual convolution module is obtained through an activation function.

4. The method for concrete crack identification based on dynamic convolution according to claim 1, characterized in that: The step of inputting the feature maps output by the dynamic residual convolution modules of the first three layers of the encoder into the spatial channel dual attention modules to obtain respective attention weight feature maps comprises: The spatial channel dual attention module performs maximum pooling and average pooling on the feature map output from the dynamic residual convolution module in the spatial dimension, and then passes through a multi-layer perceptron with shared weights to obtain two output results; The output results are added element by element, and then passed through an activation function to obtain a channel attention weight, and the channel attention weight and the feature map are multiplied element by element to obtain a channel attention feature map; The spatial channel dual attention module performs maximum pooling and average pooling on the feature map output from the dynamic residual convolution module in the channel dimension, concatenates the two output results based on the channel, performs convolution operation and activation function on them, obtains the spatial attention weight, and performs element-by-element multiplication of the spatial attention weight and the feature map to obtain the spatial attention feature map: The channel attention feature map and the spatial attention feature map are added element by element to obtain the attention weight feature map.

5. The method for concrete crack identification based on dynamic convolution according to claim 1, characterized in that: The step of inputting the feature map output by the dynamic residual convolution module of the last layer into the hole space convolution module to obtain the hole space feature map comprises: Extracting features from the feature map through three atrous convolution blocks with dilation rates of 1, 2, and 5 respectively connected in a densely connected manner in the atrous spatial convolution module to obtain a first feature map, wherein each of the atrous convolution blocks includes a 3×3 convolution layer, a normalization layer, and an activation function; The feature map is subjected to an average pooling layer, then to a 1×1 convolution layer, and finally to an upsampling operation to return the feature map to its original size to obtain a second feature map; The first feature map and the second feature map are connected and then a convolution operation is performed to obtain the hole space feature map.

6. The method for concrete crack identification based on dynamic convolution according to claim 1, characterized in that: The step of calculating the overall loss of the final feature map output by the dynamic residual convolution module of the last layer of the decoder and the corresponding original label map in the original data set comprises: Get the sum of the bidirectional Hausdorff distances in all dimensions between the point set of the original label map and the point set of the final feature map; A loss function is obtained by adding the sum of the bidirectional Hausdorff distances to the cross entropy loss function, and the overall loss is calculated based on the loss function.

7. The method for concrete crack identification based on dynamic convolution according to claim 6 is characterized in that: The calculation formula of the bidirectional Hausdorff distance is: ； ； ； in, Represent the point set of the final feature map and the point set of the original label map respectively, Represent the points in different feature maps respectively. represents the bidirectional Hausdorff distance.

8. A concrete crack identification device based on dynamic convolution, characterized in that: Concrete cracks are identified based on a concrete crack recognition model obtained by training a preset neural network, wherein the preset neural network includes an encoder, a decoder, a dynamic residual convolution module, a spatial channel dual attention module, and a void space convolution module, and the device includes: A preprocessing module, used for performing a unified data preprocessing operation on images in an original data set including concrete crack images and normal images; A convolution module, used for inputting the training images used for training in the original data set into the first layer of the dynamic residual convolution module in the encoder, and inputting the feature map output by the dynamic residual convolution module of the previous layer into the dynamic residual convolution module of the next layer after downsampling, wherein each of the dynamic residual convolution modules outputs the feature map extracted and merged by its own dynamic convolution kernel; An attention module, used for inputting the feature maps output by the dynamic residual convolution modules of the first three layers in the encoder into the spatial channel dual attention modules respectively to obtain respective attention weight feature maps; A hole module, used for inputting the feature map output by the dynamic residual convolution module of the last layer into the hole space convolution module to obtain a hole space feature map; A merging module, used to input the hole space feature map into the dynamic residual convolution module of the first layer in the decoder, and merge the target feature map output from the dynamic residual convolution module of the previous layer in the decoder with the corresponding attention weight feature map after upsampling, and then input it into the dynamic residual convolution module of the current layer for output; A training module, used for calculating the overall loss between the final feature map output by the dynamic residual convolution module of the last layer of the decoder and the original label map corresponding to the original data set, and obtaining the concrete crack recognition model through back propagation training until the model converges or reaches a preset training round; The recognition module is used to obtain the concrete image to be recognized, and input the concrete image to be recognized into the concrete crack recognition model for recognition to obtain the recognition result of the concrete image to be recognized.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the program.

Images