Surface crack detection method based on multi-scale topology-aware neural network

Abstract

The present invention belongs to the technical field of semantic segmentation methods for surface crack images, and specifically is a surface crack detection method based on a multi-scale topological perception neural network. The method comprises the following steps: step one: dividing the surface crack data set into a training set and a test set, and preprocessing the images in the training set and the test set respectively; step two: building a semantic segmentation network for surface crack images; step three: inputting the preprocessed surface crack images of the training set in step one into the surface crack semantic segmentation network in step two for training, first initializing the surface crack segmentation network using the He Uniform method, then updating the parameters in the model, and optimizing the loss function until convergence; step four: inputting the preprocessed test set remote sensing images into the surface crack segmentation network trained in step three, and outputting the segmentation results. The present invention solves the problem of poor feature representation capability in existing crack target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of semantic segmentation methods for surface crack images, and specifically is a surface crack detection method based on a multi-scale topology perception neural network. Background Art

[0002] With the development of social economy, the scale of infrastructure such as buildings, bridges, roads and railways is getting larger and larger. Maintaining good operating conditions is crucial for the normal use of these facilities and public safety. Surface cracks are not only one of the early manifestations of facility quality degradation, but also further accelerate the quality degradation of facilities by destroying the integrity and continuity of the surface structure of the facilities. Timely and accurate detection of the initiation and expansion of surface cracks can effectively extend the service life of facilities and avoid major catastrophic accidents. Therefore, surface crack detection has important application value in the fields of transportation, construction, and shipping.

[0003] Early surface crack detection mainly relied on manual inspection, which was not only time-consuming and laborious, but also had highly subjective results. In order to improve the detection efficiency, automatic surface crack detection methods based on computer vision technologies such as threshold segmentation, edge detection and region growth have been continuously proposed, and have achieved certain application results. However, these technologies rely heavily on various artificially designed surface crack detection and recognition features, cannot be applied to complex scenes, have poor generalization performance, and are easily affected by various noise interferences.

[0004] The fully convolutional neural network based on deep learning can automatically extract different levels of identification features from surface crack images through multi-layer network structure and nonlinear transformation, and has strong automatic feature expression ability. In addition, the end-to-end, pixel-to-pixel implementation of the fully convolutional neural network can provide pixel-level surface crack recognition and positioning and facilitate further morphological measurement and analysis. It is one of the most promising surface crack automatic detection methods. However, the fully convolutional neural network currently used for surface crack detection mainly focuses on pixel classification accuracy, cannot accurately perceive the topological structure of the crack, and the edge and detail information is lost in the process of semantic segmentation, which seriously affects the accuracy of the results. In addition, the fully convolutional model often relies on a large number of high-precision training samples or pre-trained models trained by a large number of samples of daily scenes. However, pixel-level annotation of surface crack images is often time-consuming and laborious. At the same time, due to the large difference between daily scene images and surface crack images, the segmentation accuracy of the pre-trained model trained by daily scene images is often unsatisfactory. In addition, the huge number of parameters of these complex models puts forward high requirements on storage and computing devices, and training and application models are also very time-consuming. These defects greatly limit the practical application of these methods. Summary of the invention

[0005] In order to comprehensively solve the problems that existing crack detection methods cannot accurately perceive topological structures, there is a large loss of edge and detail information during semantic segmentation, the model is inefficient and training is difficult, the present invention provides a surface crack detection method based on a multi-scale topology perception neural network.

[0006] The present invention adopts the following technical scheme: a surface crack image semantic segmentation network, comprising a backbone part I, a backbone network, a channel attention module, a spatial context module and a backbone part II; the backbone part I outputs an input image as a feature map of a quarter of the spatial resolution of the input image through wavelet transform; the backbone network is composed of four stages and a feature aggregation module in sequence, each stage includes a group of multi-resolution group convolutions and multi-resolution convolutions, the feature aggregation module unifies the resolution of feature maps output by all streams through bilinear interpolation and aggregates the outputs of different streams into feature maps; the aggregated feature maps are sequentially input into a channel attention module and a spatial context module, wherein the channel attention module is used to select information of appropriate channels and resolutions for fusion and avoid interference, and the spatial context module obtains the topological morphological information of cracks by obtaining spatial information between pixel rows and columns in the image; the backbone part II upsamples the feature map through inverse wavelet transform so that the spatial resolution of the feature map is the same as the spatial resolution of the network input image, and then reduces the number of feature map channels to T through 1×1 convolution, where T is the number of crack types plus 1, and transmits it to the Softmax layer to generate a segmentation prediction map.

[0007] The main part I includes wavelet transform I and wavelet transform II. The input image is divided into high-frequency information graph I and low-frequency information graph I after being processed by wavelet transform I. The low-frequency information graph I is divided into high-frequency information graph II and low-frequency information graph II after being processed by wavelet transform II.

[0008] The main part II includes wavelet inverse transform I and wavelet inverse transform II. The feature map output by the high-frequency information map II and the spatial context module is successively output to the 1×1 convolution and Softmax layers after wavelet inverse transform I and wavelet inverse transform II. The high-frequency information map I is output to the 1×1 convolution and Softmax layers after wavelet inverse transform II.

[0009] In the four stages, the multi-resolution group convolution in the mth stage contains m parallel streams corresponding to m resolutions. Each stream consists of four one-time aggregation units. The fourth one-time aggregation unit in the stream corresponding to the minimum spatial resolution also outputs a feature of the stream through a convolution of size 3×3 and stride 2. Figure 1Feature map of half spatial resolution; in the 1st to 3rd stages, the multi-resolution convolution in the mth stage consists of m+1 3×3 convolutional layers, and the 4th stage consists of 4 3×3 convolutional layers; the convolutional layers in the multi-resolution convolution connect different branches in a fully connected manner for addition, and some connections need to reduce or increase the resolution due to the different resolutions of the multi-resolution convolution: the resolution is reduced through one or more convolutions with a stride of 2 and a convolution kernel size of 3×3, and the resolution is increased through bilinear interpolation and 1×1 convolution to achieve upsampling and the same number of channels.

[0010] The structure of the feature aggregation module is as follows: Feature map After being input into the feature aggregation module, it first passes through n convolution modules to obtain n new feature maps , where the first convolution module is a deformable convolution module; then the input feature map With n feature maps Perform channel stacking operation to obtain the feature map after channel stacking ; Next, the feature map will be input Perform a convolution operation with a convolution kernel size of 1×1 and a step size of 1 to obtain the same feature map Feature maps with the same number of channels , feature map With feature map Perform element addition operation to obtain the final output result;

[0011] The deformable convolution module adds a learnable offset variable to the position of each sampling point in the convolution kernel. Through these variables, the convolution kernel can sample arbitrarily near the current position, and is no longer limited to the previous regular grid points; the deformable convolution is conducive to the extraction of complex geometric features of cracks;

[0012] The convolution module is composed of a batch normalization layer, a ReLU activation function layer, a 3×3 convolution layer or a deformable convolution layer, and a random dropout layer in sequence.

[0013] The channel attention module structure is as follows: the size is The input features of are respectively subjected to global maximum pooling and global average pooling to obtain two The feature maps are then sent to a shared multilayer perceptron: the number of neurons in the first layer is , r is the reduction rate, r=16, the activation function is Relu, and the number of neurons in the second layer is ; The two output features are added, and then the Sigmoid activation operation is performed to generate the final channel attention; the channel attention and the input feature map perform corresponding element multiplication operations, and the result is then added to the input feature map.

[0014] The spatial context module includes four stages: top-to-bottom, bottom-to-top, left-to-right, and right-to-left. The feature map of the minimum spatial resolution corresponding stream from the fourth stage is assumed to be of size , where C represents the number of channels, H represents the feature map height, and W represents the feature map width. In the first stage, it is divided into H slices from top to bottom: , first for the first slice Use size A deformable convolution operation, where k is the convolution kernel width, k=5, is performed to obtain a new slice , then With the second slice After adding, the input is The deformable convolution gets another new slice , then the new slice With the third slice Add and repeat the above process until the last slice Updated to , and finally aggregate all new slices from top to bottom into a size of The feature map is input to the next stage; in the second stage, the feature map output from the first stage is divided into H slices from bottom to top along the height direction: In this phase, these slices are updated and reaggregated into a size of The feature map of is then input to the next stage; in the "left to right" and "right to left" stages, the size is The feature map is divided into W The slices are updated and aggregated in the same way as in the first two stages.

[0015] Wavelet transform I, wavelet transform II, inverse wavelet transform I and inverse wavelet transform II use Daubechies wavelet as wavelet basis.

[0016] A surface crack detection method based on a multi-scale topology perception neural network comprises the following steps:

[0017] Step 1: Divide the surface crack dataset into a training set and a test set, and preprocess the images in the training set and the test set respectively;

[0018] Step 2: Build a semantic segmentation network for surface crack images;

[0019] Step 3: Input the preprocessed training set surface crack images in step 1 into the surface crack semantic segmentation network in step 2 for training. First, the surface crack segmentation network is initialized using the He Uniform method, and then the parameters in the model are updated to optimize the loss function until convergence.

[0020] Step 4: Input the preprocessed test set remote sensing image into the surface crack segmentation network trained in step 3 and output the segmentation result.

[0021] The preprocessing in step 1 includes manual image annotation, image cropping, and data augmentation;

[0022] The manual image annotation is specifically as follows: manually annotating the surface cracks in the image at the pixel level in the Labelme software to obtain a labeled surface crack image;

[0023] The image cropping is specifically as follows: randomly cropping the labeled surface crack image into sub-images of 512 pixels×512 pixels;

[0024] The data enhancement includes: performing random scale transformation on the sub-image, random angle image rotation, and vertical and horizontal flipping of the image to obtain a surface crack image.

[0025] The loss function in step 3 adopts Dice Loss, which is specifically:

[0026] ,

[0027] Where X represents the set of predicted values ​​corresponding to all pixels in the prediction image, and Y represents the set of values ​​corresponding to all pixels in the label image. represents the intersection of X and Y, and Represents the number of elements in X and Y.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1) The present invention uses an improved HRnet semantic segmentation model for surface crack detection. The HRnet semantic segmentation architecture divides feature extraction into four stages. During the feature extraction process, it not only maintains a high resolution, but also generates some low-resolution feature maps in parallel after each stage, and finally generates multiple sets of feature maps with different resolutions, and then fuses these feature maps to predict semantic maps. In addition, different resolution features are fused in each of the three stages. Therefore, HRNet has a strong feature expression capability and low information loss during the feature extraction process, which is very suitable for extracting crack features with rich detail information and multi-scale.

[0030] 2) The present invention uses wavelet transform and inverse wavelet transform in the network backbone parts I and II to implement downsampling and upsampling to further avoid information loss in the model, thereby improving model performance.

[0031] 3) The present invention uses a one-time aggregation module to replace the residual unit to extract crack features in the four stages. This design makes the model more reusable and reduces redundant calculations, so the model is easy to train, does not require a pre-trained model, and reduces the number of samples required for model training.

[0032] 4) The present invention adopts a channel attention module. This design is conducive to selecting information of appropriate channels and resolutions for fusion and avoiding interference, so as to better improve the semantic segmentation accuracy of the model;

[0033] 5) The present invention adopts a spatial context module. The spatial context module converts the traditional convolution layer-by-layer connection form into a slice-by-slice convolution form in the feature map, so that information can be transmitted between pixel rows and columns in the map. Therefore, even when the crack is occluded or other appearance clues are poor, the spatial context module is still suitable for detecting crack targets with long-distance continuous shapes due to its powerful topological morphological perception ability.

[0034] Through the above beneficial effects, the present invention comprehensively solves the problems existing in the existing crack target detection, such as poor feature table capability, inability to accurately perceive the topological structure of the crack, large loss of edge and detail information during semantic segmentation, and large sample size required for training. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the overall structure of the surface crack detection network constructed in the method of the present invention;

[0036] Figure 2 A schematic diagram of the structural composition of a backbone network in a surface crack detection network constructed in the method of the present invention;

[0037] Figure 3 A schematic diagram of the structural composition of a primary aggregation module combined with deformable convolution in a surface crack detection network constructed in the method of the present invention;

[0038] Figure 4 A schematic diagram of the composition structure of a channel attention module in a surface crack detection network constructed in the method of the present invention;

[0039] Figure 5 Schematic diagram of the composition structure of the spatial context module in the surface crack detection network constructed in the method of the present invention. DETAILED DESCRIPTION

[0040] The specific implementation modes of the present invention are described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention.

[0041] like Figure 1-5The present invention relates to a surface crack detection method based on a multi-scale topological perception neural network, such as Figure 1 As shown in the flow chart, the following steps are included:

[0042] Step 1: Divide the surface crack dataset into a training set and a test set, and preprocess the images in the training set and the test set respectively.

[0043] The preprocessing in step 1 includes manual image annotation, image cropping and data enhancement.

[0044] The manual image annotation specifically includes: manually annotating the surface cracks in the image at the pixel level in the Labelme software to obtain a labeled surface crack image.

[0045] The image cropping is specifically as follows: randomly cropping the labeled surface crack image into sub-images of 512 pixels×512 pixels.

[0046] The data enhancement includes: performing random scale transformation on the sub-image, random angle image rotation, and vertical and horizontal flipping of the image to obtain a surface crack image.

[0047] Step 2: Build a surface crack image semantic segmentation network. The surface crack image semantic segmentation network adopts a network structure similar to HRnet: the network consists of a backbone part I, a backbone network, a feature aggregation module, a channel attention module, a spatial context module and a backbone part II.

[0048] The backbone part I outputs the input image into a feature map with a quarter of the spatial resolution of the input image through two wavelet transforms.

[0049] The backbone network is composed of four stages and feature aggregation modules in sequence. Each stage consists of a set of multi-resolution group convolutions and multi-resolution convolutions. The multi-resolution group convolution of the mth stage contains m parallel streams corresponding to m resolutions. Each stream consists of four one-time aggregation units. The fourth one-time aggregation unit in the stream corresponding to the minimum spatial resolution also outputs a feature of the stream through a convolution of size 3×3 and stride 2. Figure 1Feature map of half spatial resolution; in the 1st to 3rd stages, the multi-resolution convolution of the mth stage consists of m+1 3×3 convolutional layers, and the 4th stage consists of 4 3×3 convolutional layers; the convolutional layers in the multi-resolution convolution connect different branches in a fully connected manner for addition, and some connections need to reduce or increase the resolution due to the different resolutions of the multi-resolution convolution: the resolution is reduced by one or more convolutions with a stride of 2 and a kernel size of 3×3, and the resolution is increased by upsampling and the same number of channels through bilinear interpolation and 1×1 convolution; the feature aggregation module unifies the resolution of the feature maps output by all streams through bilinear interpolation and aggregates the output features of different streams.

[0050] The aggregated feature map is input into the channel attention module and the spatial context module. The former is used to select information of appropriate channels and resolutions for fusion and avoid interference, and the latter obtains the topological morphological information of the crack by acquiring the spatial information between pixel rows and columns in the image.

[0051] The structure of the primary aggregation module is as follows: After being input into the primary aggregation module, it first passes through n convolution modules (n=4) to obtain n new feature maps (The first convolution module is a deformable convolution module); then the input feature map With n feature maps Perform channel stacking operation to obtain the feature map after channel stacking ; Next, the feature map will be input Perform a convolution operation with a convolution kernel size of 1×1 and a step size of 1 to obtain the same feature map Feature maps with the same number of channels , feature map With feature map Perform element addition operation to obtain the final output result; the one-time aggregation module adds the features Figure 1 One-time aggregation reuses the features extracted by the network and avoids redundant calculations, thus effectively reducing the difficulty of model training and the number of required samples; in the one-time aggregation module, the residual connection is conducive to the back propagation of the gradient.

[0052] The deformable convolution adds a learnable offset variable to the position of each sampling point in the convolution kernel; through these variables, the convolution kernel can sample arbitrarily near the current position, and is no longer limited to the previous regular grid points; the deformable convolution is conducive to the extraction of complex geometric features of cracks.

[0053] The convolution module or deformable convolution module of the primary aggregation module is composed of a batch normalization layer, a ReLU activation function layer, a 3×3 convolution layer or a deformable convolution layer, and a random inactivation layer (inactivation probability p=0.2) in sequence.

[0054] The channel attention module structure is as follows: The input features of are respectively subjected to global maximum pooling and global average pooling to obtain two The feature maps are then sent to a shared multilayer perceptron: the number of neurons in the first layer is (r is the reduction rate, r=16), the activation function is Relu, and the number of neurons in the second layer is ; The two output features are added, and then the Sigmoid activation operation is performed to generate the final channel attention; the channel attention and the input feature map perform corresponding element multiplication operations, and the result is then added to the input feature map.

[0055] The spatial context module consists of four stages: "from top to bottom", "from bottom to top", "from left to right" and "from right to left"; the feature map of the minimum spatial resolution corresponding stream from the fourth stage (assuming the size is , where C represents the number of channels, H represents the feature map height, and W represents the feature map width) is divided into H slices from top to bottom in the first stage: , first for the first slice Use size The deformable convolution operation (k is the convolution kernel width, k=5) obtains the new slice , then With the second slice After adding, the input is The deformable convolution gets another new slice , then the new slice With the third slice Add and repeat the above process until the last slice Updated to , and finally aggregate all new slices from top to bottom into a size of The feature map is input to the next stage; in the second stage, the feature map output from the first stage is divided into H slices from bottom to top along the height direction: In this phase, these slices are updated and reaggregated into a size of The feature map of is then input to the next stage; in the "left to right" and "right to left" stages, the size is The feature map is divided into W The slices are updated and aggregated in the same way as in the first two stages.

[0056] Backbone II upsamples the output to the spatial resolution of the network input image through two inverse wavelet transforms, and then 1 Convolution reduces the number of feature map channels to T (the number of crack types plus one) and transmits it to the Softmax layer to generate a segmentation prediction map.

[0057] The wavelet transform and inverse wavelet transform use Daubechies wavelet as the wavelet basis; each channel of the image after a wavelet transform is decomposed into four sub-images with half the original height and width: , , and , where the order of the letters in the subscripts represents the horizontal and vertical directions, respectively. G represents high frequency information, and D Represents low-frequency information, such as GG Represents: Both horizontal and vertical directions are high-frequency information subscripts; The sub-image is input into the network model, and the remaining sub-images are saved. Therefore, the height and width of the image in the network model are reduced by half each time after wavelet transformation, while the number of channels of the image remains unchanged. The image output by the network together with the remaining sub-images saved when inputting the network model undergoes inverse wavelet transformation, and the width and height of the image are doubled, while the number of channels remains unchanged. The advantage of wavelet transform and inverse wavelet transform over other downsampling and upsampling processes is that there is no information loss.

[0058] Step 3: Input the preprocessed training set surface crack images in step 1 into the surface crack semantic segmentation network in step 2 for training. First, use the He Uniform method to initialize the surface crack segmentation network, then update the parameters in the model, and optimize the loss function until convergence.

[0059] The loss function in step 3 adopts Dice Loss, which is specifically:

[0060] ,

[0061] in X Represents the set of predicted values ​​corresponding to all pixels in the prediction map, Y Represents the set of corresponding values ​​of all pixels in the label image, represent X and Y The intersection of and represent X and Y The number of elements in .

[0062] Step 4: Input the preprocessed test set remote sensing image into the surface crack segmentation network trained in step 3 and output the segmentation result.

[0063] What needs to be explained about the specific structure of the present invention is that the connection relationship between the various component modules adopted in the present invention is definite and feasible. Except for the special instructions in the embodiments, the specific connection relationship can bring about the corresponding technical effects, and solve the technical problems raised by the present invention without relying on the execution of the corresponding software program. Except for the specific instructions, the models and connection methods of the components, modules and specific components appearing in the present invention belong to the existing technologies such as published patents, published journal articles, or common knowledge that can be obtained by technical personnel in this field before the application date, and there is no need to elaborate, so that the technical solution provided in this case is clear, complete and feasible, and the corresponding physical products can be reproduced or obtained according to the technical means.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surface crack image semantic segmentation network system, characterized by: It includes backbone part I, backbone network, channel attention module, spatial context module and backbone part II; The backbone part I outputs the input image as a feature map with a quarter spatial resolution of the input image through wavelet transformation; The backbone network is composed of four stages and a feature aggregation module in sequence. Each stage includes a set of multi-resolution group convolutions and multi-resolution convolutions. The feature aggregation module unifies the resolution of feature maps output by all streams through bilinear interpolation and aggregates the outputs of different streams into feature maps. The aggregated feature map is sequentially input into the channel attention module and the spatial context module. The channel attention module is used to select information of appropriate channels and resolutions for fusion and avoid interference. The spatial context module obtains the topological morphological information of the crack by obtaining the spatial information between pixel rows and columns in the image. The backbone part II upsamples the feature map by inverse wavelet transform to make the spatial resolution of the feature map the same as the spatial resolution of the network input image, and then reduces the number of feature map channels to T by 1×1 convolution, where T is the number of crack types plus 1, and transmits it to the Softmax layer to generate a segmentation prediction map; The backbone part I includes wavelet transform I and wavelet transform II. The input image is divided into a high-frequency information graph I and a low-frequency information graph I after being processed by wavelet transform I. The low-frequency information graph I is divided into a high-frequency information graph II and a low-frequency information graph II after being processed by wavelet transform II. The backbone part II includes wavelet inverse transform I and wavelet inverse transform II, the high-frequency information graph II and the feature graph output by the spatial context module are successively subjected to wavelet inverse transform I and wavelet inverse transform II and then output to a 1×1 convolution and a Softmax layer, and the high-frequency information graph I is subjected to wavelet inverse transform II and then output to a 1×1 convolution and a Softmax layer; In the four stages, the multi-resolution group convolution in the mth stage contains m parallel streams corresponding to m resolutions, each stream consists of four one-time aggregation units, and the fourth one-time aggregation unit in the stream corresponding to the minimum spatial resolution also outputs a feature map with half the spatial resolution of the feature map of the stream through a convolution of size 3×3 and stride 2; in the first to third stages, the multi-resolution convolution in the mth stage consists of m+1 3×3 convolution layers, and the fourth stage consists of 4 33 convolution layers; the convolution layers in the multi-resolution convolution connect different branches in a fully connected manner for addition, and some connections need to reduce or increase the resolution due to different resolutions of the multi-resolution convolution: the resolution reduction is achieved by one or more convolutions with a stride of 2 and a convolution kernel size of 3×3, and the resolution increase is achieved by upsampling and the same number of channels through bilinear interpolation and 1×1 convolution; The structure of the feature aggregation module is as follows: Feature map After being input into the feature aggregation module, it first passes through n convolution modules to obtain n new feature maps , where the first convolution module is a deformable convolution module; then the input feature map With n feature maps Perform channel stacking operation to obtain the feature map after channel stacking ; Next, the feature map will be input Perform a convolution operation with a convolution kernel size of 1×1 and a step size of 1 to obtain the same feature map Feature maps with the same number of channels , feature map With feature map Perform element addition operation to obtain the final output result; The deformable convolution module adds a learnable offset variable to the position of each sampling point in the convolution kernel; The convolution module is composed of a batch normalization layer, a ReLU activation function layer, a 3×3 convolution layer or a deformable convolution layer, and a random dropout layer in sequence.

2. The surface crack image semantic segmentation network system according to claim 1, characterized in that: The channel attention module structure is as follows: The input features of are respectively subjected to global maximum pooling and global average pooling to obtain two The feature maps are then sent to a shared multilayer perceptron: the number of neurons in the first layer is , r is the reduction rate, r=16, the activation function is Relu, and the number of neurons in the second layer is ; The two output features are added, and then the Sigmoid activation operation is performed to generate the final channel attention; the channel attention and the input feature map perform corresponding element multiplication operations, and the result is then added to the input feature map.

3. The surface crack image semantic segmentation network system according to claim 2, characterized in that: The spatial context module includes four stages: "up to down", "down to up", "left to right" and "right to left". The feature map of the minimum spatial resolution corresponding stream from the fourth stage is assumed to be of size , where C represents the number of channels, H represents the feature map height, and W represents the feature map width. In the first stage, it is divided into H slices from top to bottom: , first for the first slice Use size A deformable convolution operation is performed, where k is the width of the convolution kernel, k=5, to obtain a new slice , then With the second slice After adding, the input is The deformable convolution gets another new slice , then the new slice With the third slice Add and repeat the above process until the last slice Updated to , and finally aggregate all new slices from top to bottom into a size of The feature map is input to the next stage; In the second stage, the feature map output from the first stage is divided into H slices from bottom to top along the height direction: In this phase, the slices are updated and reaggregated into a size of The feature map of is then input to the next stage; in the "left to right" and "right to left" stages, the size is The feature map is divided into W The slices are updated and aggregated in the same way as in the first two stages.

4. The surface crack image semantic segmentation network system according to claim 1 or 3, characterized in that: The wavelet transform I, wavelet transform II, inverse wavelet transform I and inverse wavelet transform II use Daubechies wavelet as wavelet basis.

5. A surface crack detection method based on a multi-scale topological perception neural network, characterized in that: The following steps are included: Step 1: Divide the surface crack dataset into a training set and a test set, and preprocess the images in the training set and the test set respectively; Step 2: Building a surface crack image semantic segmentation network system as described in any one of claims 1 to 4; Step 3: Input the preprocessed training set surface crack images in step 1 into the surface crack semantic segmentation network in step 2 for training. First, use the He Uniform method to initialize the surface crack segmentation network system, then update the parameters in the model, and optimize the loss function until convergence. Step 4: Input the preprocessed test set remote sensing image into the surface crack segmentation network trained in step 3 and output the segmentation result.

6. The surface crack detection method based on multi-scale topology perception neural network according to claim 5 is characterized in that: The preprocessing in step 1 includes manual image annotation, image cropping and data enhancement; The manual image annotation is specifically as follows: manually annotating the surface cracks in the image at the pixel level in the Labelme software to obtain a labeled surface crack image; The image cropping is specifically as follows: randomly cropping the labeled surface crack image into sub-images of 512 pixels×512 pixels; The data enhancement includes: performing random scale transformation, random angle image rotation, and vertical and horizontal flipping of the sub-image to obtain a surface crack image; The loss function in step 3 adopts Dice Loss, which is specifically: ， Where X represents the set of predicted values ​​corresponding to all pixels in the prediction image, and Y represents the set of values ​​corresponding to all pixels in the label image. represents the intersection of X and Y, and Represents the number of elements in X and Y.

Images