Pavement crack detection method and device

Abstract

The embodiment of the invention provides a pavement crack detection method and device. The method comprises the following steps: acquiring a crack image; inputting the crack image into a pre-constructed crack detection model, and outputting a crack detection result by the crack detection model; wherein the crack detection model comprises a block processing module, a feature extraction module and a feature fusion module, and the block processing module is used for performing down-sampling on an input crack image to obtain a crack feature map; the feature extraction module is used for extracting local slender morphological features and overall topological connection features of a crack based on the crack feature map to obtain a local feature map and an overall feature map, and the feature fusion module is used for fusing the local feature map and the overall feature map to obtain a fused crack feature map. The method can improve the recognition precision of the complex crack of the pavement.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method and device for detecting pavement cracks. Background Art

[0002] Cracks, as a common type of road pavement damage, determine the health of the road surface. Accurate extraction and characterization of pavement cracks can accurately quantify and evaluate the technical condition of the pavement, and ultimately serve as an important technical indicator to guide the formulation of scientific pavement maintenance decisions. Pavement cracks generally have morphological characteristics of thin branches and strong connectivity. The crack detection model based on deep learning can identify pavement cracks, but it is difficult to simultaneously capture the local detailed edge features and overall topological connection features of the cracks, resulting in low crack recognition accuracy. Summary of the invention

[0003] In view of this, an object of the embodiments of the present application is to provide a pavement crack detection method and device to solve the problem of low crack identification accuracy.

[0004] Based on the above purpose, the embodiment of the present application provides a road crack detection method, including:

[0005] Acquire crack images;

[0006] The crack image is input into a pre-built crack detection model, and the crack detection model outputs the crack detection result; wherein the crack detection model includes a block processing module, a feature extraction module, and a feature fusion module, the block processing module is used to downsample the input crack image to obtain a crack feature map, the feature extraction module is used to extract the local slender morphological features and overall topological connection features of the crack based on the crack feature map to obtain a local feature map and an overall feature map, and the feature fusion module is used to fuse the local feature map and the overall feature map to obtain a fused crack feature map.

[0007] Optionally, the feature extraction module includes a first high-resolution branch, a second high-resolution branch and a low-resolution branch;

[0008] The first high-resolution branch is used to extract a local elongated morphological feature having a first scale based on the crack feature map and the feature map extracted by the second high-resolution branch, so as to obtain a first local feature map;

[0009] The second high-resolution branch is used to extract local elongated morphological features with a second scale based on the feature map extracted by the first high-resolution branch, so as to obtain a second local feature map;

[0010] The low-resolution branch is used to extract overall topological connection features with a third scale based on the feature maps extracted by the first high-resolution branch and the second high-resolution branch to obtain the overall feature map.

[0011] Optionally, the first high-resolution branch includes a first VSS module, a second VSS module and a third VSS module, the second high-resolution branch includes a fourth VSS module and a fifth VSS module, and the low-resolution branch includes a sixth VSS module;

[0012] The input of the first VSS module is the crack feature map, the input of the second VSS module is the feature map output by the first VSS module, the input of the third VSS module is the feature map output by the second VSS module, and the feature map obtained by upsampling the feature map output by the fourth VSS module;

[0013] The input of the fourth VSS module is a feature map obtained by downsampling the feature map output by the first VSS module, and the input of the fifth VSS module is a feature map output by the fourth VSS module and a feature map obtained by downsampling the feature map output by the second VSS module;

[0014] The input of the sixth VSS module is a feature map obtained by downsampling the feature map output by the second VSS module and a feature map obtained by downsampling the feature map output by the fourth VSS module.

[0015] Optionally, the first VSS module includes a patch embedding layer and a VSS block;

[0016] The patch embedding layer is used to maintain the resolution of the input feature map. The feature map of the input VSS block enters the first branch and the second branch for processing respectively after layer normalization. The first branch performs convolution operation and Silu activation function processing on the feature map. The second branch performs convolution operation, batch normalization and Silu activation function processing on the feature map, and then inputs it into the SS2D module for processing. The feature map output by the SS2D module is layer normalized and then merged with the feature map output by the first branch by element-by-element dot product. The merged feature map is convolved to mix features, and the mixed feature map is fused with the feature map of the input VSS block.

[0017] Optionally, the first layer of the second branch performs 1X1 convolution, batch normalization and Silu activation function on the feature map to obtain the feature map processed by the first layer. The feature map processed by the first layer enters the second layer, and through depth-wise separable convolution, each input channel is operated independently without mixing channel information, thereby retaining the spatial information of the feature map of the first layer and obtaining the feature map processed by the second layer.

[0018] Optionally, the second VSS module, the third VSS module, the fourth VSS module, the fifth VSS module and the sixth VSS module include a patch merging layer and a VSS block.

[0019] Optionally, the feature fusion module includes a first deformable convolution layer, a second deformable convolution layer and a self-attention layer;

[0020] The first deformable convolution layer is used to enhance the first local feature map using deformable convolution to obtain an enhanced first local feature map, the second deformable convolution layer is used to enhance the second local feature map using deformable convolution to obtain an enhanced second local feature map, and the self-attention layer is used to enhance the overall feature map based on the self-attention mechanism to obtain an enhanced overall feature map.

[0021] Optionally, the feature fusion module also includes a fully connected layer, which is used to convert the enhanced first local feature map, the second local feature map and the overall feature map into three feature maps of the same size, splice the three feature maps of the same size, and fuse the spliced ​​feature maps using 1x1 convolution to obtain the fused crack feature map.

[0022] Optionally, the block processing module is used to reduce the height of the crack image to a predetermined height, and extract features of the crack image to obtain the crack feature map.

[0023] The present application also provides a road crack detection device, including:

[0024] An acquisition module, used for acquiring crack images;

[0025] A detection module is used to input the crack image into a pre-built crack detection model, and the crack detection model outputs the crack detection result; wherein the crack detection model includes a block processing module, a feature extraction module, and a feature fusion module, the block processing module is used to downsample the input crack image to obtain a crack feature map, the feature extraction module is used to extract the local slender morphological features and overall topological connection features of the crack based on the crack feature map to obtain a local feature map and an overall feature map, and the feature fusion module is used to fuse the local feature map and the overall feature map to obtain a fused crack feature map.

[0026] From the above description, it can be seen that the pavement crack detection method and device provided in the embodiment of the present application input the acquired crack image into a pre-built crack detection model, and use the crack detection model to output the crack detection result. The model uses three resolution branches to respectively extract the local detail features and overall topological connection features of the cracks, and uses the VSS module instead of the convolution operation to enhance the model's ability to understand global information. For the extracted local detail features, the local slender morphology of the cracks is further enhanced by deformable convolution. For the extracted overall topological connection features, the contextual semantic information is globally modeled based on the self-attention mechanism to enhance the overall complex morphology of the cracks. Finally, the features of multiple branches are fused through convolution. The fused crack feature map can accurately depict the local details and overall topological morphology of the cracks, thereby improving the recognition accuracy of complex pavement cracks. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 A schematic diagram of a method flow of an embodiment of the present application;

[0029] Figure 2 This is a schematic diagram of the model structure of an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the processing flow of a VSS block according to an embodiment of the present application;

[0031] Figure 4 A schematic diagram of the processing flow of the SS2D module of an embodiment of the present application;

[0032] Figure 5 A schematic diagram of the principle of variable convolution feature extraction in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of the principle of the self-attention mechanism of an embodiment of the present application;

[0034] Figure 7 This is a block diagram of the device structure of an embodiment of the present application;

[0035] Figure 8 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0037] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0038] like Figure 1 As shown, the embodiment of the present application provides a road surface crack detection method, comprising:

[0039] S101: Acquire a crack image;

[0040] S102: Input the crack image into a pre-built crack detection model, and the crack detection model outputs the crack detection result; wherein the crack detection model includes a block processing module, a feature extraction module, and a feature fusion module. The block processing module is used to downsample the input crack image to obtain a crack feature map. The feature extraction module is used to extract the local slender morphological features and overall topological connection features of the crack based on the crack feature map to obtain a local feature map and an overall feature map. The feature fusion module is used to fuse the local feature map and the overall feature map to obtain a fused crack feature map.

[0041] In this embodiment, an image acquisition device is used to acquire crack images including road cracks, and after preprocessing the crack images, the preprocessed crack images are input into a pre-built crack detection model, and the crack images are detected using the crack detection model. In some embodiments, preprocessing the crack images includes image enhancement, denoising, etc., and the specific preprocessing method is not described in detail.

[0042] like Figure 2As shown, the crack detection model constructed in this embodiment includes a block processing module, a feature extraction module and a feature fusion module. Among them, the block processing module is used for image block and patch coding processing, specifically, the input crack image is divided into multiple non-overlapping image blocks (patches) of PxP size, each image block can be regarded as a small image area, and each image block has the same pixel value. To meet the needs of subsequent processing, the image block is encoded into a sequence format using patch coding. For example, each image block of size PxPx3 is processed into a one-dimensional vector, and then the one-dimensional vector corresponding to each image block is linearly transformed by a fully connected layer and mapped to a fixed high-dimensional space, that is, each image block of the input crack image is mapped to a high-dimensional space using a linear transformation. After passing through the block processing module, the height of the crack image is reduced from H to H / 4, and the features (tokens) of the crack image are extracted at the same time to obtain a crack feature map.

[0043] In some embodiments, the feature extraction module includes a first high-resolution branch, a second high-resolution branch, and a low-resolution branch;

[0044] The first high-resolution branch is used to extract a local elongated morphological feature having a first scale based on the crack feature map and the feature map extracted by the second high-resolution branch, so as to obtain a first local feature map;

[0045] The second high-resolution branch is used to extract local elongated morphological features with a second scale based on the feature map extracted by the first high-resolution branch, so as to obtain a second local feature map;

[0046] The low-resolution branch is used to extract overall topological connection features with a third scale based on the feature maps extracted by the first high-resolution branch and the second high-resolution branch to obtain an overall feature map.

[0047] In this embodiment, the feature extraction module uses three different resolution branches, namely, the first high-resolution branch, the second high-resolution branch and the low-resolution branch, to extract image features of different scales in parallel. The first high-resolution branch and the second high-resolution branch use feature maps of larger sizes to focus on local detail features of cracks, and the low-resolution branch uses feature maps of smaller sizes to focus on contextual semantic information of the image, such as topological connection features of cracks. Among them, the first high-resolution branch and the second high-resolution branch are high-resolution branches, which are used to extract local elongated morphological features in crack images, and the low-resolution branch is used to extract overall topological connection features in crack images. Each resolution branch fuses feature information of other resolution branches to increase the interactive capability of feature information of different resolution branches. Specifically, the first high-resolution branch is used to extract local elongated morphological features with a first scale based on the feature map extracted by the crack feature map and the second high-resolution branch, and obtain a first local feature map; the second high-resolution branch is used to extract local elongated morphological features with a second scale based on the feature map extracted by the first high-resolution branch, and obtain a second local feature map; the low-resolution branch is used to extract overall topological connection features with a third scale based on the feature maps extracted by the first high-resolution branch and the second high-resolution branch, and obtain an overall feature map.

[0048] In some embodiments, the first high-resolution branch includes a first VSS module, a second VSS module, and a third VSS module, the second high-resolution branch includes a fourth VSS module and a fifth VSS module, and the low-resolution branch includes a sixth VSS module;

[0049] The input of the first VSS module is the crack feature map, the input of the second VSS module is the feature map output by the first VSS module, the input of the third VSS module is the feature map output by the second VSS module, and the feature map obtained by upsampling the feature map output by the fourth VSS module;

[0050] The input of the fourth VSS module is a feature map obtained by downsampling the feature map output by the first VSS module, and the input of the fifth VSS module is a feature map output by the fourth VSS module and a feature map obtained by downsampling the feature map output by the second VSS module;

[0051] The input of the sixth VSS module is a feature map obtained by downsampling the feature map output by the second VSS module and a feature map obtained by downsampling the feature map output by the fourth VSS module.

[0052] like Figure 2As shown, in this embodiment, the first high-resolution branch R1 includes a first VSS module, a second VSS module and a third VSS module. After three convolution stages, the depth of the model is increased and the feature extraction capability is improved. In the convolution stage S1, the crack feature map is input into the first VSS module for processing. In the convolution stage S2, the feature map output by the first VSS module is input into the second VSS module for processing. In the convolution stage S3, the input of the third VSS module is the feature map output by the second VSS module and the feature map obtained by upsampling the feature map output by the fourth VSS module. The input of the third VSS module can be expressed as: f(R1, S3) = f(R1, S2) + f(R2, S2) ↑ .

[0053] The second high-resolution branch includes a fourth VSS module and a fifth VSS module. In the convolution stage S2, the input of the fourth VSS module is a feature map obtained by downsampling the feature map output by the first VSS module, and the feature map output by the fourth VSS module is input to the fifth VSS module for processing. In the convolution stage S3, the input of the fifth VSS module is the feature map output by the fourth VSS module and the feature map obtained by downsampling the feature map output by the second VSS module. The input of the fifth VSS module can be expressed as: f(R2, S3) = f(R1, S2) ↓ +f(R2,S2).

[0054] The low-resolution branch includes a sixth VSS module. In the convolution stage S3, the input of the sixth VSS module is a feature map obtained by downsampling the feature map output by the second VSS module and a feature map obtained by downsampling the feature map output by the fourth VSS module. The input of the sixth VSS module can be expressed as: f(R3, S3) = f(R1, S2) ↓ +f(R2,S2) ↓ .

[0055] In some embodiments, the first VSS module includes a patch embedding layer and a VSS block; the patch embedding layer is used to maintain the resolution of the input feature map, and the feature map of the input VSS block enters the first branch and the second branch for processing respectively after layer normalization. The first branch performs convolution operation and Silu activation function processing on the feature map, and the second branch performs convolution operation, batch normalization and Silu activation function processing on the feature map, and then inputs it into the SS2D module for processing. The feature map output by the SS2D module is layer normalized and then element-by-element dot product is performed with the feature map output by the first branch to improve channel correlation, and the merged feature map is then convolved to mix features, and the mixed feature map and the feature map of the input VSS block are fused through residual splicing, so that the model can understand and integrate features of different dimensions in the image, thereby improving the robustness to scale changes.

[0056] The size of the patch embedding layer (patch encoding) is 2X2, which is used to maintain the resolution of the feature map at 1 / 4x. Figure 3 As shown in Figure 1, point-by-point convolution is designed in the VSS block to extract more high-dimensional features, taking advantage of local convolution to improve detection accuracy and the model's understanding of complex scenes. After the first layer L1 of the second branch performs 1X1 convolution, batch normalization, and Silu activation function on the feature map, the obtained feature can be expressed as x L1 =Φ(BN(Conv 1X1 (LN(x)))), where the feature x of the input VSS block is processed by layer normalization to LN(x) and processed by 1X1 convolution to Conv 1X1 (LN(x)), the convolutional features are processed by batch normalization to BN(Conv 1X1 (LN(x))), and then processed by Silu activation function to Φ(BN(Conv 1X1 (LN(x)))). The feature map processed by the first layer L1 enters the second layer L2 through the depth-separable convolution to operate each input channel independently without mixing channel information, retaining the spatial information of the feature map of the first layer L1. The feature after processing in the second layer can be expressed as x L2 =Φ(BN(DWConv 3X3 (x L1 ))), where feature x L1 After 3X3 depth-separable convolution processing, it becomes DWConv 3X3 (x L1 ), the activation function Φ() uses the nonlinear GELU to change the number of channels of the feature without changing the spatial dimension, thereby enhancing the feature representation.

[0057] In some embodiments, the second VSS module, the third VSS module, the fourth VSS module, the fifth VSS module, and the sixth VSS module include a patch merging layer and a VSS block. The patch merging layer (patch merging) is used to reduce feature dimensions by 2x downsampling.

[0058] like Figure 4 As shown, in some embodiments, the SS2D module backs up four copies of the input feature map. For the four identical feature maps, cross-search scanning is performed in four directions from top to bottom, from bottom to top, from left to right, and from right to left, and converted into a patch sequence to establish a global context relationship. The converted patch sequence is processed by four identical S6 blocks, and the processed results are spliced ​​to obtain the spliced ​​feature map. Among them, the learning parameters A, D, B, and C in the S6 block are the mapping matrices obtained after linear operations, and h t is the hidden state sequence.

[0059] In some embodiments, the feature fusion module includes a first deformable convolution layer, a second deformable convolution layer, and a self-attention layer;

[0060] The first deformable convolution layer is used to enhance the first local feature map by using deformable convolution to obtain an enhanced first local feature map. The second deformable convolution layer is used to enhance the second local feature map by using deformable convolution to obtain an enhanced second local feature map. The self-attention layer is used to enhance the overall feature map based on the self-attention mechanism to obtain an enhanced overall feature map.

[0061] In this embodiment, after using three resolution branches to extract the first local feature map, the second local feature map and the overall feature map of the crack respectively, the feature fusion module is used to fuse the features extracted by the multi-resolution branches. For the first local feature map output by the first high-resolution branch, the first deformable convolution layer is used to further extract the local slender morphological features of the crack through deformable convolution to enhance the local slender morphology; for the second local feature map output by the second high-resolution branch, the second deformable convolution layer is used to further extract the local slender morphological features of the crack through deformable convolution to enhance the local slender morphology; for the overall feature map output by the low-resolution branch, the self-attention layer is used to globally connect the contextual semantic information through the self-attention mechanism to enhance the overall topological connection morphology of the crack.

[0062] In some embodiments, the deformable convolution uses a full convolution layer to predict a set of offsets, corresponding to the translation of each position of a convolution kernel in the x and y directions. Through a set of offsets, the receptive field of the convolution kernel can flexibly select grid positions that can accurately depict the elongated shape of the crack.

[0063] like Figure 5 As shown in the figure, when a convolution kernel with a fixed size of 3x3 is used to extract features, the feature output at the (i, j) position of the feature map is obtained by weighted summation of the feature of the convolution kernel at the grid position of the feature map and the weight at the corresponding position, which can be expressed as:

[0064] Y(i,j)=w1X(i-1,j-1)+w2X(i,j-1)+w3X(i+1,j-1)+w4X(i-1,j)+w5X(i,j)+w6X(i+1,j)+w7X(i-1,j+1)+w8X(i,j+1)+

[0065] w9X(i+1,j+1)(1)

[0066] Assuming there is a crack around the position (i, j), the deformable convolution can automatically learn a set of offsets according to the shape of the crack, so that the receptive field of the convolution kernel changes, and the fixed convolution kernel is changed into an irregular convolution kernel. The characteristics of the irregular convolution kernel output are:

[0067] Y(i,j) * =w1X(i-2,j-2)+w2X(i,j-2)+w3X(i+1,j-1)+

[0068] w4X(i-2,j)+w5X(i-1,j)+w6X(i+2,j+1)+w7X(i-1,j+1)+

[0069] w8X(i,j+1)+w9X(i+3,j+3)(2)

[0070] The features extracted by the irregular convolution kernel are more consistent with the local slender shape of the crack and can more accurately represent the local slender shape characteristics of the crack.

[0071] like Figure 6 As shown in the figure, the self-attention mechanism is used to establish the correlation between the current position and all other positions in the feature map, and fuse the features of other positions with the features of the current position according to the correlation. By establishing a global connection, each feature point on the crack is integrated with the information of feature points at other positions in the image. The processing flow of the self-attention mechanism is as follows: for the input feature map C1 is the number of channels. The feature map is dimensionalized by reshape operation of tensor, which is expressed as Where N = H × W; three fully connected layers are used to map the input feature map into three sets of feature vectors Query (Q), Key (K), Value (V), where Q = W Q f,K=W K f, C2 is the number of channels, W Q , W K , W V is the weight; calculate the correlation between each position of the feature vector Q and each position of the feature vector K, and normalize the calculation results to obtain a normalized correlation score, which is expressed as: According to the calculated correlation score, for each position of the feature vector V, the features at all other positions in the feature vector V are weighted and summed, expressed as: Z = HV T , the matrix Z is the feature map obtained after the self-attention mechanism is operated. The feature map is integrated with the feature map of the input self-attention layer to obtain the feature map F=f+Z output by the self-attention layer.

[0072] In some embodiments, the feature fusion module also includes a fully connected layer, which is used to convert the enhanced first local feature map, the second local feature map, and the overall feature map into three feature maps of the same size, splice the three feature maps of the same size, and fuse the spliced ​​feature maps using 1x1 convolution to obtain a fused crack feature map.

[0073] like Figure 2 As shown in the figure, the second local feature map output by the second deformable convolution layer is upsampled by 2 times, and the overall feature map output by the self-attention layer is upsampled by 4 times and converted into a feature map of the same size as the first local feature map output by the first deformable convolution layer. After the three feature maps of the same size are spliced, the features on the three resolution branches are fused using 1x1 convolution to obtain a fused crack feature map. The fused crack feature map can finely depict the local detail features and overall topological connection features of the cracks, thereby improving the accuracy of identifying complex cracks on the road surface.

[0074] The pavement crack detection method and device provided in the embodiment of the present application input the acquired crack image into a pre-built crack detection model, and use the crack detection model to output the crack detection result. The model uses three resolution branches to respectively extract the local detail features and the overall topological connection features of the cracks, and uses the VSS module instead of the convolution operation to enhance the model's ability to understand global information. The local slender morphology of the cracks is further enhanced by deformable convolution for the extracted local detail features. For the extracted overall topological connection features, the contextual semantic information is globally modeled based on the self-attention mechanism to enhance the overall complex morphology of the cracks. Finally, the features of multiple branches are fused through convolution. The fused crack feature map can accurately depict the local details and overall topological morphology of the cracks, thereby improving the recognition accuracy of complex pavement cracks.

[0075] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0076] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0077] like Figure 7 As shown, the embodiment of the present application also provides a road surface crack detection device, including:

[0078] An acquisition module, used for acquiring crack images;

[0079] The detection module is used to input the crack image into a pre-built crack detection model, and the crack detection model outputs the crack detection result; wherein the crack detection model includes a block processing module, a feature extraction module, and a feature fusion module. The block processing module is used to downsample the input crack image to obtain a crack feature map. The feature extraction module is used to extract the local slender morphological features and overall topological connection features of the crack based on the crack feature map to obtain a local feature map and an overall feature map. The feature fusion module is used to fuse the local feature map and the overall feature map to obtain a fused crack feature map.

[0080] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0081] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0082] Figure 8 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0083] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0084] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0085] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0086] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0087] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0088] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0089] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0090] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0091] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0092] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0093] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0094] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present disclosure.

Claims

1. A road crack detection method, characterized in that: include: Acquire crack images; The crack image is input into a pre-built crack detection model, and the crack detection model outputs the crack detection result; wherein the crack detection model includes a block processing module, a feature extraction module, and a feature fusion module, the block processing module is used to downsample the input crack image to obtain a crack feature map, the feature extraction module is used to extract the local slender morphological features and overall topological connection features of the crack based on the crack feature map to obtain a local feature map and an overall feature map, and the feature fusion module is used to fuse the local feature map and the overall feature map to obtain a fused crack feature map.

2. The method according to claim 1, characterized in that The feature extraction module includes a first high-resolution branch, a second high-resolution branch and a low-resolution branch; The first high-resolution branch is used to extract a local elongated morphological feature having a first scale based on the crack feature map and the feature map extracted by the second high-resolution branch, so as to obtain a first local feature map; The second high-resolution branch is used to extract local elongated morphological features with a second scale based on the feature map extracted by the first high-resolution branch, so as to obtain a second local feature map; The low-resolution branch is used to extract overall topological connection features with a third scale based on the feature maps extracted by the first high-resolution branch and the second high-resolution branch to obtain the overall feature map.

3. The method according to claim 2, characterized in that The first high-resolution branch includes a first VSS module, a second VSS module, and a third VSS module, the second high-resolution branch includes a fourth VSS module and a fifth VSS module, and the low-resolution branch includes a sixth VSS module; The input of the first VSS module is the crack feature map, the input of the second VSS module is the feature map output by the first VSS module, the input of the third VSS module is the feature map output by the second VSS module, and the feature map obtained by upsampling the feature map output by the fourth VSS module; The input of the fourth VSS module is a feature map obtained by downsampling the feature map output by the first VSS module, and the input of the fifth VSS module is a feature map output by the fourth VSS module and a feature map obtained by downsampling the feature map output by the second VSS module; The input of the sixth VSS module is a feature map obtained by downsampling the feature map output by the second VSS module and a feature map obtained by downsampling the feature map output by the fourth VSS module.

4. The method according to claim 3, characterized in that The first VSS module includes a patch embedding layer and a VSS block; The patch embedding layer is used to maintain the resolution of the input feature map. The feature map of the input VSS block enters the first branch and the second branch for processing respectively after layer normalization. The first branch performs convolution operation and Silu activation function processing on the feature map. The second branch performs convolution operation, batch normalization and Silu activation function processing on the feature map, and then inputs it into the SS2D module for processing. The feature map output by the SS2D module is layer normalized and then merged with the feature map output by the first branch by element-by-element dot product. The merged feature map is convolved to mix features, and the mixed feature map is fused with the feature map of the input VSS block.

5. The method according to claim 4, characterized in that The first layer of the second branch performs 1X1 convolution, batch normalization and Silu activation function on the feature map to obtain the feature map processed by the first layer. The feature map processed by the first layer enters the second layer, and each input channel is operated independently without mixing channel information through depth-wise separable convolution, thereby retaining the spatial information of the feature map of the first layer and obtaining the feature map processed by the second layer.

6. The method according to claim 4, characterized in that The second VSS module, the third VSS module, the fourth VSS module, the fifth VSS module and the sixth VSS module include a patch merging layer and a VSS block.

7. The method according to claim 2 or 3, characterized in that: The feature fusion module includes a first deformable convolution layer, a second deformable convolution layer and a self-attention layer; The first deformable convolution layer is used to enhance the first local feature map using deformable convolution to obtain an enhanced first local feature map, the second deformable convolution layer is used to enhance the second local feature map using deformable convolution to obtain an enhanced second local feature map, and the self-attention layer is used to enhance the overall feature map based on the self-attention mechanism to obtain an enhanced overall feature map.

8. The method according to claim 7, characterized in that The feature fusion module also includes a fully connected layer, which is used to convert the enhanced first local feature map, the second local feature map and the overall feature map into three feature maps of the same size, splice the three feature maps of the same size, and fuse the spliced ​​feature maps using 1x1 convolution to obtain the fused crack feature map.

9. The method according to claim 1, characterized in that: The block processing module is used to reduce the height of the crack image to a predetermined height, and extract the features of the crack image to obtain the crack feature map.

10. A road crack detection device, characterized in that: include: An acquisition module, used for acquiring crack images; A detection module is used to input the crack image into a pre-built crack detection model, and the crack detection model outputs the crack detection result; wherein the crack detection model includes a block processing module, a feature extraction module, and a feature fusion module, the block processing module is used to downsample the input crack image to obtain a crack feature map, the feature extraction module is used to extract the local slender morphological features and overall topological connection features of the crack based on the crack feature map to obtain a local feature map and an overall feature map, and the feature fusion module is used to fuse the local feature map and the overall feature map to obtain a fused crack feature map.

Images