Edge-enhanced pavement crack segmentation network processing method and system

Abstract

The invention relates to deep learning, and discloses an edge-enhanced pavement crack segmentation network processing method and system, so as to reduce information loss and improve segmentation accuracy. The method comprises the steps of image preprocessing, up-sampling, down-sampling, feature fusion and image restoration processing, and recursive aggregation processing is carried out after the image preprocessing, between two adjacent sampling layers and before the image restoration processing. According to the recursive aggregation processing, global context information is aggregated by scaling an input feature map, local details can be enhanced, information loss can be reduced, global modeling is realized based on an attention mechanism, and meanwhile, high-order space information and edge information are respectively extracted from the input feature map by applying recursive gated convolution and an edge operator; the global information capturing performance is greatly improved; therefore, the accuracy of the segmentation result is ensured.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an edge-enhanced road crack segmentation network processing method and system. Background Art

[0002] Pavement cracks are the most common external damage manifestation of highway infrastructure, which seriously affects the service life and safety performance of roads. Therefore, pavement crack detection is an important part of highway maintenance work and has great economic and social value. For a long time, pavement crack detection has mainly relied on visual inspection and empirical judgment by staff, which is not only inefficient but also prone to errors and poor accuracy. Compared with traditional time-consuming and highly professional manual inspection, automatic crack detection methods based on computer vision have attracted widespread attention due to their low cost and high efficiency. Specifically, deep learning is widely used to build crack detection networks through end-to-end learning methods, which improves detection accuracy and efficiency while reducing dependence on artificial features.

[0003] However, many existing deep learning-based segmentation networks do not pay enough attention to crack edge information, and there is a large amount of information loss in the feature fusion module, which leads to poor segmentation performance such as inaccurate crack location positioning and discontinuous segmentation of long cracks. Summary of the invention

[0004] The present invention aims to disclose an edge-enhanced pavement crack segmentation network processing method and system to reduce information loss and improve segmentation accuracy.

[0005] To achieve the above-mentioned purpose, the edge-enhanced road crack segmentation network processing method disclosed in the present invention includes image preprocessing, upsampling, downsampling, feature fusion and image restoration processing. Recursive aggregation processing is performed after image preprocessing, between two adjacent sampling layers and before image restoration processing. The recursive aggregation processing includes:

[0006] For a given input X in , a coarse-grained representative feature map X′ is obtained through deep convolution; then X′ is refined through large-kernel separable convolution, and the output of large-kernel separable convolution is:

[0007] Z 1 =W (2d-1)×1 *(W 1×(2d-1) *X ′ );

[0008]

[0009] Where d and k represent the dilation rate and the original convolution kernel size respectively; W represents the convolution operation; X′ is obtained by two depthwise convolutions with kernel sizes of 1×(2d-1) and (2d-1)×1. 1, and then Z 1 Through two kernel sizes and The dilated depth convolution gets Z 2 ; Afterwards, to Z 2 The representative feature map channel C is scaled to c by 1×1 convolution to obtain X map ;

[0010] Then input feature X in Reshape into Q, transform X map Reshape into K and V; generate the attention matrix A, where:

[0011] Q=W q (X in ),K=W k (X map ),V=W v (X map );

[0012] A=Softmax(K T Q);

[0013] X out =W p (A.V);

[0014] Among them, W q , W k , W v All are linear layers, which transform the corresponding input into Q, K, and V respectively; Softmax(·) represents the softmax function, and the Cartesian product of K and Q is performed to obtain the A matrix; the linear layer W p Used to ensure that the output dimension is H×W×C, where H, W, and C represent the height, width, and number of channels respectively; X out Represents the output after the cross-attention mechanism is processed;

[0015] Then for X out With X in The fused first feature is normalized to obtain the second feature, and then the edge refinement feedforward network processing is performed on the second feature. During the processing, edge operators and recursive gated convolutions are used to extract high-order spatial edge information.

[0016] Preferably, the specific process of the edge thinning feedforward network processing includes:

[0017] In the processing branch of the recursive gated convolution, a large core attention mechanism is applied to expand the receptive field. The large core attention mechanism includes 5×5 depth convolution, 7×7 depth expansion convolution and 1×1 convolution in sequence. In the other branch, the Sobel operator is divided into two parts: the horizontal part S x and the vertical part Sy :

[0018]

[0019] The feature map of the input edge detection branch is respectively extracted using horizontal and vertical Sobel filters to extract edge information, and then the extracted horizontal edge information is fused with the vertical edge information to obtain the branch output;

[0020] After the data of the two branches are fused, they are processed by 3×3 convolution, activation function, layer normalization and linear layer in sequence to obtain the third feature; finally, the third feature is fused with the first feature and used as the input of the next sampling module or the functional module for image restoration processing.

[0021] Preferably, in the process of fusion of features at all levels, after conventional fusion processing, the following steps are further included:

[0022] The feature map after conventional fusion processing is first layer normalized and then extracted with three branches consisting of an omnidirectional selective scanning module, a multi-scale deep convolution and a Laplacian operator to extract features of different scales. The linear layer then performs linear processing on the features after the three-branch fusion and then superimposes them with the feature map after conventional fusion processing as the input for the next recursive aggregation processing.

[0023] To achieve the above objectives, the present invention also discloses an edge-enhanced pavement crack segmentation network processing system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0024] The present invention has the following beneficial effects:

[0025] Through unique recursive aggregation processing, global context information is aggregated by scaling the input feature map, which can enhance local details and reduce information loss, and realize global modeling based on the attention mechanism. At the same time, recursive gated convolution and edge operators are used to extract high-order spatial information and edge information from the input feature map respectively, which greatly improves the global information capture performance; thereby ensuring the accuracy of the segmentation results.

[0026] The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0028] Figure 1 It is a diagram of the REM-Net architecture disclosed in an embodiment of the present invention.

[0029] Figure 2 In (a), Figure 1 (b) and (c) are the internal architecture diagrams corresponding to the two functional sub-modules of the recursive large core attention mechanism and the edge refinement feedforward network in (a).

[0030] Figure 3 In the figure, (a) is a schematic diagram of the logical architecture comparing the large-core recursive gated convolution disclosed in an embodiment of the present invention with the existing gated convolution, and (b) is an internal architecture diagram corresponding to the large-core attention submodule in (a).

[0031] Figure 4 It is a visualization diagram of the REM-Net feature extraction and fusion process disclosed in an embodiment of the present invention.

[0032] Figure 5 In the figure, (a) is a schematic diagram of the architecture of the multi-scale fusion Mamba disclosed in the embodiment of the present invention, and (b) is Figure 5 (a) Internal architecture diagram corresponding to the OSSM functional sub-module. DETAILED DESCRIPTION

[0033] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0034] Example 1

[0035] This embodiment discloses an edge-enhanced road crack segmentation network processing method.

[0036] For ease of description, this embodiment defines the functional module for performing recursive aggregation processing as a recursive aggregation transformer (RAT), the large kernel separable convolution is expressed as LKSC, the edge refinement feedforward network is expressed as ERFN, the functional module for performing feature fusion is expressed as MSFM, and the entire network is expressed as a pixel-level crack segmentation network (REM-Net).

[0037] This embodiment can be summarized as follows: by integrating the global Transformer and the state space model (SSM), an edge-enhanced pavement crack precise segmentation network REM-Net is disclosed. Specifically, it includes: designing a recursive large kernel attention mechanism (RLKA) to refine features through recursive calls and large kernel separable convolutions to achieve global information extraction. At the same time, an edge refinement feedforward network (ERFN) is established to extract high-order spatial edge information using edge operators and recursive convolutions. In addition, a multi-scale fusion Mamba (MSFM) is designed to aggregate the output features of the encoder and decoder to compensate for the feature loss of downsampling. Finally, the performance of REM-Net can be evaluated through experiments on a public crack segmentation dataset. Details are as follows:

[0038] In order to meet the current challenges of crack segmentation, it is necessary to achieve the effective fusion of edge high-frequency information and global contextual information. To this end, REM-Net in this embodiment is based on the U-Net framework, and Transformer is used in the construction of both the encoder and the decoder to fully capture the long-distance dependencies between sample features. At the same time, MSFM is integrated into the jump connection to alleviate the loss of semantic information caused by multi-layer feature extraction; therefore, this embodiment uses multi-scale deep separable convolution to enhance local information, uses the Mamba module to enhance global semantic understanding, and applies the Laplacian operator to enrich edge high-frequency details. These enhancements promote the effective interaction between high-level and low-level features and reduce the loss of semantic information. The core architecture of the network is as follows Figure 1 and Figure 2 shown.

[0039] In order to improve the quadratic linear complexity brought by Transformer, this embodiment uses the method of extracting representative feature maps to perform deep feature extraction on the crack area in the core codec architecture. The preprocessed crack image is initially processed using a convolutional layer. Subsequently, the processed samples are input to the coding layer for feature extraction, and each coding layer contains a RAT and a downsampling module. The network uses four coding layers to enhance and extract crack information. The last coding layer module serves as a bottleneck layer to establish a connection between the encoder and the decoder. Similarly, in addition to the RAT module, the decoder also contains an upsampling module that recovers and extracts features from the feature map. Finally, a convolutional layer is applied to resize the feature map to its original size and generate the final prediction.

[0040] like Figure 2 As shown in (a), the RAT module consists of two parts, namely the recursive large kernel attention module (RLKA) and the edge refinement feedforward network (ERFN). This module comprehensively extracts the features of the crack area with linear complexity. By enhancing the crack edges using edge operators, the focus of the network is effectively shifted from the non-crack area, thereby significantly improving the detection accuracy of the network. Jump connections are required to alleviate information loss during encoder feature extraction and decoder feature recovery. Therefore, this embodiment incorporates a multi-scale fusion module (MSFM) into the jump connection to process the fused feature map. This module merges the output of each encoding layer with the corresponding output feature map in the decoding layer to improve the efficiency of overall feature representation and semantic information transmission.

[0041] The internal data processing logic of the RAT module is detailed as follows:

[0042] Modeling long sequence dependencies is critical for determining crack locations and identifying cracks. Transformer is able to establish connections within a sequence of image blocks, allowing for comprehensive global modeling. Although Transformers are good at extracting global information, their quadratic computational complexity limits their application in crack segmentation. Some Transformer variants use window settings to reduce computational complexity, but such methods may not fully utilize the Transformer's original global information capture performance. To alleviate this problem, this embodiment proposes a core backbone network called Recursive Aggregation Transformer (RAT). RAT consists of RLKA, ERFN, LayerNorm layers, and residual connections. Figure 2 As shown in (b), multiple deep convolutions are applied to extract representative feature maps, which will lead to significant local information loss, thus limiting the subsequent global modeling work. To solve this problem, this embodiment applies large kernel separable convolution (LKSC) and point convolution to accurately refine the representative feature map to enhance local details and reduce information loss. The representative feature map aggregates the detailed information in the entire input feature map and provides a global view. Subsequently, the input feature map interacts with the representative feature map and calculates their cross attention to achieve global modeling. Specifically, for a given input X in ∈R H×W×C , X in First proceed Sub-depth convolution to obtain a coarse-grained representative feature map X′∈R h×w×C , the whole process can be described as:

[0043]

[0044] Among them, W s is a 3×3 depth convolution, s is the convolution step size, h is a configurable constant, In this embodiment, since H=W of the preprocessed image, h=w.

[0045] Then, this embodiment refines X′ through large-core separable convolution. Since large-core convolution has high computational complexity, it is difficult to embed into the network. To this end, this embodiment introduces large-core separable convolution, which splits the convolution into four parts to reduce computational complexity while maintaining the original receptive field. The output of large-core separable convolution is:

[0046] Z 1 =W (2d-1)×1 *(W 1×(2d-1) *X ′ );

[0047]

[0048] Where d and k represent the dilation rate and the original convolution kernel size, respectively. W represents the convolution operation. X′ is obtained by two depthwise convolutions with kernel sizes of 1×(2d-1) and (2d-1)×1. 1 . Through two kernels of size and The dilated depth convolution gets Z 2 Finally, this embodiment uses 1×1 convolution to scale the representative feature map channel C to c to obtain X map ∈R h ×w×c , further reducing the computational complexity and eliminating channel redundancy. Then, this embodiment inputs feature X in Reshape into Q∈R H ×W×c (query), X map Reshape into K∈R h×w×c (key) and V∈R h×w×C (value) and calculate their cross attention. The interaction between query and key generates the attention matrix A∈R HW×hw The above process can be summarized as follows:

[0049] Q=W q (X in ),K=W k (X map ),V=W v (X map );

[0050] A=S oft max(K T Q);

[0051] X out =W p (A.V);

[0052] Among them, W q , W k , W v All are linear layers, which transform the corresponding input into Q, K, and V respectively. Softmax(·) represents the softmax function, which performs a Cartesian product of K and Q to obtain the A matrix. Linear layer W p Ensure that the output dimensions are H×W×C. out Represents the output of RLKA.

[0053] Traditional feedforward networks (FFNs) usually consist of two fully connected layers and two nonlinear activation functions. These components enhance the network's position representation capability by performing dimensional transformation and nonlinear activation on the input tokens. However, the input features have high-order spatial features and rich edge information, and a single FFN cannot fully represent these complex details. To solve this problem, this embodiment designs an edge refinement feedforward network (ERFN) based on FFN to enhance the representation of crack locations. Figure 2 As shown in (c), ERFN combines refined recursive gated convolution and Sobel edge detection operators in parallel, as well as linear layers, normalization layers, and activation functions. By applying large kernel recursive gated convolution (LK-gnConv), local features are further refined to capture deep spatial semantic information. Subsequently, the Sobel operator runs in parallel to enhance high-frequency edge information, making up for the lack of local spatial high-order semantic information and edge information enhancement in RAT.

[0054] Preferably, if Figure 3 As shown, this embodiment applies the large kernel attention (LKA) mechanism in the edge refinement feedforward network to replace the existing g n The 7×7 depth convolution in Conv results in the LK-g n Conv large kernel recursive gated convolution; thereby reducing the information loss caused by ordinary large kernel convolution and expanding the receptive field to improve performance. In another branch, this embodiment uses the Sobel operator for edge enhancement. In this embodiment, the Sobel operator is divided into two parts: the horizontal part S x and the vertical part S y .

[0055]

[0056] Let X s To input the feature map of the edge detection branch, this embodiment uses horizontal and vertical Sobel filters to extract edge information, as shown below.

[0057]

[0058] M=M x +M y ;

[0059] Among them, W 1×1 is a regular convolution with a kernel size of 1, and The kernel matrix is ​​S x and S y The extracted horizontal edge information M x With vertical edge information M y Fusion, to get the final output M.

[0060] The backbone network in this embodiment is able to capture deep spatial global features and local edge details from various crack regions. However, a lot of information is lost in the process of deep feature extraction. Therefore, it is necessary to effectively fuse features at different levels to achieve context information exchange between the encoder and decoder; the visualization diagram of REM-Net feature extraction and fusion process can be referred to Figure 4 Furthermore, this embodiment designs a multi-scale fusion Mamba (MSFM) to achieve feature aggregation. The internal data processing logic of the MSFM functional module is as follows:

[0061] like Figure 5 As shown in the figure, MSFM is designed as a multi-branch architecture that combines an omnidirectional selective scanning module (OSSM), multi-scale deep convolution, Laplacian operator, batch normalization, and linear layers. The OSSM component decouples the crack information and enhances the modeling of long-range dependencies in the fused features. At the same time, the multi-scale deep convolution refines the local information by processing the fused features with different convolution kernels. In addition, a second-order edge operator is introduced in MSFM to enrich the edge details in the fused features. In summary, MSFM effectively promotes the information exchange between different feature levels within REM-Net to refine and enhance the crack features, significantly alleviating the information loss caused by the network depth.

[0062] The conventional Mamba scanning mechanism is not comprehensive and cannot cope with complex and diverse crack features. In the prior art, Vim introduced horizontal scanning; Vmamba optimized the scanning mechanism by combining scanning in the horizontal and vertical directions. Due to the complexity and irregularity of cracks, simple directional scanning is not sufficient to extract features. Therefore, this embodiment uses OSSM with comprehensive scanning capabilities to extract large-scale crack features from multiple angles to capture the spatial representation of complex cracks. Initially, OSSM scans the input image in horizontal, vertical, diagonal, anti-diagonal directions and their opposite directions to generate 8 feature sequences. Subsequently, the aforementioned 8 feature sequences are connected in a new dimension and input into the S6 block. The S6 block adopts a unique selective scanning mechanism to model global feature information from input sequences in all directions. Specifically, the calculation method of the S6 block is as follows:

[0063] h ′ (t) = Ph(t) + Qx(t);

[0064] y(t)=Uh(t)+Vx(t);

[0065] Among them, the evolution parameter P∈R N×N , projection parameter Q,U∈R N, the jump connection parameter V∈R. N is the length of the input feature sequence. h(t) is the state vector from input x(t) to output y(t). In addition, in the deep learning network, the discretization process is:

[0066]

[0067] in They represent the discretization of P, Q, U, and V respectively, △ represents the time scale parameter, and h t-1 and h t Represent the state of the previous time step and the state of the current time step respectively.

[0068] In the convolution branch, this embodiment uses multi-scale deep convolution to divide the input features into different segments according to the channel dimension. Each segment undergoes convolution of different sizes. Subsequently, these segments are merged to produce fine local features. This embodiment specifies convolution kernels of [1×1, 3×3, 5×5, 7×7, 9×9, 11×11] to enhance the fine granularity of local features. In the edge detection branch, this embodiment uses the Laplacian operator to calculate the quadratic gradient of the edge details extracted from the backbone to enhance edge modeling. The outputs of the three branches are integrated and coordinated through a linear layer to generate a fused feature input for the REM-Net decoder.

[0069] In this embodiment, the loss function can quantify the deviation between the predicted result and the actual result, which is very important when training a deep learning network. The loss function needs to be configured reasonably, especially in the crack segmentation task. Crack segmentation is a binary classification task, in which pixels are divided into crack pixels and background pixels, represented by 0 and 1, respectively. It is worth noting that the number of background pixels and crack pixels is seriously unbalanced, and crack pixels only account for a small proportion of the image. To solve this problem, in REM-Net, this embodiment sets the loss function to binary cross entropy (BCE) loss and DICE loss. BCE loss is specially designed for binary classification tasks, focusing on how to accurately classify crack pixels. DICE loss enhances the network's attention to crack areas during training to effectively solve the problem of unbalanced distribution of crack and background pixel ratios. The loss function of this embodiment is specifically designed as:

[0070] BCELoss(p,t)=-(1-t)log(1-p)-tlog(p);

[0071]

[0072] Loss(p,t)=a*BCELoss(p,t)+b*DICELoss(p,t);

[0073] Among them, p represents the prediction result of the crack image, t represents the true value label; p i represents the prediction result of the i-th pixel, t i represents the true value of the i-th pixel; N represents the total number of pixels in the crack image; ε is the smoothing factor, which is set to 1 to prevent the denominator from being predicted as zero; a and b are two constants used to control the proportion of BCE loss and DICE loss in the entire loss function.

[0074] In summary, the crack segmentation network REM-Net disclosed in this embodiment is composed of an encoder constructed by RAT and a jump connection integrated with MSFM to achieve comprehensive global feature extraction and edge information modeling. RAT aggregates global context information by scaling the input feature map and uses edge operators to accurately locate the crack position; MSFM uses multiple operators to extract global information and edge information from the input feature map to promote the fusion of features of the same scale, thereby improving the efficiency of overall feature representation and semantic information transmission; thereby reducing information loss and improving segmentation accuracy.

[0075] Example 2

[0076] Corresponding to the above embodiment, this embodiment discloses an edge-enhanced road crack segmentation network processing system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above method is implemented. The method includes image preprocessing, upsampling, downsampling, feature fusion, and image restoration processing. Recursive aggregation processing is performed after image preprocessing, between two adjacent sampling layers, and before image restoration processing. The recursive aggregation processing includes:

[0077] For a given input X in , obtain a coarse-grained representative feature map X′ through deep convolution (preferably through the method based on recursive calling of multiple deep convolutions described in the above embodiment); then refine X′ through large-kernel separable convolution, and the output of the large-kernel separable convolution is:

[0078] Z 1 =W (2d-1)×1 *(W 1×(2d-1) *X ′ );

[0079]

[0080] Where d and k represent the dilation rate and the original convolution kernel size respectively; W represents the convolution operation; X′ is obtained by two depthwise convolutions with kernel sizes of 1×(2d-1) and (2d-1)×1. 1 , and then Z 1 Through two kernel sizes and The dilated depth convolution gets Z 2 ; Afterwards, to Z 2 The representative feature map channel C is scaled to c by 1×1 convolution to obtain X map .

[0081] Then input feature X in Reshape into Q, transform X map Reshape into K and V; generate the attention matrix A, where:

[0082] Q=W q (X in ),K=W k (X map ),V=W v (X map );

[0083] A=Softmax(K T Q);

[0084] X out =W p (A.V);

[0085] Among them, W q , W k , W v All are linear layers, which transform the corresponding input into Q, K, and V respectively; Softmax(·) represents the softmax function, and the Cartesian product of K and Q is performed to obtain the A matrix; the linear layer W p Used to ensure that the output dimension is H×W×C, where H, W, and C represent the height, width, and number of channels respectively; X out Represents the output after the cross-attention mechanism.

[0086] Then for X out With X in The fused first feature is normalized to obtain the second feature, and then the edge refinement feedforward network processing is performed on the second feature. During the processing, edge operators and recursive gated convolutions are used to extract high-order spatial edge information.

[0087] Preferably, the specific process of the above-mentioned edge thinning feedforward network processing includes:

[0088] In the processing branch of the recursive gated convolution, a large core attention mechanism is applied to expand the receptive field. The large core attention mechanism includes 5×5 depth convolution, 7×7 depth expansion convolution and 1×1 convolution in sequence. In the other branch, the Sobel operator is divided into two parts: the horizontal part S x and the vertical part S y :

[0089]

[0090] The edge information of the feature map of the input edge detection branch is extracted using horizontal and vertical Sobel filters respectively, and then the extracted horizontal edge information is fused with the vertical edge information to obtain the branch output.

[0091] After the data of the two branches are fused, they are processed by 3×3 convolution, activation function, layer normalization and linear layer in sequence to obtain the third feature; finally, the third feature is fused with the first feature and used as the input of the next sampling module or the functional module for image restoration processing.

[0092] Preferably, in the process of feature fusion at each level, after the conventional fusion processing, it also includes: the feature map after the conventional fusion processing is first subjected to layer normalization processing and then the three branches consisting of an omnidirectional selective scanning module, a multi-scale deep convolution and a Laplace operator are used to extract features of different scales, and then the linear layer is used to linearly process the features after the fusion of the three branches and then superimposed with the feature map after the conventional fusion processing as the input for the next recursive aggregation processing.

[0093] In summary, the method and system disclosed in the above two embodiments of the present invention respectively, through unique recursive aggregation processing, can enhance local details and reduce information loss by scaling the input feature map to aggregate global context information, and realize global modeling based on the attention mechanism. At the same time, recursive gated convolution and edge operators are used to extract high-order spatial information and edge information from the input feature map, respectively, thereby greatly improving the global information capture performance; thereby ensuring the accuracy of the segmentation results.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A network processing method for edge-enhanced pavement crack segmentation, comprising image preprocessing, upsampling, downsampling, feature fusion and image restoration processing, characterized in that: After image preprocessing, recursive aggregation processing is performed between two adjacent sampling layers and before image restoration processing, and the recursive aggregation processing includes: For a given input X in , a coarse-grained representative feature map X′ is obtained through deep convolution; then X′ is refined through large-kernel separable convolution, and the output of large-kernel separable convolution is: Z1=W (2d-1)×1 *(IN 1×(2d-1) *X′); Where d and k represent the dilation rate and the original convolution kernel size, respectively; W represents the convolution operation; X′ is obtained by two depthwise convolutions with kernel sizes of 1×(2d-1) and (2d-1)×1 to obtain Z1, and then Z1 is convolved through two kernel sizes of and The expanded depth convolution is performed to obtain Z2; then, the representative feature map channel C is scaled to c by 1×1 convolution on Z2 to obtain X map ; Then input feature X in Reshape into Q, transform X map Reshape into K and V; generate the attention matrix A, where: Q=W q (X in ),K=W k (X map ),V=W v (X map ); A=Softmax(K T Q); X out =W p (A·V); Among them, W q , W k , W v All are linear layers, which transform the corresponding input into Q, K, and V respectively; Softmax(·) represents the softmax function, and the Cartesian product of K and Q is performed to obtain the A matrix; the linear layer W p Used to ensure that the output dimension is H×W×C, where H, W, and C represent the height, width, and number of channels respectively; X out Represents the output after the cross-attention mechanism is processed; Then for X out With X in The fused first feature is normalized to obtain the second feature, and then the edge refinement feedforward network processing is performed on the second feature. During the processing, edge operators and recursive gated convolutions are used to extract high-order spatial edge information.

2. The method according to claim 1, characterized in that The specific process of the edge refinement feedforward network processing includes: In the processing branch of the recursive gated convolution, a large core attention mechanism is applied to expand the receptive field. The large core attention mechanism includes 5×5 depth convolution, 7×7 depth expansion convolution and 1×1 convolution in sequence. In the other branch, the Sobel operator is divided into two parts: the horizontal part S x and the vertical part S y : The feature map of the input edge detection branch is respectively extracted using horizontal and vertical Sobel filters to extract edge information, and then the extracted horizontal edge information is fused with the vertical edge information to obtain the branch output; After the data of the two branches are fused, they are processed by 3×3 convolution, activation function, layer normalization and linear layer in sequence to obtain the third feature; finally, the third feature is fused with the first feature and used as the input of the next sampling module or the functional module for image restoration processing.

3. The method according to claim 1 or 2, characterized in that: In the process of feature fusion at all levels, after the conventional fusion processing, it also includes: The feature map after conventional fusion processing is first layer normalized and then extracted with three branches consisting of an omnidirectional selective scanning module, a multi-scale deep convolution and a Laplacian operator to extract features of different scales. The linear layer then performs linear processing on the features after the three-branch fusion and then superimposes them with the feature map after conventional fusion processing as the input for the next recursive aggregation processing.

4. An edge-enhanced pavement crack segmentation network processing system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 3 is implemented.

Images