Edge-enhanced pavement crack segmentation network processing method and system

Abstract

This invention relates to deep learning and discloses a network processing method and system for edge-enhanced pavement crack segmentation to reduce information loss and improve segmentation accuracy. The method includes image preprocessing, upsampling, downsampling, feature fusion, and image restoration. Recursive aggregation is performed after image preprocessing, between adjacent sampling layers, and before image restoration. This recursive aggregation aggregates global contextual information by scaling the input feature map, enhancing local details and reducing information loss. Global modeling is achieved based on an attention mechanism. Recursive gated convolution and edge operators are used to extract high-order spatial and edge information from the input feature map, respectively, significantly improving global information capture performance and ensuring the accuracy of the segmentation results.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an edge-enhanced pavement crack segmentation network processing method and system. Background Art

[0002] Pavement cracks are the most common external damage manifestation of highway infrastructure, seriously affecting the service life and safety performance of roads. Therefore, pavement crack detection is an important component of highway maintenance and has significant economic and social value. For a long time, pavement crack detection has relied primarily on visual inspection and empirical judgment by personnel, which is not only inefficient but also prone to errors and poor accuracy. Compared with traditional, time-consuming and highly specialized manual inspection, automatic crack detection methods based on computer vision have attracted widespread attention due to their low cost and high efficiency. Specifically, deep learning is widely used to construct crack detection networks through end-to-end learning methods, improving detection accuracy and efficiency while reducing reliance on artificial features.

[0003] However, many existing deep learning-based segmentation networks do not pay enough attention to crack edge information, and there is a large amount of information loss in the feature fusion module, resulting in inaccurate crack location positioning, discontinuous segmentation of long cracks and other poor segmentation performance problems. Summary of the Invention

[0004] The present invention aims to disclose an edge-enhanced pavement crack segmentation network processing method and system to reduce information loss and improve segmentation accuracy.

[0005] To achieve the above objectives, the present invention discloses an edge-enhanced pavement crack segmentation network processing method, which includes image preprocessing, upsampling, downsampling, feature fusion, and image restoration processing. Recursive aggregation processing is performed after image preprocessing, between two adjacent sampling layers, and before image restoration processing. The recursive aggregation processing includes:

[0006] For a given input X in , a coarse-grained representative feature map X′ is obtained through deep convolution; then X′ is refined through large kernel separable convolution, and the output of large kernel separable convolution is:

[0007] Z1=W (2d-1)×1 *(W 1×(2d-1) *X ′ );

[0008]

[0009] Where d and k represent the dilation rate and the original convolution kernel size respectively; W represents the convolution operation; X′ is obtained by two depthwise convolutions with kernel sizes of 1×(2d-1) and (2d-1)×1 to obtain Z1, and then Z1 is convolved through two kernel sizes of and The expanded depth convolution is performed to obtain Z2; then, the representative feature map channel C is scaled to c by 1×1 convolution on Z2 to obtain X map ;

[0010] Then input feature X in Reshape into Q, transform X map Reshape into K and V; generate the attention matrix A, where:

[0011] Q=W q (X in ),K=W k (X map ),V=W v (X map );

[0012] A=Softmax(K T Q);

[0013] X out =W p (A·V);

[0014] Among them, W q , W k , W v They are all linear layers, transforming the corresponding inputs into Q, K, and V respectively; Softmax(·) represents the softmax function, and the Cartesian product of K and Q is obtained to obtain the A matrix; the linear layer W p Used to ensure that the output dimension is H×W×C, where H, W, and C represent the height, width, and number of channels respectively; X out Represents the output after cross-attention mechanism processing;

[0015] Then for X out With X in The fused first feature is normalized to obtain the second feature, and then the edge refinement feedforward network processing is performed on the second feature. During the processing, the edge operator and recursive gated convolution are used to extract high-order spatial edge information.

[0016] Preferably, the specific process of the edge thinning feedforward network processing includes:

[0017] In the processing branch of recursive gated convolution, a large core attention mechanism is applied to expand the receptive field, which includes 5×5 depth convolution, 7×7 depth expansion convolution and 1×1 convolution in sequence; in the other branch, the Sobel operator is divided into two parts: the horizontal part S x and the vertical part S y :

[0018]

[0019] The edge information of the input edge detection branch feature map is extracted using horizontal and vertical Sobel filters respectively, and then the extracted horizontal edge information is fused with the vertical edge information to obtain the branch output;

[0020] After the data of the two branches are fused, they are processed in sequence through 3×3 convolution, activation function, layer normalization and linear layer to obtain the third feature; finally, the third feature is fused with the first feature and used as the input of the next sampling module or image restoration processing function module.

[0021] Preferably, in the process of fusion of features at all levels, after conventional fusion processing, the following steps are further included:

[0022] The feature map after conventional fusion processing is first layer-normalized and then three branches consisting of an omnidirectional selective scanning module, a multi-scale deep convolution and a Laplacian operator are used to extract features of different scales. The features after the three-branch fusion are then linearly processed by a linear layer and superimposed with the feature map after conventional fusion processing as the input for the next recursive aggregation processing.

[0023] To achieve the above objectives, the present invention also discloses an edge-enhanced pavement crack segmentation network processing system, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above method is implemented.

[0024] The present invention has the following beneficial effects:

[0025] Through unique recursive aggregation processing, global context information is aggregated by scaling the input feature map, which can enhance local details and reduce information loss, and realize global modeling based on the attention mechanism. At the same time, recursive gated convolution and edge operators are applied to extract high-order spatial information and edge information from the input feature map respectively, which greatly improves the global information capture performance; thereby ensuring the accuracy of the segmentation results.

[0026] The present invention will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0028] Figure 1 This is a diagram of the REM-Net architecture disclosed in an embodiment of the present invention.

[0029] Figure 2 In the equation (a), Figure 1(b) and (c) are the internal architecture diagrams corresponding to the two functional sub-modules of the recursive large core attention mechanism and the edge refinement feedforward network in (a).

[0030] Figure 3 In the figure, (a) is a schematic diagram of the logical architecture comparing the large-core recursive gated convolution disclosed in an embodiment of the present invention with the existing gated convolution, and (b) is the internal architecture diagram corresponding to the large-core attention submodule in (a).

[0031] Figure 4 It is a visual diagram of the REM-Net feature extraction and fusion process disclosed in an embodiment of the present invention.

[0032] Figure 5 In the figure, (a) is a schematic diagram of the architecture of the multi-scale fusion Mamba disclosed in the embodiment of the present invention, and (b) is Figure 5 (a) Internal architecture diagram corresponding to the OSSM functional sub-module. DETAILED DESCRIPTION

[0033] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered by the claims.

[0034] Example 1

[0035] This embodiment discloses an edge-enhanced pavement crack segmentation network processing method.

[0036] For ease of description, this embodiment defines the functional module for performing recursive aggregation processing as a recursive aggregation transformer (RAT), the large kernel separable convolution as LKSC, the edge refinement feedforward network as ERFN, the functional module for performing feature fusion as MSFM, and the entire network as a pixel-level crack segmentation network (REM-Net).

[0037] This embodiment can be summarized as follows: by integrating the global Transformer and the state space model (SSM), an edge-enhanced road crack accurate segmentation network REM-Net is disclosed. Specifically, it includes: designing a recursive large kernel attention mechanism (RLKA) to refine features through recursive calls and large kernel separable convolutions to achieve global information extraction. At the same time, an edge refinement feedforward network (ERFN) is established to extract high-order spatial edge information using edge operators and recursive convolutions. In addition, a multi-scale fusion Mamba (MSFM) is designed to aggregate the output features of the encoder and decoder to compensate for the feature loss of downsampling. Finally, the performance of REM-Net can be evaluated through experiments on a public crack segmentation dataset. Details are as follows:

[0038] In order to meet the current challenges of crack segmentation, it is necessary to achieve an effective fusion of edge high-frequency information and global contextual information. To this end, the REM-Net of this embodiment is based on the U-Net framework, and uses Transformer in the construction of both the encoder and the decoder to fully capture the long-distance dependencies between sample features. At the same time, MSFM is integrated into the jump connection to alleviate the loss of semantic information caused by multi-layer feature extraction; therefore, this embodiment uses multi-scale depth-separable convolution to enhance local information, uses the Mamba module to enhance global semantic understanding, and applies the Laplacian operator to enrich edge high-frequency details. These enhancements promote the effective interaction between high-level and low-level features and reduce the loss of semantic information. The core architecture of the network is as follows Figure 1 and Figure 2 shown.

[0039] To improve the quadratic linear complexity introduced by the Transformer, this embodiment uses a method to extract representative feature maps to perform deep feature extraction on crack regions within the core codec architecture. The preprocessed crack image undergoes preliminary coarse processing using a convolutional layer. The processed samples are then input into the coding layer for feature extraction. Each coding layer includes a RAT and a downsampling module. The network uses four coding layers to enhance and extract crack information. The final coding layer module serves as a bottleneck layer, establishing a connection between the encoder and decoder. Similarly, in addition to the RAT module, the decoder also includes an upsampling module that recovers and extracts features from the feature map. Finally, a convolutional layer is applied to resize the feature map to its original size and produce the final prediction.

[0040] like Figure 2 As shown in (a), the RAT module consists of two parts, namely the recursive large kernel attention module (RLKA) and the edge refinement feedforward network (ERFN). This module comprehensively extracts the features of the crack area with linear complexity. By utilizing the edge operator to enhance the crack edge, the focus of the network is effectively shifted from the non-crack area, thereby significantly improving the detection accuracy of the network. Jump connections are required to reduce information loss in the encoder feature extraction and decoder feature recovery processes. Therefore, this embodiment incorporates the multi-scale fusion module (MSFM) into the jump connection to process the fused feature map. This module merges the output of each encoding layer with the corresponding output feature map in the decoding layer to improve the efficiency of overall feature representation and semantic information transmission.

[0041] The internal data processing logic of the RAT module is detailed as follows:

[0042] Modeling long sequence dependencies is crucial for determining crack locations and identifying cracks. Transformer is able to establish connections within a sequence of image blocks, allowing for comprehensive global modeling. Although Transformers are good at extracting global information, their quadratic computational complexity limits their application in crack segmentation. Some Transformer variants use window settings to reduce computational complexity, but such methods may not fully utilize the Transformer's original global information capture performance. To alleviate this problem, this embodiment proposes a core backbone network called Recursive Aggregation Transformer (RAT). RAT consists of RLKA, ERFN, LayerNorm layers, and residual connections. Figure 2 As shown in (b), multiple depthwise convolutions are applied to extract representative feature maps, which results in significant local information loss, thus limiting subsequent global modeling work. To address this issue, this embodiment applies large kernel separable convolution (LKSC) and point convolution to accurately refine the representative feature maps to enhance local details and reduce information loss. The representative feature maps aggregate detailed information in the entire input feature map and provide a global view. Subsequently, the input feature map interacts with the representative feature map and their cross attention is calculated to achieve global modeling. Specifically, for a given input X in ∈R H×W×C , X in First proceed Sub-depth convolution to obtain coarse-grained representative feature map X′∈R h×w×C , the whole process can be expressed as:

[0043]

[0044] Among them, W s is a 3×3 depth convolution, s is the convolution step size, h is a configurable constant, In this embodiment, since H=W of the pre-processed image, h=w.

[0045] This embodiment then refines X′ through large-core separable convolution. Since large-core convolution has high computational complexity, it is difficult to embed into the network. To this end, this embodiment introduces large-core separable convolution, splitting the convolution into four parts to reduce computational complexity while maintaining the original receptive field. The output of large-core separable convolution is:

[0046] Z1=W (2d-1)×1 *(W 1×(2d-1) *X ′ );

[0047]

[0048] Where d and k represent the dilation rate and the original convolution kernel size, respectively. W represents the convolution operation. X′ is obtained by two depthwise convolutions with kernel sizes of 1×(2d-1) and (2d-1)×1 to obtain Z1. and Finally, this embodiment uses 1×1 convolution to scale the representative feature map channel C to c to obtain X map ∈R h ×w×c , further reducing computational complexity and eliminating channel redundancy. Subsequently, this embodiment inputs feature X in Reshape into Q∈R H ×W×c (query), X map Reshape into K∈R h×w×c (key) and V∈R h×w×C (value) and calculate their cross attention. The interaction between query and key generates the attention matrix A∈R HW×hw The above process can be summarized as follows:

[0049] Q=W q (X in ),K=W k (X map ),V=W v (X map );

[0050] A=S oft max(K T Q);

[0051] X out =W p (A·V);

[0052] Among them, W q , W k , W v They are all linear layers, transforming the corresponding inputs into Q, K, and V respectively. Softmax(·) represents the softmax function, which performs a Cartesian product of K and Q to obtain the A matrix. Linear layer W p Ensure the output dimensions are H×W×C. out Represents the output of RLKA.

[0053] Traditional feedforward networks (FFNs) usually consist of two fully connected layers and two nonlinear activation functions. These components enhance the network's position representation capability by performing dimensionality transformation and nonlinear activation on the input tokens. However, the input features have high-order spatial features and rich edge information, and a single FFN cannot fully represent these complex details. To solve this problem, this embodiment designs an edge refinement feedforward network (ERFN) based on FFN to enhance the representation of crack locations. Figure 2 As shown in (c), ERFN combines refined recursive gated convolutions and Sobel edge detection operators in parallel, along with linear layers, normalization layers, and activation functions. Large kernel recursive gated convolutions (LK-gnConv) are applied to further refine local features to capture deep spatial semantic information. Subsequently, the Sobel operator is run in parallel to enhance high-frequency edge information, compensating for the RAT's lack of local spatial high-order semantic information and edge information enhancement.

[0054] Preferably, if Figure 3 As shown, this embodiment applies the large kernel attention (LKA) mechanism in the edge refinement feedforward network to replace the existing g n The 7×7 depth convolution in Conv is obtained, which is named LK-g in this embodiment. n Conv's large kernel recursive gated convolution; thereby reducing the information loss caused by ordinary large kernel convolution and expanding the receptive field to improve performance. In another branch, this embodiment uses the Sobel operator for edge enhancement. In this embodiment, the Sobel operator is divided into two parts: the horizontal part S x and the vertical part S y .

[0055]

[0056] Let X s To input the feature map of the edge detection branch, this embodiment uses horizontal and vertical Sobel filters to extract edge information, as shown below.

[0057]

[0058] M=M x +M y ;

[0059] Among them, W 1×1 is a regular convolution with a kernel size of 1, and The kernel matrix is ​​S x and S y The extracted horizontal edge information M x With vertical edge information M y Fusion, get the final output M.

[0060] The backbone network in this embodiment is able to capture profound spatial global features and local edge details from various crack regions. However, a large amount of information is lost during the deep feature extraction process. Therefore, it is necessary to effectively fuse features at different levels to achieve contextual information exchange between the encoder and decoder. The visualization diagram of the REM-Net feature extraction and fusion process can be found in Figure 4 Furthermore, this embodiment designs a multi-scale fusion Mamba (MSFM) to achieve feature aggregation. The internal data processing logic of the MSFM functional module is detailed as follows:

[0061] like Figure 5 As shown in the figure, MSFM is designed as a multi-branch architecture that combines an omnidirectional selective sweep module (OSSM), multi-scale deep convolution, Laplacian operator, batch normalization, and linear layers. The OSSM component decouples crack information and enhances the modeling of long-range dependencies in the fused features. At the same time, the multi-scale deep convolution refines local information by processing the fused features with different convolution kernels. In addition, a second-order edge operator is introduced in MSFM to enrich the edge details in the fused features. In summary, MSFM effectively promotes information exchange between different feature levels within REM-Net to refine and enhance crack features, significantly mitigating the information loss caused by network depth.

[0062] The conventional Mamba scanning mechanism is not comprehensive and cannot cope with complex and diverse crack features. In the prior art, Vim introduced horizontal scanning; Vmamba optimized the scanning mechanism by combining horizontal and vertical scanning. Due to the complexity and irregularity of cracks, simple directional scanning is not sufficient to extract features. Therefore, this embodiment uses OSSM with comprehensive scanning capabilities to extract large-scale crack features from multiple angles to capture the spatial representation of complex cracks. Initially, OSSM scans the input image along the horizontal, vertical, diagonal, anti-diagonal directions and their opposite directions to generate 8 feature sequences. Subsequently, the aforementioned 8 feature sequences are concatenated in a new dimension and input into the S6 block. The S6 block uses a unique selective scanning mechanism to model global feature information from input sequences in all directions. Specifically, the calculation method of the S6 block is as follows:

[0063] h ′ (t) = Ph(t) + Qx(t);

[0064] y(t)=Uh(t)+Vx(t);

[0065] Among them, the evolution parameter P∈R N×N , projection parameters Q,U∈R N, the skip connection parameter V∈R. N is the length of the input feature sequence. h(t) is the state vector from input x(t) to output y(t). In addition, in the deep learning network, the discretization process is:

[0066]

[0067] in Represent the discretization of P, Q, U, and V respectively, △ represents the time scale parameter, and h t-1 and h t Represent the state of the previous time step and the state of the current time step respectively.

[0068] In the convolution branch, this embodiment uses multi-scale depthwise convolution to divide the input features into different segments according to the channel dimension. Each segment undergoes convolution of different sizes. Subsequently, these segments are merged to produce fine local features. This embodiment specifies convolution kernels of [1×1, 3×3, 5×5, 7×7, 9×9, 11×11] to enhance the fine granularity of local features. In the edge detection branch, this embodiment uses the Laplacian operator to calculate the quadratic gradient of the edge details extracted from the backbone to enhance edge modeling. The outputs of the three branches are integrated and coordinated through a linear layer to generate a fused feature input for the REM-Net decoder.

[0069] In this embodiment, the loss function can quantify the deviation between the predicted results and the actual results, which is very important for training deep learning networks. The loss function needs to be configured reasonably, especially in the crack segmentation task. Crack segmentation is a binary classification task, in which pixels are divided into crack pixels and background pixels, represented by 0 and 1 respectively. It is worth noting that the number of background pixels and crack pixels is seriously unbalanced, and crack pixels only account for a small proportion in the image. To solve this problem, in REM-Net, this embodiment sets the loss function to binary cross entropy (BCE) loss and DICE loss. BCE loss is specially designed for binary classification tasks, focusing on how to accurately classify crack pixels. DICE loss enhances the network's attention to crack areas during training to effectively solve the problem of unbalanced distribution of crack and background pixel ratios. The loss function of this embodiment is specifically designed as follows:

[0070] BCELoss(p,t)=-(1-t)log(1-p)-tlog(p);

[0071]

[0072] Loss(p,t)=a*BCELoss(p,t)+b*DICELoss(p,t);

[0073] Among them, p represents the prediction result of the crack image, t represents the true value label; p i Represents the prediction result of the i-th pixel, t i represents the true value of the i-th pixel; N represents the total number of pixels in the crack image; ε is the smoothing factor, which is set to 1 to prevent the denominator from being predicted as zero; a and b are two constants used to control the proportion of BCE loss and DICE loss in the entire loss function.

[0074] In summary, the crack segmentation network REM-Net disclosed in this embodiment consists of an encoder constructed using RAT and skip connections integrated with MSFM to achieve comprehensive global feature extraction and edge information modeling. RAT aggregates global contextual information by scaling the input feature map and uses edge operators to accurately locate crack locations. MSFM applies multiple operators to extract global and edge information from the input feature map to promote the fusion of features at the same scale, improving the efficiency of overall feature representation and semantic information transmission, thereby reducing information loss and improving segmentation accuracy.

[0075] Example 2

[0076] Corresponding to the above embodiment, this embodiment discloses an edge-enhanced pavement crack segmentation network processing system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above method is implemented. The method includes image preprocessing, upsampling, downsampling, feature fusion, and image restoration processing. Recursive aggregation processing is performed after image preprocessing, between two adjacent sampling layers, and before image restoration processing. The recursive aggregation processing includes:

[0077] For a given input X in , obtain a coarse-grained representative feature map X′ through depthwise convolution (preferably through the method based on recursive calling of multiple depthwise convolutions as described in the above embodiment); then refine X′ through large kernel separable convolution, and the output of the large kernel separable convolution is:

[0078] Z1=W (2d-1)×1 *(W 1×(2d-1) *X ′ );

[0079]

[0080] Where d and k represent the dilation rate and the original convolution kernel size respectively; W represents the convolution operation; X′ is obtained by two depthwise convolutions with kernel sizes of 1×(2d-1) and (2d-1)×1 to obtain Z1, and then Z1 is convolved through two kernel sizes of and The expanded depth convolution is performed to obtain Z2; then, the representative feature map channel C is scaled to c by 1×1 convolution on Z2 to obtain X map .

[0081] Then input feature X in Reshape into Q, transform X map Reshape into K and V; generate attention matrix A, where:

[0082] Q=W q (X in ),K=W k (X map ),V=W v (X map );

[0083] A=Softmax(K T Q);

[0084] X out =W p (A·V);

[0085] Among them, W q , W k , W v They are all linear layers, transforming the corresponding inputs into Q, K, and V respectively; Softmax(·) represents the softmax function, and the Cartesian product of K and Q is obtained to obtain the A matrix; the linear layer W p Used to ensure that the output dimension is H×W×C, where H, W, and C represent the height, width, and number of channels respectively; X out Represents the output after cross-attention mechanism processing.

[0086] Then for X out With X in The fused first feature is normalized to obtain the second feature, and then the edge refinement feedforward network processing is performed on the second feature. During the processing, the edge operator and recursive gated convolution are used to extract high-order spatial edge information.

[0087] Preferably, the specific process of the edge thinning feedforward network processing includes:

[0088] In the processing branch of recursive gated convolution, a large core attention mechanism is applied to expand the receptive field, which includes 5×5 depth convolution, 7×7 depth expansion convolution and 1×1 convolution in sequence; in the other branch, the Sobel operator is divided into two parts: the horizontal part S x and the vertical part S y :

[0089]

[0090] The edge information of the feature map of the input edge detection branch is extracted using horizontal and vertical Sobel filters respectively, and then the extracted horizontal edge information is fused with the vertical edge information to obtain the branch output.

[0091] After the data of the two branches are fused, they are processed in sequence through 3×3 convolution, activation function, layer normalization and linear layer to obtain the third feature; finally, the third feature is fused with the first feature and used as the input of the next sampling module or the functional module for image restoration processing.

[0092] Preferably, in the process of feature fusion at all levels, after the conventional fusion processing, it also includes: the feature map after the conventional fusion processing is first subjected to layer normalization processing, and then the three branches consisting of an omnidirectional selective scanning module, a multi-scale deep convolution and a Laplace operator are used to extract features of different scales, and then the linear layer is used to linearly process the features after the three branches are fused, and then superimposed with the feature map after the conventional fusion processing as the input of the next recursive aggregation processing.

[0093] In summary, the methods and systems disclosed in the above two embodiments of the present invention, through unique recursive aggregation processing, aggregate global context information by scaling the input feature map, which can enhance local details and reduce information loss, and realize global modeling based on the attention mechanism. At the same time, recursive gated convolution and edge operators are used to extract high-order spatial information and edge information from the input feature map, respectively, greatly improving the global information capture performance; thereby ensuring the accuracy of the segmentation results.

[0094] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A network processing method for edge-enhanced pavement crack segmentation, comprising image preprocessing, upsampling, downsampling, feature fusion and image restoration processing, characterized in that: After image preprocessing, recursive aggregation processing is performed between two adjacent sampling layers and before image restoration processing. The recursive aggregation processing includes: For a given input X in , a coarse-grained representative feature map X′ is obtained through deep convolution; then X′ is refined through large kernel separable convolution, and the output of large kernel separable convolution is: Z1=W (2d-1)×1 *(IN 1×(2d-1) *X′); Where d and k represent the dilation rate and the original convolution kernel size respectively; W represents the convolution operation; X′ is obtained by two depthwise convolutions with kernel sizes of 1×(2d-1) and (2d-1)×1 to obtain Z1, and then Z1 is convolved through two kernel sizes of and The expanded depth convolution is performed to obtain Z2; then, the representative feature map channel C is scaled to c by 1×1 convolution on Z2 to obtain X map ; Then input feature X in Reshape into Q, transform X map Reshape into K and V; generate the attention matrix A, where: Q=W q (X in ),K=W k (X map ),V=W v (X map ); A=Softmax(K T Q); X out =W p (A·V); Among them, W q , W k , W v They are all linear layers, transforming the corresponding inputs into Q, K, and V respectively; Softmax(·) represents the softmax function, and the Cartesian product of K and Q is obtained to obtain the A matrix; the linear layer W p Used to ensure that the output dimension is H×W×C, where H, W, and C represent the height, width, and number of channels respectively; X out Represents the output after cross-attention mechanism processing; Then for X out With X in The fused first feature is normalized to obtain the second feature, and then the edge refinement feedforward network processing is performed on the second feature. During the processing, the edge operator and recursive gated convolution are used to extract high-order spatial edge information.

2. The method according to claim 1, characterized in that The specific process of the edge refinement feedforward network processing includes: In the processing branch of recursive gated convolution, a large core attention mechanism is applied to expand the receptive field, which includes 5×5 depth convolution, 7×7 depth expansion convolution and 1×1 convolution in sequence; in the other branch, the Sobel operator is divided into two parts: the horizontal part S x and the vertical part S y : The edge information of the input edge detection branch feature map is extracted using horizontal and vertical Sobel filters respectively, and then the extracted horizontal edge information is fused with the vertical edge information to obtain the branch output; After the data of the two branches are fused, they are processed in sequence through 3×3 convolution, activation function, layer normalization and linear layer to obtain the third feature; finally, the third feature is fused with the first feature and used as the input of the next sampling module or the functional module for image restoration processing.

3. The method according to claim 1 or 2, characterized in that In the process of feature fusion at all levels, after the conventional fusion process, the following steps are also included: The feature map after conventional fusion processing is first normalized by the layer, and then the three branches consisting of omnidirectional selective scanning module, multi-scale deep convolution and Laplacian operator are used to extract features of different scales. The linear layer then performs linear processing on the features after the three-branch fusion and then superimposes them with the feature map after conventional fusion processing as the input for the next recursive aggregation processing.

4. A pavement crack segmentation network processing system with edge enhancement, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 3 is implemented.

Images