A road surface crack detection method based on edge reconstruction network

Abstract

This invention discloses a road surface crack detection method based on an edge reconstruction network. It constructs a binary cross-entropy loss function, an edge loss function, and a detail loss function, with the sum of the three functions serving as the total loss function. The edge loss function is used to enhance the extraction of boundary features by the edge branch, the detail loss function is used to enhance the extraction of boundary features by the spatial branch, and the binary cross-entropy loss function is used to enhance pixel classification. During the training phase, training set images are input into the network, and the loss function is used to calculate the loss of the network's prediction results. The network parameters are optimized through backpropagation and iterative training. After the loss stabilizes, the test set images are input into the trained neural network to obtain the final detection results. This invention can effectively enhance the generalization of road crack detection and improve the detection accuracy of crack edges.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and computer vision processing, and in particular to a road surface crack detection method based on an edge reconstruction network. Background Art

[0002] my country's highway infrastructure network is already among the largest in the world. Rapid and efficient detection of road damage is becoming increasingly important in road maintenance and management. In the field of transportation infrastructure management, road damage detection helps reduce maintenance costs, prevent traffic accidents, and support the safety and sustainable development of transportation infrastructure. Road damage detection technology can promptly detect signs of damage, enabling necessary repair and maintenance measures to ensure the sustainable development of the road system. However, the complex and irregular distribution of road cracks presents many challenges: 1) The boundary between cracks and the road is low in contrast, resulting in blurred transitions and unclear differences between classes, making it difficult to accurately identify edges. 2) The road surface is affected by interference from lighting, shadows, and road debris, which can obscure damaged areas and lead to false detections. 3) Narrow road cracks typically appear as a few to a dozen pixels wide in imagery, resulting in poor extraction integrity. Therefore, detecting road surface crack damage is a challenging task.

[0003] Current road crack detection methods can be divided into three categories. The first category is image processing-based methods, which manually discriminate features and utilize information such as edges and textures to design automated detection methods that meet current feature recognition requirements. These methods require careful algorithm design, rely on manually preset parameters, and are susceptible to lighting conditions and distortion noise in real-world environments. The second category is traditional machine learning-based methods, which extract meaningful features such as edges, textures, and shapes from raw images. These features are then used to train classifiers such as support vector machines (SVMs) and random forests to automatically identify and classify cracks. However, these methods are less adaptable to different scenarios and suffer from limitations in adaptability and generalization. The third category is deep learning-based methods, which automatically learn and extract hierarchical feature representations from raw images without the need for manual feature design. Compared to traditional methods, these methods offer significantly improved accuracy and adaptability. However, existing road crack detection methods still suffer from issues such as missed detection of small crack branches and difficulty restoring crack edge details. Their limited generalization also leads to reduced accuracy in practical applications, making them difficult to meet practical needs. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a road surface crack detection method based on edge reconstruction network, which can effectively enhance the generalization of road crack detection and improve the detection accuracy of crack edges.

[0005] To solve the above technical problems, the present invention provides a road surface crack detection method based on an edge reconstruction network, comprising the following steps:

[0006] Step 1: Perform image enhancement on the acquired road loss image data, perform multi-angle flipping and random occlusion on the image, and divide the resulting enhanced dataset into a training set, a validation set, and a test set. The training set is used to train the network, the validation set is used to monitor model performance and generalization ability, optimize hyperparameters, and save the optimal model. The test set is used to evaluate the performance of the trained network.

[0007] Step 2: Construct a three-branch edge reconstruction network to generate detection results. The edge reconstruction network is an encoder-decoder structure. The encoder includes a semantic branch, a detail branch, and an edge branch. The enhanced image I is input and semantic features, detail features, and edge features are extracted respectively. The decoder stage fuses the above features through a multi-level feature fusion module to obtain a global feature map. Finally, the global feature map is upsampled to obtain the network detection result.

[0008] Step 3: Construct an edge branch based on the edge reconstruction module, extract the rough edge features of the image, filter and reduce the noise of the rough edge features, generate a fine edge feature map, and input the edge feature map into the edge loss function to generate the edge prediction result;

[0009] Step 4: In the semantic branch, a global feature extraction module based on the convolutional self-attention mechanism is constructed. This module establishes the dependencies between long-distance pixels of semantic features through the self-attention mechanism. Decomposed convolution is introduced to achieve lightweight fitting of large kernel convolution, thereby efficiently extracting global context information.

[0010] Step 5: In the decoder stage, a coordinate-based multi-level feature fusion module is constructed. This module concatenates edge features with semantic features and detail features, aggregates the concatenated features along two spatial directions, captures feature representations of different scales, and fuses shallow spatial contour information with deep semantic information to generate a global feature map. Finally, the global feature map is upsampled to obtain the network's prediction result.

[0011] Step 6. Construct a binary cross entropy loss function, an edge loss function, and a detail loss function, and use the sum of the three as the total loss function; use the edge loss function to strengthen the extraction of boundary features by the edge branch, use the detail loss function to strengthen the extraction of boundary features by the spatial branch, and use the binary cross entropy loss function to strengthen pixel classification; in the training stage, input the training set images into the network, calculate the loss of the network's prediction results through the loss function, and optimize the network parameters through back propagation and iterative training; after the loss stabilizes, input the test set images into the trained neural network to obtain the final detection results.

[0012] Preferably, in step 1, image enhancement includes unifying image size, rotating, cropping, and pixel grayscale normalization.

[0013] Preferably, in step 2, the edge reconstruction network uses the edge branch to extract edge features, downsamples the enhanced image I by 1 / 8, then inputs the edge branch, and extracts edge information through the edge reconstruction module; by using a multi-level feature fusion module to fuse the semantic features, detail features and edge features to generate a global feature map, and finally upsamples the global feature map to obtain the network's prediction result, so that the network can accurately locate and extract the cracks.

[0014] Preferably, the edge branch includes two stages: edge extraction and edge reconstruction. In the edge extraction stage, the edge of the 1 / 8 downsampled image is extracted by a Sobel edge extractor to obtain a rough edge feature F0. In the edge reconstruction stage, two edge reconstruction modules are connected in series to combine the edge feature F0 with the feature X of the semantic branch. out1 Input to the first edge reconstruction module to get F1; then combine the edge feature F1 and the feature X of the semantic branch out2 Input to the second edge reconstruction module to obtain the output feature F2 of the edge branch; the two edge reconstruction modules use different levels of semantic information X out1 and X out2 Guide edge information to improve overall scene understanding and edge reconstruction accuracy. In the edge reconstruction module, semantic features with rich semantic information are used to guide rough edge features with rough contour information to generate a fine edge feature map. During the training phase, an edge loss function is used to fit crack edges, calculate edge prediction results, and optimize network parameters through backpropagation to improve edge reconstruction capabilities.

[0015] Preferably, the edge reconstruction module constructs residual learning for rough edge features based on Taylor finite difference equations to improve the network's modeling ability for edge information; based on the gating mechanism, semantic features are used to suppress high-frequency background noise in rough edge features; the mathematical form of Taylor finite difference equations is converted into Rewritten as residual form From this, we can get the residual learning process of the edge reconstruction module as u j+2 =u gate +u j+1 -3Δu j , where u j is the input feature F of the edge reconstruction module i ,u j+2 The output feature F of the edge reconstruction module i+1The specific process is as follows: First, the filter sequence composed of two residual structures is used to perform nonlinear transformation on the shallow edge feature map to obtain features with less noise and clutter, so as to achieve the goal of i arrive Then, the semantic features and edge feature maps in the semantic branch are input into the gated convolution, and the gated convolution is used to guide the rough edge features for directional learning to obtain the output features of the gated convolution. Better suppression of background noise, where Expressed as:

[0016]

[0017] Where σ represents the Sigmoid activation function, Represents the multiplication operation of the corresponding element matrix, φ represents the ReLU activation function, W f represents the convolution filter sequence, X OUT_i Represents semantic features; finally and Adding the residual structure to alleviate the problem of gradient diffusion during training, and finally obtaining the output feature F i+1 .

[0018] Preferably, the global feature extraction module in the semantic branch decomposes the large kernel convolution by using the dilated convolution and depthwise separable convolution techniques, extracts the long-range dependency between pixels based on the self-attention mechanism, and realizes the extraction of global context information in the semantic branch. The specific process is as follows: Step 1: Input feature X in After the 1×1 convolution layer and activation layer, the GELU activation function is used to maintain the expressive power of the model; the second step is to pass the 5×5 deep convolution layer to capture the close relationship of pixels; the third step is to input the features into two branches respectively. Both branches are composed of deep hole convolution. The kernel sizes of deep hole convolution are 3 and 7 respectively. Their receptive fields are equivalent to standard convolutions with kernel sizes of 7 and 19 respectively, while the computational complexity is only 1 / 49 and 1 / 361 of the standard convolution respectively; deep hole convolution with different kernel sizes in the dual-branch structure can capture long-range relationships at different spatial scales; the fourth step is to add and fuse the output features of the deep hole convolution and input it into the 1×1 point convolution to reduce the feature map to a smaller size to achieve feature fusion in the channel dimension, thereby capturing global context information and realizing information interaction between space and channel; the fifth step is to use the output of the 1×1 convolution as the input feature X in The self-attention weight map of X in Weighted, the output feature X is obtained out ; The process of extracting global context information is shown in the formula:

[0019] Att=Conv1×1 (DW-DConv branch1 (X in )+DW-DConv branch2 (X in ))

[0020]

[0021] Among them, X in represents the input features, Represents the multiplication operation of the corresponding element matrix, Conv 1×1 Represents 1×1 convolution, DW-DConv branch_i represents the depth of the hollow convolution of the i-th branch, Att represents the attention weight map, X out Represents the output features.

[0022] Preferably, in step 5, the multi-level feature fusion module concatenates the edge features with the shallow features and the deep features respectively, and then aggregates the output features along the two spatial directions to capture different scale feature representations; first, through the average pooling operation, the feature map of size H×W is compressed into two scales of C×H×1 and C×1×W in the X and Y coordinate directions respectively; in this process, the obtained feature map is aggregated into two independent direction perception feature maps along the X and Y dimensions, thereby establishing the spatial correlation of the X and Y dimensions; the fused horizontal and vertical spatial information is encoded through a 3×3 convolution layer + BN+ReLU, and finally a coordinate attention weight map is generated through a Sigmoid function; on the other hand, a 3×3 depthwise separable convolution is used to encode the spatial information and semantic information respectively, and the two coordinate attention weight maps are used to weight the encoded spatial information and semantic information respectively, so as to realize the adaptive fusion of feature information between different levels and different modalities.

[0023] Preferably, in step 6, the sum of the binary cross entropy loss function, the edge loss function and the detail loss function is used as the total loss function, wherein the binary cross entropy loss is as shown in the formula:

[0024] L bce -log(p t )

[0025]

[0026] where p t reflects the closeness of the result to the true value category y, p t The larger it is, the more accurate the classification of the pixel is;

[0027] The edge loss function fits the overall edge based on Dice's edge perception loss, applies the squash function on the gradient information graph, and obtains the edge prediction result. The definition of the obtained edge prediction result is shown in the formula:

[0028]

[0029] in and g ij Represent the edge prediction result and gradient information vector at (i, j), e ij represents the edge truth obtained directly from the detail label, α is a hyperparameter that controls the model's sensitivity to target edges, and α is set to 1 to balance inter-class differences and intra-class uniformity;

[0030] The boundary refinement problem is expressed as a Dice coefficient maximization problem, as shown in the following formula:

[0031]

[0032] Among them, i represents the i-th pixel, Represents the segmentation map label value, Represents the edge prediction result, Θ represents the parameters of the segmentation network, H and W represent the height and width of the feature map respectively;

[0033] In order to implement stochastic gradient descent during training, the marginal loss is constructed as follows:

[0034] L edge (p d , g d ,Θ)=1-Dice(p d , g d , Θ)

[0035] During the training process, the detail loss function uses dice loss, which focuses on the detail features and balances the class imbalance problem, expressed as:

[0036]

[0037] Among them, i represents the i-th pixel, Represents the segmentation map label value, Represents the predicted segmentation map, H and W represent the height and width of the feature map respectively, ∈ represents the Laplace smoothing term, and ∈ is set to 1 to avoid zero division;

[0038] The binary cross entropy loss is used as the main loss function of the edge reconstruction network, and the detail loss function and the edge loss function are set as auxiliary loss functions. During the training process, the output result of the detail branch is calculated with the label value to obtain the results of the detail loss function and the binary cross entropy loss function, and the detail branch is optimized; the output result of the edge branch is calculated with the label value to obtain the results of the edge loss function and the binary cross entropy loss function, and the edge branch is optimized; iterative training is performed to optimize the network parameters. After the loss stabilizes, the detection image is input into the trained neural network to obtain the final detection result graph.

[0039] The beneficial effects of the present invention are as follows: (1) a three-branch semantic segmentation network is constructed based on semantic branch, detail branch and edge branch, which realizes the extraction of high-frequency features of crack edges while improving network parallelism. The edge reconstruction module uses the residual structure to learn edge features. Aiming at the common characteristics of blurred road crack boundaries, a gating mechanism is used to extract edge information in a targeted manner, thereby improving the accuracy of crack edge positioning and segmentation; (2) the global feature extraction module realizes lightweight decomposition of large kernel convolution through the cross-application of deep convolution, void convolution and point convolution, establishes long-distance dependency between model pixels, and realizes global semantic segmentation. The extraction of information and, through the attention mechanism, adaptive expression in the spatial domain and channel domain are realized to improve the anti-interference ability of the network in complex backgrounds; (3) The multi-level feature fusion module can effectively enhance and fuse the features obtained from the three branches, and use multi-scale feature fusion to encode feature information of different levels and modalities, further improving the ability of location information to guide the overall network; (4) The auxiliary task enables the network to learn more edge high-frequency information, thereby improving the detection accuracy of narrow roads. By optimizing multiple classification losses with the same label information, their complementary advantages are used to jointly solve the category imbalance problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the method of the present invention.

[0041] Figure 2 It is a schematic diagram of the overall architecture of the present invention.

[0042] Figure 3 Schematic diagram of the edge reconstruction module structure of the present invention.

[0043] Figure 4 Schematic diagram of the global feature extraction module structure of the present invention.

[0044] Figure 5 Schematic diagram of the multi-level feature fusion module structure of the present invention.

[0045] Figure 6 Schematic diagram of the detection results of the present invention on the test data set. DETAILED DESCRIPTION

[0046] like Figure 1 As shown, a road surface crack detection method based on edge reconstruction network includes the following steps:

[0047] Step 1: Perform image enhancement on the acquired road loss image data: unify the image size, rotate, crop, normalize pixel grayscale, flip the image at multiple angles, and perform random occlusion. The resulting enhanced dataset is divided into a training set, a validation set, and a test set. The training set is used to train the network, the validation set is used to monitor model performance and generalization ability, optimize hyperparameters, and save the optimal model. The test set is used to evaluate the performance of the trained network.

[0048] Step 2: Construct a three-branch edge reconstruction network to generate detection results. The edge reconstruction network is an encoder-decoder structure. The encoder includes a semantic branch, a detail branch, and an edge branch. The enhanced image I is input and semantic features, detail features, and edge features are extracted respectively. In the decoder stage, the above features are fused through a multi-level feature fusion module to obtain a global feature map. Finally, the global feature map is upsampled to obtain the network detection result.

[0049] Step 3: Construct an edge branch based on the edge reconstruction module, extract the rough edge features of the image, filter and reduce the noise of the rough edge features, generate a fine edge feature map, and input the edge feature map into the edge loss function to generate the edge prediction result.

[0050] Step 4. Construct a global feature extraction module based on the convolutional self-attention mechanism in the semantic branch. This module establishes the dependency relationship between long-distance pixels of semantic features through the self-attention mechanism; introduces decomposed convolution to achieve lightweight fitting of large kernel convolution, thereby efficiently extracting global context information.

[0051] Step 5. In the decoder stage, a coordinate-based multi-level feature fusion module is constructed. This module concatenates edge features with semantic features and detail features respectively, aggregates the concatenated features along two spatial directions, captures feature representations of different scales, fuses shallow spatial contour information with deep semantic information, generates a global feature map, and finally upsamples the global feature map to obtain the network's prediction result.

[0052] Step 6: Construct a binary cross-entropy loss function, an edge loss function, and a detail loss function, and use the sum of these three as the total loss function. The edge loss function is used to enhance the edge branch's extraction of boundary features, the detail loss function is used to enhance the spatial branch's extraction of boundary features, and the binary cross-entropy loss function is used to enhance pixel classification. During the training phase, the training set images are fed into the network, and the loss function is used to calculate the loss of the network's prediction results. The network parameters are optimized through backpropagation and iterative training. Once the loss stabilizes, the test set images are fed into the trained neural network to obtain the final detection results.

[0053] The present invention provides a specific implementation case, in which a data set of 3368 road crack images with different target shapes and sizes is used. The data set has a complex background, including asphalt roads, gravel, water stains, vegetation and debris. The image size is uniformly 640×360. The entire data set is divided into a training set, a validation set and a test set in a ratio of 6:2:2. The training set is used to train the network, and the validation set is used to monitor the model performance and generalization ability, optimize hyperparameters, and save the optimal model. The test set is used to compare the performance of the method of the present invention with other existing methods. This embodiment is implemented in a computer with an Intel i7-12700k CPU, a GeForce RTX 3090Ti (24G) GPU, and 64G memory, and the programming language is python.

[0054] During the training phase, the input road images are grouped into groups of 8. The SGD optimizer is used to train the network weight parameters. The initial learning rate is set to 7e-3, which is adaptively adjusted according to the batch size. The weight decay coefficient is 1e-4. The network training is iterated until the loss function remains stable, at which point training is stopped. During the training process, the total training cycle is set to 120, and a performance verification is performed and the model is saved after each iteration. During the detection process, test images are input to verify the network performance and save the detection results, such as Figure 6 shown.

[0055] like Figure 2 As shown in Figure 1, the network has three branches: the semantic branch, the detail branch, and the edge branch. The detail and semantic branches share the same shallow backbone, which consists of two convolutional layers, a batch normalization (BN) layer, and a ReLU layer, stacked with two STDC modules. The output of the STDC module integrates multi-scale information. As the number of convolutional layers increases, the receptive field increases, and the fusion operation preserves the scalable receptive field. The enhanced image I is input into the shallow backbone, resulting in the shallow backbone output feature X0. X0 is downsampled by 1 / 8 compared to the original input, and the number of channels is reduced to 256. The shallow backbone output feature X0 is used both for the subsequent edge detection auxiliary task and as the input feature of the semantic branch.

[0056] like Figure 2 and Figure 4 As shown in Figure 2, the output feature X0 of the shallow backbone is input into the semantic branch. The semantic branch consists of two stages, each of which consists of two STDC modules and one global feature extraction module. Assume that the input feature of the global feature extraction module is X in And the output feature is X out The process of global feature extraction module is shown as follows:

[0057] Att=Conv 1×1 (DW-DConv branch1 (X in )+DW-DConv branch2 (X in ))

[0058]

[0059] Among them, X in represents the input features, Represents the multiplication operation of the corresponding element matrix, Conv 1×1 Represents 1×1 convolution, DW-DConv branch_i represents the depth of the hollow convolution of the i-th branch, Att represents the attention weight map, X out Represents the output feature. The specific process is: the first step is to input feature X in After the 1×1 convolution layer and activation layer, the GELU activation function is used to maintain the expressive power of the model; the second step is to pass the 5×5 deep convolution layer to capture the close relationship of pixels; the third step is to input the features into two branches respectively. Both branches are composed of deep hole convolution. The kernel sizes of the deep hole convolution are 3 and 7 respectively. Their receptive fields are equivalent to standard convolutions with kernel sizes of 7 and 19 respectively, while the computational complexity is only 1 / 49 and 1 / 361 of the standard convolution respectively. The use of deep hole convolutions with different kernel sizes in the dual-branch structure can capture long-range relationships at different spatial scales; the fourth step is to add and fuse the output features of the deep hole convolution and input them into the 1×1 point convolution to reduce the feature map to a smaller size, realize feature fusion in the channel dimension, thereby capturing global context information and realizing information interaction between space and channel; the fifth step is to use the output of the 1×1 convolution as the input feature X in The self-attention weight map of X in Weighted, the output feature X is obtained out . 2 global feature extraction modules output feature X out1 and X out2 are used for two edge reconstruction modules of the edge branch, where X out2It is also used as the final output feature of the semantic branch and is input into the coordinate-based multi-level feature fusion module together with the output features of the other two branches in the feature fusion stage.

[0060] like Figure 2 and Figure 3 As shown in the figure, the edge branch of the network is constructed, and the enhanced image I is downsampled by 1 / 8 and input into the edge branch. The edge branch includes two stages: edge extraction and edge reconstruction. In the edge extraction stage, the edge of the 1 / 8 downsampled image is extracted by the Sobel edge extractor to obtain the rough edge feature F0; the edge reconstruction stage is implemented by two serially connected edge reconstruction modules, and the edge feature F0 is combined with the feature X of the semantic branch. out1 Input to the first edge reconstruction module to get F1; then combine the edge feature F1 and the feature X of the semantic branch out2 Input to the second edge reconstruction module to obtain the output feature F2 of the edge branch. The two edge reconstruction modules use semantic information X at different levels respectively. out1 and X out2 Guide the edge information to improve the understanding of the overall scene and the accuracy of edge reconstruction. The residual learning process of the edge reconstruction module is u j+2 =u gate +u j+1 -3Δu j , where u j is the input feature F of the edge reconstruction module i ,u j+2 The output feature F of the edge reconstruction module i+1 In the edge reconstruction module, the shallow edge feature map is first nonlinearly transformed by forming a filter sequence through convolution of two residual structures to obtain features with less noise and clutter, thus realizing the i arrive Then, the semantic features and edge feature maps in the semantic branch are input into the gated convolution, and the gated convolution is used to guide the rough edge features for directional learning to obtain the output features of the gated convolution. Better suppression of background noise, where Expressed as:

[0061]

[0062] Where σ represents the Sigmoid activation function, Represents the multiplication operation of the corresponding element matrix, φ represents the ReLU activation function, W f represents the convolution filter sequence, X OUT_i Represents semantic features; finally and Adding the residual structure to alleviate the problem of gradient diffusion during training, and finally obtaining the output feature Fi+1 .

[0063] like Figure 5 As shown in the figure, the output features of the three branches of the network are input into a coordinate-based multi-level feature fusion module. This module concatenates edge features with shallow and deep features, respectively, and then aggregates the output features along two spatial directions to capture feature representations at different scales. First, an average pooling operation is used to compress the H×W feature map into two scales, C×H×1 and C×1×W, in the X and Y coordinate directions, respectively. During this process, the resulting feature maps are aggregated into two independent direction-aware feature maps along the X and Y dimensions, thereby establishing spatial correlation in the X and Y dimensions. A 3×3 convolutional layer followed by a batch normalization (BN) and a ReLU layer are used to encode the fused horizontal and vertical spatial information. Finally, a sigmoid function is used to generate a coordinate attention weight map. Furthermore, a 3×3 depthwise separable convolution is used to encode spatial and semantic information, respectively. The two coordinate attention weight maps are used to weight the encoded spatial and semantic information, respectively, achieving adaptive fusion of feature information across different levels and modalities.

[0064] like Figure 2 As shown in the figure, during the training phase, binary cross-entropy loss is used as the network's primary loss function, while detail loss and edge loss functions are set as auxiliary loss functions. During training, network parameters are iteratively optimized. Deep supervision results are generated for the edge branch during the training phase. The edge loss function is used to fit the overall edge, and edge predictions are calculated. Backpropagation through the deep network improves edge reconstruction capabilities. Deep supervision results are generated for the detail branch during the training phase. The output of the detail branch is then compared with the label value to calculate the results of the detail loss function and binary cross-entropy loss function, and the detail branch is optimized. After the loss stabilizes, the test image is input into the trained neural network to obtain the final detection result image. During the testing phase, the test sample is input into the network to obtain the final detection result.

[0065] Figure 6 The following is a diagram showing the detection effect of this embodiment, where a and c are the original input road crack images, and b and d are the output images after edge reconstruction network detection. Figure 6 The detection results show that the edge reconstruction network proposed in the present invention can effectively detect the target and retain the complete edge contour, which is an effective road surface crack detection method.

Claims

1. A road surface crack detection method based on edge reconstruction network, characterized in that: The steps include: Step 1: Perform image enhancement on the acquired road loss image data, perform multi-angle flipping and random occlusion on the image, and divide the obtained enhanced dataset into training set, validation set, and test set; The training set is used to train the network, the validation set is used to monitor model performance and generalization ability, optimize hyperparameters, and save the optimal model, and the test set is used to evaluate the performance of the trained network. Step 2: Construct a three-branch edge reconstruction network to generate detection results. The edge reconstruction network is an encoder-decoder structure. The encoder includes a semantic branch, a detail branch, and an edge branch. The enhanced image I is input and the semantic features, detail features, and edge features are extracted respectively. In the decoder stage, the above features are fused through a multi-level feature fusion module to obtain a global feature map, and finally the global feature map is upsampled to obtain the network detection result; Step 3: Construct an edge branch based on the edge reconstruction module, extract the rough edge features of the image, filter and reduce the noise of the rough edge features, generate a fine edge feature map, and input the edge feature map into the edge loss function to generate the edge prediction result; Step 4: In the semantic branch, a global feature extraction module based on the convolutional self-attention mechanism is constructed. This module establishes the dependencies between long-distance pixels of semantic features through the self-attention mechanism. Decomposed convolution is introduced to achieve lightweight fitting of large kernel convolution, thereby efficiently extracting global context information. Step 5: In the decoder stage, a coordinate-based multi-level feature fusion module is constructed. This module concatenates edge features with semantic features and detail features, aggregates the concatenated features along two spatial directions, captures feature representations of different scales, and fuses shallow spatial contour information with deep semantic information to generate a global feature map. Finally, the global feature map is upsampled to obtain the network's prediction result. Step 6: Construct a binary cross entropy loss function, an edge loss function, and a detail loss function, and use the sum of the three as the total loss function; use the edge loss function to strengthen the extraction of boundary features by the edge branch, use the detail loss function to strengthen the extraction of boundary features by the spatial branch, and use the binary cross entropy loss function to strengthen pixel classification; During the training phase, the training set images are input into the network, the network's prediction results are calculated using the loss function, and the network parameters are optimized through back propagation and iterative training. After the loss stabilizes, the test set images are input into the trained neural network to obtain the final detection results.

2. The road surface crack detection method based on edge reconstruction network according to claim 1, characterized in that: In step 1, image enhancement includes unifying image size, rotation, cropping, and pixel grayscale normalization.

3. The road surface crack detection method based on edge reconstruction network according to claim 1, characterized in that: In step 2, the edge reconstruction network uses the edge branch to extract edge features, downsamples the enhanced image I by 1 / 8, then inputs it into the edge branch, and extracts edge information through the edge reconstruction module; by using a multi-level feature fusion module to fuse semantic features, detail features, and edge features to generate a global feature map, and finally upsamples the global feature map to obtain the network's prediction result, enabling the network to accurately locate and extract cracks.

4. The road surface crack detection method based on edge reconstruction network according to claim 3, characterized in that: The edge branch includes two stages: edge extraction and edge reconstruction. In the edge extraction stage, the edge of the 1 / 8 downsampled image is extracted by the Sobel edge extractor to obtain the rough edge feature F0. The edge reconstruction stage is implemented by two serially connected edge reconstruction modules to combine the edge feature F0 with the feature X of the semantic branch. out1 Input to the first edge reconstruction module to get F1; then combine the edge feature F1 and the feature X of the semantic branch out2 Input to the second edge reconstruction module to obtain the output feature F2 of the edge branch; the two edge reconstruction modules use different levels of semantic information X out1 and X out2 Guide edge information to improve the understanding of the overall scene and the accuracy of edge reconstruction.

5. The road surface crack detection method based on edge reconstruction network according to claim 3, characterized in that: The edge reconstruction module constructs residual learning for rough edge features based on Taylor finite difference equations to improve the network's modeling ability for edge information; based on the gating mechanism, semantic features are used to suppress high-frequency background noise in rough edge features; the mathematical form of Taylor finite difference equations is converted into Rewritten as residual form From this, we can get the residual learning process of the edge reconstruction module as u j+2 =u gate +u j+1 -3Δu j , where u j is the input feature F of the edge reconstruction module i ,u j+2 The output feature F of the edge reconstruction module i+1 .

6. The road surface crack detection method based on edge reconstruction network according to claim 5, characterized in that: First, the convolution of two residual structures forms a filter sequence to perform nonlinear transformation on the shallow edge feature map to obtain features with less noise and clutter, so as to achieve the goal of i arrive Then, the semantic features and edge feature maps in the semantic branch are input into the gated convolution, and the gated convolution is used to guide the rough edge features for directional learning to obtain the output features of the gated convolution. Better suppression of background noise, where Expressed as: Where σ represents the Sigmoid activation function, Represents the multiplication operation of the corresponding element matrix, φ represents the ReLU activation function, W f represents the convolution filter sequence, X OUT_i Represents semantic features; finally and Adding the residual structure to alleviate the problem of gradient diffusion during training, and finally obtaining the output feature F i+1 .

7. The road surface crack detection method based on edge reconstruction network according to claim 1, characterized in that: The global feature extraction module decomposes the large kernel convolution through the dilated convolution and depthwise separable convolution techniques, extracts the long-range dependency between pixels based on the self-attention mechanism, and realizes the extraction of global context information in the semantic branch. The specific process is as follows: First, the input feature X in After a 1×1 convolutional layer and an activation layer, the GELU activation function is used to maintain the expressive power of the model. In the second step, a 5×5 deep convolutional layer is used to capture the close relationship between pixels. In the third step, the features are input into the two branches respectively. Both branches are composed of deep hole convolution. The kernel sizes of deep hole convolution are 3 and 7 respectively. Their receptive fields are equivalent to standard convolutions with kernel sizes of 7 and 19 respectively, while the computational complexity is only 1 / 49 and 1 / 361 of that of standard convolution respectively. Deep hole convolution with different kernel sizes in the dual-branch structure can capture long-range relationships at different spatial scales. In the fourth step, the output features of the deep hole convolution are added and fused and input into the 1×1 point convolution to shrink the feature map to a smaller size, realize feature fusion in the channel dimension, thereby capturing global context information and realizing information interaction between space and channel. In the fifth step, the output of the 1×1 convolution is used as the input feature X in The self-attention weight map of X in Weighted, the output feature X is obtained out ; The process of extracting global context information is shown in the formula: To=Conv 1×1 (DW-DConv branch1 (X in )+DW-DConv branch2 (X in )) Among them, X in represents the input features, Represents the multiplication operation of the corresponding element matrix, Conv 1×1 Represents 1×1 convolution, DW-DConv branch_i represents the depth of the hollow convolution of the i-th branch, Att represents the attention weight map, X out Represents the output features.

8. The road surface crack detection method based on edge reconstruction network according to claim 1, characterized in that: In step 5, the multi-level feature fusion module concatenates edge features with shallow features and deep features respectively, and then aggregates the output features along two spatial directions to capture different scale feature representations. First, the H×W feature map is compressed into two scales of C×H×1 and C×1×W in the X and Y coordinate directions through an average pooling operation. In this process, the obtained feature map is aggregated into two independent direction-aware feature maps along the X and Y dimensions, thereby establishing the spatial correlation of the X and Y dimensions. The fused horizontal and vertical spatial information is encoded through a 3×3 convolution layer + BN + ReLU, and finally a coordinate attention weight map is generated through a Sigmoid function. On the other hand, a 3×3 depthwise separable convolution is used to encode spatial information and semantic information respectively, and the encoded spatial information and semantic information are weighted using two coordinate attention weight maps, respectively, to achieve adaptive fusion of feature information between different levels and different modalities.

9. The road surface crack detection method based on edge reconstruction network according to claim 1, characterized in that: In step 6, the sum of the binary cross entropy loss function, the edge loss function, and the detail loss function is used as the total loss function, where the binary cross entropy loss is as shown in the formula: L bce =-log(p t ) where p t reflects the closeness of the result to the true value category y, p t The larger it is, the more accurate the classification of the pixel is; The edge loss function fits the overall edge based on Dice's edge perception loss, applies the squash function on the gradient information graph, and obtains the edge prediction result. The definition of the obtained edge prediction result is shown in the formula: in and g ij Represent the edge prediction result and gradient information vector at (i, j), e ij represents the edge truth obtained directly from the detail label, α is a hyperparameter that controls the model's sensitivity to target edges, and α is set to 1 to balance inter-class differences and intra-class uniformity; The boundary refinement problem is expressed as a Dice coefficient maximization problem, as shown in the following formula: Among them, i represents the i-th pixel, Represents the segmentation map label value, Represents the edge prediction result, e represents the parameters of the segmentation network, H and W represent the height and width of the feature map respectively; In order to implement stochastic gradient descent during training, the marginal loss is constructed as follows: L edge (p d ,g d ,Θ)=1-Dice(p d ,g d ,Θ) During the training process, the detail loss function uses dice loss, which focuses on the detail features and balances the class imbalance problem, expressed as: Among them, i represents the i-th pixel, Represents the segmentation map label value, Represents the predicted segmentation map, H and W represent the height and width of the feature map respectively, ∈ represents the Laplace smoothing term, and ∈ is set to 1 to avoid zero division.

10. The road surface crack detection method based on edge reconstruction network according to claim 9, characterized in that: The binary cross entropy loss is used as the main loss function of the edge reconstruction network, and the detail loss function and edge loss function are set as auxiliary loss functions. During the training process, the output of the detail branch is calculated with the label value to obtain the results of the detail loss function and the binary cross entropy loss function, and the detail branch is optimized; Calculate the output of the edge branch and the label value to obtain the results of the edge loss function and the binary cross entropy loss function, and optimize the edge branch; Iterative training optimizes network parameters. After the loss stabilizes, the detection image is input into the trained neural network to obtain the final detection result image.

Images