Highway pavement disease sensing method based on deep learning fusion model

Abstract

The invention discloses a deep learning fusion model-based highway pavement disease perception method. The method comprises the following steps of: constructing a southeast university highway pavement disease image set; constructing a DL-M-PDS model used for the pixel-level semantic segmentation of the pavement disease in the high-speed scene; fusing the pixel-level segmentation information of the pavement disease image with the original image; an FCNN-PDP model used for sensing the pavement disease in the expressway scene is constructed; and carrying out an ablation experiment, and selecting a model FCNN-PDP with the best pavement disease perception performance in the highway scene. According to the construction method of the road surface disease image data set in the high-speed scene, the road surface disease sensing method in the high-speed scene based on the YOLOv8 target detection network and the DeepLabv3 semantic segmentation network is constructed, and therefore high-precision real-time detection of highway road surface diseases is achieved.

Description

Technical Field

[0001] The present invention belongs to the research field of intelligent transportation and smart highways, and specifically relates to a highway pavement disease perception method based on a deep learning fusion model. Background Art

[0002] With the construction of a huge highway traffic network, the problems of highway maintenance and management have become increasingly prominent. Under the long-term effects of vehicle loads and external natural conditions, different types of road surface diseases have gradually appeared on the highway pavement, which has greatly reduced the road performance, road quality, service level and service quality, and also caused certain road traffic safety hazards. Therefore, the present invention proposes a highway pavement disease perception method based on a deep learning fusion model, which combines the target detection network for highway pavement disease detection and classification with the semantic segmentation network for realizing pixel-level segmentation of highway pavement diseases, thereby solving the problem of high-precision and high-real-time detection of highway pavement diseases. Summary of the invention

[0003] Purpose of the invention: In order to overcome the shortcomings of the prior art, a highway pavement defect perception method based on a deep learning fusion model is provided. A highway pavement defect image set of Southeast University is established, and a pavement defect perception method for highway pavement target detection and pixel-level semantic segmentation fusion model in high-speed scenes is constructed, which can provide technical support for highway disease perception in high-speed scenes.

[0004] Technical solution: To achieve the above purpose, the present invention provides a road pavement disease perception method based on a deep learning fusion model, comprising the following steps:

[0005] S1: Construct the Southeast University highway pavement disease image collection.

[0006] S2: Build a DL-M-PDS model for pixel-level semantic segmentation of road surface defects in high-speed scenarios.

[0007] S3: Fusion of pixel-level segmentation information of the pavement damage image with the original image.

[0008] S4: Construct an FCNN-PDP model for pavement damage perception in highway scenarios.

[0009] S5: Perform ablation experiments and select the model FCNN-PDP with the best performance in road surface disease perception in highway scenarios.

[0010] Furthermore, the specific method of constructing the Southeast University highway pavement disease image set in step S1 is as follows:

[0011] S1-1: A total of 6,000 original images of highway pavement defects were collected in expressway scenarios, including 1,000 transverse cracks, 1,000 potholes, 2,000 others, 1,000 network cracks, and 1,000 longitudinal cracks. The pixel accuracy is accurate to the millimeter level, and the resolution is 2048×2000.

[0012] S1-2: Manually label the kilometer road surface disease image dataset using labelImg software.

[0013] S1-3: In the detection data set part, target boxes are drawn for each disease type and marked as five categories: transverse crack, longitudinal crack, net crack, pit slot, and others.

[0014] S1-4: Segment the parts and outline the shapes of the damages. Mark the damaged parts as damage and the unmarked parts as background.

[0015] S1-5: Before training each model, randomly extract image data to divide it into training set and test set.

[0016] Furthermore, the specific contents of constructing the DL-M-PDS model for pixel-level semantic segmentation of road surface defects in high-speed scenes in step S2 are:

[0017] S2-1: Normalize the highway pavement damage image with a size of 2048×2000×3 to a size of 512×512×3 and input it into the DL-M-PDS model.

[0018] S2-2: The input image is extracted through MobileNetV3-small to generate a road disease feature map.

[0019] S2-3: Input the above pavement damage feature map into the decoder upsampling layer, and output a pavement damage feature map with a size of 32×32×1280.

[0020] S2-4: Through the decoder convolutional layer, a road surface disease feature map with a size of 32×32×256 is output.

[0021] S2-5: Output the final pavement damage segmentation result through the classifier.

[0022] Furthermore, in step S2-2, the input image is subjected to feature extraction by MobileNetV3-small to generate a pavement disease feature map. The specific content is:

[0023] ① The first depth-separable convolutional layer with a convolution kernel size of 3×3 and a stride of 2 outputs a pavement disease feature map with a size of 256×256×16.

[0024] ② Continuously pass through 13 MBConv6 modules, and add additional batch normalization and NL activation functions after each module to output a pavement disease feature map of size 32×32×160. Each MBConv6 module consists of a 1×1 convolutional layer, a Depthwise convolutional layer with a convolution kernel of 3×3, an SE module, and a 1×1 convolutional layer that restores the number of channels to the original dimension.

[0025] ③ Through the convolution kernel with a size of 1×1, a pavement disease feature map with a size of 16×16×1280 is output.

[0026] ④The pavement disease feature map size is changed to 1×1×1280 through a fully connected layer.

[0027] Furthermore, the specific contents of fusing the pixel-level segmentation information of the pavement damage image with the original image in step S3 are as follows:

[0028] S3-1: Binarize the segmented output result and calculate the Euclidean distance d between the three-channel values ​​of each pixel of the output image and (0,0,0) and (128,0,0) respectively. 1 With d 2 , if d 1 >d 2 , then adjust the pixel to (128,0,0), otherwise adjust it to (0,0,0).

[0029] S3-2: Weighted fusion of the binary image obtained by segmentation and the original image, setting the transparency α of the binary image to 0.3, the transparency α of the original image to 0.7, the image format is RGB format, and the transparency blending function is used to superimpose the binary image and the original image to obtain a superimposed image with more semantic information. The superposition formula is as follows:

[0030] imgC=imgA·(1-α)+imgB·α

[0031] Among them, imgC is the superimposed image, imgA is the original image, imgB is the binary image after segmentation, and α is the transparency of the binary image.

[0032] Furthermore, the specific method of constructing the FCNN-PDP model for road surface disease perception in the highway scene in step S4 is as follows:

[0033] S4-1: The pavement damage image with a size of 2048×2000×3 is normalized to 640×640×3 and input into the YOLOv8s-PDD model.

[0034] S4-2: Build C2F-D (C2F-Damage) module.

[0035] S4-3: The pavement damage feature map is processed through two CBL convolution modules of size 3×3 and one C2F-D convolution module to obtain a pavement damage feature map of size 160×160×128. It then passes through three CBL and C2F-D combined convolution modules and an SPPF multi-scale feature pyramid pooling structure to obtain a pavement damage feature map of size 20×20×1024.

[0036] S4-4: Construct the BiFPN-Neck structure and generate 80×80×256, 40×40×512, and 20×20×512 pavement disease feature maps at different network levels.

[0037] S4-5: After the first detect module, a road surface disease feature map with a size of 80×80×30 is obtained, where the output dimension is related to the number of target categories. In this article, the road surface disease images are divided into five categories, so the number of output channels is 3×(5+5)=30.

[0038] S4-6: After a convolutional module composed of CBS and C2F and a second detect module, a pavement disease feature map with a size of 40×40×30 is obtained.

[0039] S4-7: After a convolutional module composed of CBS and C2F and the third detect module, a pavement pavement disease feature map with a size of 20×20×30 is obtained.

[0040] S4-8: Generate prediction boxes of different sizes for the three-dimensional pavement disease feature map, and perform border regression and non-maximum suppression (NMS) on the prediction boxes to obtain the final detection and classification results, and output the detection, classification, and target edge results at the same time.

[0041] Furthermore, the specific method of constructing the C2F-D (C2F-Damage) module in step S4-2 is as follows:

[0042] ①Replace the first ordinary convolution layer of the C2F module with partial convolution (PConv).

[0043] ② Connect two ordinary convolutional layers of size 1×1 as a reverse residual block, and add a normalization layer and ReLU activation function between the two 1×1Conv.

[0044] ③ Use shortcut connection to alleviate the gradient vanishing and expansion problems caused by increasing model depth.

[0045] Furthermore, the specific method of constructing the BiFPN-Neck structure in step S4-4 is as follows:

[0046] ① Construct a series of bidirectionally connected pyramid modules, each of which contains a set of bidirectionally connected feature maps, which are composed of multiple bidirectionally connected input feature maps and output feature maps for information transmission and fusion.

[0047] ② Use the Fusion structure to give feature maps of different resolutions additional weights for weighted fusion, so that the network can assign appropriate importance to each input feature map during the learning process. The weighted fusion process of the nth layer is as follows, where w i Represents feature map I i The corresponding weight, each w i Then add the rectified linear activation function to ensure w i Non-negative. ∈ is set to 0.0001 to mitigate potential numerical instability;

[0048]

[0049] ③ Considering the stability of training, the weight range is scaled between [0,1], and the fast-normalized fusion method is selected as the weighted fusion method. The specific formula is as follows:

[0050]

[0051] ④ Generate 80×80×256, 40×40×512, and 20×20×512 pavement disease feature maps at different network levels.

[0052] Furthermore, the step S5 performs an ablation experiment to select the FCNN-PDP model with the best pavement damage perception performance in the highway scene.

[0053] Beneficial effects: Compared with the existing technology, the present invention achieves high-precision perception of highway pavement defects in high-speed scenarios based on deep learning technology, improves model performance, and can provide technical support for intelligent operation and maintenance of highways. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Schematic diagram of the FCNN-PDP model structure.

[0055] Figure 2The left figure is the FCNN-PDP model training loss change graph and the right figure is the FCNN-PDP model verification loss change graph. DETAILED DESCRIPTION

[0056] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments.

[0057] The present invention provides a road pavement disease perception method based on a deep learning fusion model, comprising the following steps:

[0058] S1: Constructing the Southeast University highway pavement disease image collection:

[0059] S1-1: A total of 6,000 original images of highway pavement defects were collected in expressway scenarios, including 1,000 transverse cracks, 1,000 potholes, 2,000 others, 1,000 network cracks, and 1,000 longitudinal cracks. The pixel accuracy is accurate to the millimeter level, and the resolution is 2048×2000.

[0060] S1-2: Manually label the kilometer road surface disease image dataset using labelImg software.

[0061] S1-3: In the detection data set part, target boxes are drawn for each disease type and marked as five categories: transverse crack, longitudinal crack, net crack, pit slot, and others.

[0062] S1-4: Segment the parts and outline the shapes of the damages. Mark the damaged parts as damage and the unmarked parts as background.

[0063] S1-5: Before training each model, randomly extract image data to divide it into training set and test set.

[0064] S2: Building a DL-M-PDS model for pixel-level semantic segmentation of road surface defects in high-speed scenarios:

[0065] S2-1: Normalize the highway pavement damage image with a size of 2048×2000×3 to a size of 512×512×3 and input it into the DL-M-PDS model.

[0066] S2-2: The input image is extracted through MobileNetV3-small to generate a road disease feature map.

[0067] ① The first depth-separable convolutional layer with a convolution kernel size of 3×3 and a stride of 2 outputs a pavement disease feature map with a size of 256×256×16.

[0068] ② Continuously pass through 13 MBConv6 modules, and add additional batch normalization and NL activation functions after each module to output a pavement disease feature map of size 32×32×160. Each MBConv6 module consists of a 1×1 convolutional layer, a Depthwise convolutional layer with a convolution kernel of 3×3, an SE module, and a 1×1 convolutional layer that restores the number of channels to the original dimension.

[0069] ③ Through the convolution kernel with a size of 1×1, a pavement disease feature map with a size of 16×16×1280 is output.

[0070] ④ The pavement disease feature map is resized to 1×1×1280 through a fully connected layer.

[0071] S2-3: Input the above pavement damage feature map into the decoder upsampling layer, and output a pavement damage feature map with a size of 32×32×1280.

[0072] S2-4: Through the decoder convolutional layer, a road surface disease feature map with a size of 32×32×256 is output.

[0073] S2-5: Output the final pavement damage segmentation result through the classifier.

[0074] S3: fusion of the pixel-level segmentation information of the pavement damage image and the original image:

[0075] S3-1: Binarize the segmented output result and calculate the Euclidean distance d between the three-channel values ​​of each pixel of the output image and (0,0,0) and (128,0,0) respectively. 1 With d 2 , if d 1 >d 2 , then adjust the pixel to (128,0,0), otherwise adjust it to (0,0,0).

[0076] S3-2: Weighted fusion of the binary image obtained by segmentation and the original image, setting the transparency α of the binary image to 0.3, the transparency α of the original image to 0.7, the image format is RGB format, and the transparency blending function is used to superimpose the binary image and the original image to obtain a superimposed image with more semantic information. The superposition formula is as follows:

[0077] imgC=imgA·(1-α)+imgB·α

[0078] Among them, imgC is the superimposed image, imgA is the original image, imgB is the binary image after segmentation, and α is the transparency of the binary image.

[0079] S4: Construct an FCNN-PDP model for pavement damage perception in highway scenarios;

[0080] S4-1: The pavement damage image with a size of 2048×2000×3 is normalized to 640×640×3 and input into the YOLOv8s-PDD model.

[0081] S4-2: Build C2F-D (C2F-Damage) module.

[0082] ①Replace the first ordinary convolution layer of the C2F module with partial convolution (PConv).

[0083] ② Connect two ordinary convolutional layers of size 1×1 as a reverse residual block, and add a normalization layer and ReLU activation function between the two 1×1Conv.

[0084] ③ Use shortcut connection to alleviate the gradient vanishing and expansion problems caused by increasing model depth.

[0085] S4-3: The pavement damage feature map is processed through two CBL convolution modules of size 3×3 and one C2F-D convolution module to obtain a pavement damage feature map of size 160×160×128. It then passes through three CBL and C2F-D combined convolution modules and an SPPF multi-scale feature pyramid pooling structure to obtain a pavement damage feature map of size 20×20×1024.

[0086] S4-4: Construct the BiFPN-Neck structure and generate 80×80×256, 40×40×512, and 20×20×512 pavement disease feature maps at different network levels.

[0087] ① Construct a series of bidirectionally connected pyramid modules, each of which contains a set of bidirectionally connected feature maps, which are composed of multiple bidirectionally connected input feature maps and output feature maps for information transmission and fusion.

[0088] ② Use the Fusion structure to give feature maps of different resolutions additional weights for weighted fusion, so that the network can assign appropriate importance to each input feature map during the learning process. The weighted fusion process of the nth layer is as follows, where w i Represents feature map I i The corresponding weight, each w i Then add the rectified linear activation function to ensure w iNon-negative. ∈ is set to 0.0001 to mitigate potential numerical instabilities.

[0089]

[0090] ③ Considering the stability of training, the weight range is scaled between [0,1], and the fast-normalized fusion method is selected as the weighted fusion method. The specific formula is as follows:

[0091]

[0092] ④ Generate 80×80×256, 40×40×512, and 20×20×512 pavement disease feature maps at different network levels.

[0093] S4-5: After the first detect module, a road surface disease feature map with a size of 80×80×30 is obtained, where the output dimension is related to the number of target categories. In this article, the road surface disease images are divided into five categories, so the number of output channels is 3×(5+5)=30.

[0094] S4-6: After a convolutional module composed of CBS and C2F and a second detect module, a pavement disease feature map with a size of 40×40×30 is obtained.

[0095] S4-7: After a convolutional module composed of CBS and C2F and the third detect module, a pavement pavement disease feature map with a size of 20×20×30 is obtained.

[0096] S4-8: Generate prediction boxes of different sizes for the three-dimensional pavement disease feature map, and perform border regression and non-maximum suppression (NMS) on the prediction boxes to obtain the final detection and classification results, and output the detection, classification, and target edge results at the same time.

[0097] S5: Perform ablation experiments and select the model FCNN-PDP with the best performance in road surface disease perception in highway scenarios. Based on the established Southeast University highway road surface disease image dataset, the training, validation, and test sets are divided into 8:1:1, and the dataset format is VOC2007 format. The Adam optimizer is used in the model training process, the number of training rounds is 300, the momentum is set to 0.937, the learning rate decay strategy is cos, and the weight decay is set to 0.

[0098] Precision, recall, AP (Average Precision), and mAP (mean Average Precision) are selected as the basis for the perception effect of the discriminant network for a certain category of pavement damage, where mAP is the average of APs of multiple categories. The evaluation index values ​​of the highway pavement perception fusion model FCNN-PDP are shown in Table 1, where Params is the model parameter quantity, FLOPs is the model calculation quantity, and real-time is the time used to detect an image. As can be seen from the table, FCNN-PDD has achieved good performance in detection accuracy, classification accuracy, model parameter quantity and real-time performance. First, the average classification accuracy of FCNN-PDP for various types of damage is 98.88%; secondly, the recall rate of detection of various types of damage is high, indicating that the pixel-level segmentation and superposition of asphalt pavement damage images by the segmentation model can effectively increase the amount of information contained in the pavement image and reduce the occurrence of missed detection; thirdly, Params is 68.17M and GFLOPs is 272.23G, which is more suitable for mobile use in engineering applications. Finally, the FCNN-PDP model also achieved good real-time performance, taking only 20.20ms to detect each image.

[0099] Table 1 Statistics for highway asphalt pavement disease detection and perception model evaluation

[0100]

[0101]

[0102] The present invention proposes three improved modules for the original YOLOv8s-PDD model (Y: unimproved YOLOv8s-PDD network; D: FCNN-PDP model replaces the original C2F module of the YOLOv8-PDD model with a C2F-D module; B: BiFPN-Neck structure is used as the neck network of the FCNN-PDD model; F: used for the fusion of the highway asphalt pavement disease detection model and the segmentation model). The three improved modules can be integrated into the original model separately. In order to verify the effectiveness of the improved modules, this section conducts an ablation experiment. In order to ensure that the fusion model can simultaneously output the location information and shape information of the pavement disease, the detection model and the segmentation model are fused after the second experiment. The ablation experiment results are shown in Table 2, where Params is the model parameter quantity, FLOPs is the model calculation amount, and real-time is the time used to detect an image.

[0103] Table 2 Comparison results of ablation experiments on different models

[0104]

[0105] After the highway asphalt pavement disease detection model was fused with the segmentation model, although the fusion segmentation led to an increase in the number of model parameters and calculations, the average classification accuracy of the model was significantly improved, with an increase of 2.1%. After the C2F-D module was used to replace the ordinary C2F module, the Params and GFLOPs of the model decreased by 2.81M and 6.34G, respectively, indicating that the parameters and complexity of the model were reduced, the feature extraction ability of the network was enhanced, the redundant information interference was suppressed, and the computational overhead was reduced. After the BiFPN-Neck structure was used to replace the traditional FPN as the Neck part of the fusion FCNN-PDP model, the average classification accuracy reached the optimal level of 98.92%, but at the same time, the model Params and GFLOPs increased by 1.80M and 4.42G, respectively. The final FCNN-PDP model reduces the model Params and GFLOPs to 68.17M and 272.23G respectively while maintaining the average classification accuracy. The model is more concise and has good real-time performance while achieving high-precision perception. It only takes 20.20s to detect an image.

[0106] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A highway pavement disease perception method based on a deep learning fusion model, characterized by: The steps include: S1: Construct a highway pavement disease image set; S2: Build a DL-M-PDS model for pixel-level semantic segmentation of road surface defects in high-speed scenarios; S3: fusion of the pixel-level segmentation information of the pavement damage image and the original image; S4: Construct an FCNN-PDP model for pavement damage perception in highway scenarios; S5: Perform ablation experiments and select the model FCNN-PDP with the best performance in road surface disease perception in highway scenarios.

2. The highway pavement disease perception method based on deep learning fusion model according to claim 1 is characterized by: The specific method of constructing the highway pavement disease image set in step S1 is as follows: S1-1: A total of 6,000 original images of road pavement defects were collected in highway scenarios, including 1,000 transverse cracks, 1,000 potholes, 1,000 reticular cracks, 1,000 longitudinal cracks, and 2,000 other types. The pixel accuracy was accurate to the millimeter level, and the resolution was 2048×2000. S1-2: Manually label the kilometer-long road surface disease image dataset using labelImg software; S1-3: In the detection data set part, target boxes are drawn for each disease type and labeled as five categories: transverse_crack, longitudinal_crack, net_crack, pit_slot, and others. S1-4: Segment the parts and outline the shapes of the diseases. The diseased parts are marked as damage, and the unmarked parts are marked as background. S1-5: Before training each model, randomly extract image data to divide it into training set and test set.

3. The highway pavement disease perception method based on deep learning fusion model according to claim 2 is characterized by: The specific method of constructing the DL-M-PDS model for pixel-level semantic segmentation of road surface defects in high-speed scenes in step S2 is as follows: S2-1: Normalize the highway pavement damage image with a size of 2048×2000×3 to a size of 512×512×3 and input it into the DL-M-PDS model; S2-2: The input image is extracted through MobileNetV3-small to generate a road surface disease feature map; S2-3: input the pavement damage feature map into the decoder upsampling layer, and output a pavement damage feature map with a size of 32×32×1280; S2-4: Through the decoder convolution layer, a road surface disease feature map with a size of 32×32×256 is output; S2-5: Output the final pavement damage segmentation result through the classifier.

4. The highway pavement disease perception method based on deep learning fusion model according to claim 3 is characterized by: The specific content of the step S2-2 in which the input image is subjected to feature extraction by MobileNetV3-small to generate the pavement disease feature map is: ① The first depth-separable convolutional layer with a convolution kernel size of 3×3 and a stride of 2 outputs a pavement disease feature map with a size of 256×256×16; ② Continuously pass through 13 MBConv6 modules, and add additional batch normalization and NL activation functions after each module to output a pavement disease feature map of size 32×32×160. Each MBConv6 module consists of a 1×1 convolutional layer, a Depthwise convolutional layer with a convolution kernel of 3×3, an SE module, and a 1×1 convolutional layer that restores the number of channels to the original dimension. ③ Through the convolution kernel with a size of 1×1, the pavement disease feature map with a size of 16×16×1280 is output; ④The pavement disease feature map size is changed to 1×1×1280 through a fully connected layer.

5. The highway pavement disease perception method based on deep learning fusion model according to claim 4 is characterized by: The specific method of fusing the pixel-level segmentation information of the pavement damage image with the original image in step S3 is as follows: S3-1: Binarize the output result after segmentation, and calculate the Euclidean distances d1 and d2 of the three-channel values ​​of each pixel of the output image with (0,0,0) and (128,0,0) respectively. If d1>d2, adjust the pixel to (128,0,0), otherwise adjust it to (0,0,0); S3-2: Weighted fusion of the binary image obtained by segmentation and the original image, setting the transparency α of the binary image to 0.3, the transparency α of the original image to 0.7, the image format is RGB format, and the transparency blending function is used to superimpose the binary image and the original image to obtain a superimposed image with more semantic information. The superposition formula is as follows: imgC=imgA·(1-α)+imgB·α Among them, imgC is the superimposed image, imgA is the original image, imgB is the binary image after segmentation, and α is the transparency of the binary image.

6. The highway pavement disease perception method based on deep learning fusion model according to claim 5 is characterized by: The specific method of constructing the FCNN-PDP model for road surface disease perception in the highway scene in step S4 is as follows: S4-1: The pavement damage image with a size of 2048×2000×3 is normalized to 640×640×3 and input into the YOLOv8s-PDD model; S4-2: Construction of C2F-D module; S4-3: The pavement disease feature map is processed by two CBL convolution modules of size 3×3 and one C2F-D convolution module to obtain a pavement disease feature map of size 160×160×128. Then, it is processed by three CBL and C2F-D convolution modules and an SPPF multi-scale feature pyramid pooling structure to obtain a pavement disease feature map of size 20×20×1024. S4-4: Construct the BiFPN-Neck structure and generate 80×80×256, 40×40×512, and 20×20×512 pavement disease feature maps at different network levels; S4-5: After the first detect module, a road surface disease feature map with a size of 80×80×30 is obtained, where the output dimension is related to the number of target categories. In this paper, the road surface disease images are divided into five categories, so the number of output channels is 3×(5+5)=30; S4-6: After a convolutional module composed of CBS and C2F and a second detect module, a pavement disease feature map with a size of 40×40×30 is obtained; S4-7: After a convolution module composed of CBS and C2F and the third detect module, a pavement pavement disease feature map with a size of 20×20×30 is obtained; S4-8: Generate prediction boxes of different sizes for the three-dimensional pavement disease feature map, and perform border regression and non-maximum suppression operations on the prediction boxes to obtain the final detection and classification results, and output the detection, classification, and target edge results at the same time.

7. The highway pavement disease perception method based on deep learning fusion model according to claim 6 is characterized by: The specific method of constructing the C2F-D module in step S4-2 is as follows: ①Replace the first ordinary convolution layer of the C2F module with partial convolution PConv; ② Connect two ordinary convolutional layers of size 1×1 as a reverse residual block, and add a normalization layer and ReLU activation function between the two 1×1Conv; ③ Use shortcut connection to alleviate the gradient vanishing and expansion problems caused by increasing model depth.

8. The highway pavement disease perception method based on deep learning fusion model according to claim 7 is characterized by: The specific method for constructing the BiFPN-Neck structure in step S4-4 is as follows: ① Construct a series of bidirectionally connected pyramid modules, each of which contains a set of bidirectionally connected feature maps, which are composed of multiple bidirectionally connected input feature maps and output feature maps for information transmission and fusion; ② Use the Fusion structure to give feature maps of different resolutions additional weights for weighted fusion, so that the network can assign appropriate importance to each input feature map during the learning process. The weighted fusion process of the nth layer is as follows, where w i Represents feature map I i The corresponding weight, each w i Then add the rectified linear activation function to ensure w i Non-negative; ∈ is set to 0.0001 to mitigate potential numerical instability; ③ Considering the stability of training, the weight range is scaled between [0,1], and the fast normalization fusion method is selected as the weighted fusion method. The specific formula is as follows: ④ Generate 80×80×256, 40×40×512, and 20×20×512 pavement disease feature maps at different network levels.

9. The highway pavement disease perception method based on deep learning fusion model according to claim 8 is characterized by: S5 conducted an ablation experiment and selected the FCNN-PDP model with the best pavement disease perception performance in highway scenarios. The specific method is as follows: first, the original pavement disease feature map is input into the pixel-level semantic segmentation model to obtain a binary highway pavement disease segmentation image. Secondly, the transparency blending function is used to superimpose the binary image with the original image to obtain a superimposed image with more semantic information. Finally, the semantic information superimposed image is input into the pavement disease target detection model in the highway scenario, and the final perception accuracy of the model is calculated.

Images