Bituminous pavement damage detection method
Through the asphalt pavement damage detection model of the improved Darknet53 network module and the recursive feature pyramid network module, the problems of low road surface detection accuracy and high cost are solved, and the automatic, rapid and accurate detection of asphalt pavement damage is achieved.
Patent Information
- Application Number
- CN202510903766.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-01
- Publication Date
- 2025-10-17
AI Technical Summary
Existing technologies for road damage detection suffer from low detection accuracy and high costs. In particular, due to the difference between the foreground of the road damage image and the detection target, severe image noise caused by lighting and shadows, and the complexity of the road damage location, the cost of combining multiple image data to determine road damage detection increases.
An asphalt pavement damage detection model based on the improved Darknet53 network module and the recursive feature pyramid network module is constructed. Through feature extraction, fusion and prediction box clustering screening methods, automatic detection of asphalt pavement damage areas is achieved.
It realizes the automated, rapid and accurate detection of asphalt pavement damage, and can fully identify six types of damaged areas including longitudinal cracks, transverse cracks, cracks, block cracks, potholes and looseness.
Smart Images
Figure CN120808294A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of road surface damage detection, and in particular to a method for detecting asphalt pavement damage. BACKGROUND
[0002] With the development and progress of computer vision technology, more and more fields have developed rapidly. Using machines to simulate human visual features can more efficiently analyze images and videos. Using computer vision to analyze images or videos is one of the most popular research topics at present, mainly including target detection, target tracking, image classification, image segmentation, action recognition and other related research directions. In recent years, road damage detection technology has developed rapidly.
[0003] Due to the recognition barrier of different road damage degrees, computer vision-based road damage detection can use computer vision technology to identify and judge road damage. However, in actual detection, due to the difference between the foreground of the road damage image and the detection target, the serious image noise caused by the light shadow, and the directional complexity of the road damage position itself, road detection has not been able to achieve good results. In addition, since the collected images are obtained in the vertical direction facing the road surface, this method has the problem of combining multiple image data to judge road damage detection, greatly increasing the use cost. SUMMARY
[0004] The present application provides a method for detecting asphalt pavement damage, which is based on an improved Darknet53 network module and a recursive feature pyramid network module to construct an asphalt pavement damage detection model, in order to overcome the problems of low road damage detection accuracy and high use cost.
[0005] In order to achieve the above purpose, the technical scheme of the present application is:
[0006] A method for detecting asphalt pavement damage, the specific steps comprising:
[0007] S1, acquiring asphalt pavement image data set through vehicle-mounted image acquisition device; the asphalt pavement image contains damage area;
[0008] S2, constructing an asphalt pavement damage detection model to process the asphalt pavement image data set, the processing process comprising:
[0009] S21, feature extraction step: constructing a feature extraction module to extract multi-scale crack feature maps from the asphalt pavement image data set;
[0010] S22, feature fusion step: constructing a feature fusion module to perform feature fusion on the multi-scale crack feature maps;
[0011] S3, a prediction box clustering and screening method based on confidence fusion, detecting the asphalt pavement image processed by the asphalt pavement damage detection model and outputting coordinate information of an asphalt pavement damage area and a corresponding confidence score.
[0012] Further, the feature extraction module in step S21 is an improved Darknet53 network module, which includes an initial convolution layer, a maximum pooling layer, and at least three series-connected residual blocks, the residual block is composed of a channel adjustment layer, a feature refinement layer, and a cross-connection structure; the process executed by the feature extraction module includes:
[0013] S211, inputting the image of the asphalt pavement image dataset to the initial convolution layer, preliminarily extracting the feature information of the crack edge and crack texture in the image and generating an initial feature map, and outputting to the maximum pooling layer;
[0014] S212, performing a down-sampling operation on the initial feature map through the maximum pooling layer to obtain a sampling feature map and output to the first residual block;
[0015] S213, receiving the input feature map through the channel adjustment layer in the first residual block, adjusting the channel number of the sampling feature map to obtain an adjusted feature map, and outputting to the corresponding feature refinement layer, wherein the input feature map in the first residual block is the sampling feature map;
[0016] S214, receiving the adjusted feature map through the feature refinement layer, extracting local features with high-order nonlinearity and capturing multi-directional feature information with spatial resolution enhancement to obtain a local refined feature map, and outputting to the cross-connection structure;
[0017] S215, receiving the sampling feature map and the local refined feature map through the cross-connection structure and performing an addition operation to obtain a crack feature map of the first residual block; outputting the crack feature map output by the first residual block to the feature fusion module and the next level residual block;
[0018] S216, taking the output of the residual block of the previous level as the input image of the next level, based on the crack feature map extraction process corresponding to steps S213 to S215, iteratively processing each residual block in turn to obtain the crack feature map corresponding to each residual block, and outputting to the feature fusion module, wherein the sizes of the crack feature maps corresponding to each residual block are different.
[0019] Further, the feature fusion module in step S22 is a four-layer feature pyramid network with recursive connection; the execution process of the feature fusion module includes:
[0020] S221, first feature pyramid processing: receiving different scale crack feature maps output by the feature extraction module through the first feature pyramid network, performing upsampling and downsampling fusion operations on the crack feature maps, and generating a first feature information map;
[0021] S222, first feature fusion processing: cascading and fusing the first feature information map and the crack feature map to generate a fusion feature map;
[0022] S223, second feature pyramid processing: receiving the fusion feature map through the second feature pyramid network, and performing upsampling and downsampling fusion operations on the fusion feature map to generate a second feature information map;
[0023] S224, second feature fusion processing: performing splicing operation on the first feature information map and the second feature information map to obtain a multi-scale fusion feature information map.
[0024] Further, the recursive feature pyramid network module in step S2 is used for feature fusion operation on the crack features to obtain different scale damage feature information, and the specific steps include:
[0025] S221, a four-layer feature pyramid network model is built, and four layers of feature maps with different output scales are extracted from the backbone network, wherein the output scale of the fourth scale feature map is larger than that of the traditional three-layer scale feature map, and the feature information of the fourth scale feature map and the adjacent upper layer map is spliced;
[0026] S222, a recursive feedback path is established, and the feedback information output by the feature pyramid is integrated into the backbone network to enhance the correlation of the feature maps; the recursive feedback process realizes twice information fusion, including: a first feature information map output from the backbone network to the feature pyramid network; and a second information output from the backbone network to the feature pyramid network after fusion.
[0027] Further, the prediction box clustering and screening method based on confidence fusion in step S3 includes the following steps:
[0028] S31, for the multi-scale fusion feature information map output by the asphalt pavement damage detection model, all candidate prediction boxes in the preset are sorted in descending order according to the confidence score, and the prediction box with the highest score is selected;
[0029] S32, the prediction box with the highest score and the remaining candidate prediction boxes are calculated by IoU, and the prediction boxes exceeding the set threshold are marked as the same object and removed from the candidate prediction boxes;
[0030] S33, the candidate prediction boxes marked as the same object are taken as clustering targets, the candidate prediction boxes are divided into a first clustering sub-cluster and a second clustering sub-cluster through K-Means, the sizes of the two clustering sub-clusters are compared, and the clustering sub-cluster containing fewer candidate prediction boxes is deleted, and the candidate prediction boxes in the remaining clustering sub-cluster are taken as to-be-fused candidate prediction boxes;
[0031] S34, the confidence of the to-be-fused candidate prediction box is taken as a weight, and a fused candidate prediction box is obtained, and a final screened result is obtained.
[0032] Further, the expression of the fused candidate prediction box is defined as:
[0033]
[0034] In the formula, Mi represents the confidence of the i th to-be-fused prediction box, A is the number of boxes needing to be fused, S represents the coordinate vector of the boundary box after fusion, Si represents the coordinate vector of the i th to-be-fused boundary box, the maximum confidence is taken as a final prediction score, and the score and S coordinate information are output as a final result.
[0035] The present application has the following beneficial effects:
[0036] The improved Darknet53 network module combined with the recursive feature pyramid network module is used to construct the asphalt pavement damage detection model, automatic detection of asphalt pavement damage is realized, and six kinds of damage regions, i.e., longitudinal cracks, transverse cracks, cracks, block cracks, pits and looseness in the asphalt pavement image can be completely, quickly and accurately detected. BRIEF DESCRIPTION OF DRAWINGS
[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0038] Figure 1 The flowchart of the detection method in the present application;
[0039] Figure 2 The structure diagram of the improved Darknet53 network module in the present application;
[0040] Figure 3 The structure diagram of the recursive feature pyramid network module in the present application;
[0041] Figure 4 The schematic diagram of the prediction box fusion in the present application;
[0042] Figure 5 The detection result figure in the present application. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.
[0044] The present embodiment provides a method for detecting damage of asphalt pavement, such as Figure 1 The specific steps of the method include:
[0045] S1, acquiring an asphalt pavement image dataset through a vehicle-mounted image acquisition device; the asphalt pavement image contains a damage area;
[0046] S2, constructing an asphalt pavement damage detection model to process the asphalt pavement image dataset, and the processing process includes:
[0047] S21, a feature extraction step: constructing a feature extraction module to extract a multi-scale crack feature map from the asphalt pavement image dataset;
[0048] S22, a feature fusion step: constructing a feature fusion module to perform feature fusion on the multi-scale crack feature map;
[0049] S3, a prediction box clustering and screening method based on confidence fusion, detecting the asphalt pavement image processed by the asphalt pavement damage detection model, and outputting the coordinate information of the asphalt pavement damage area and the corresponding confidence score.
[0050] Specifically, first, an asphalt pavement image dataset is acquired through a vehicle-mounted image acquisition device; the asphalt pavement image contains a damage area, which serves as a basic data source for damage detection;
[0051] Secondly, an asphalt pavement damage detection model is constructed to process the asphalt pavement image dataset, which serves to realize intelligent feature analysis of the pavement image; wherein the steps of realizing image processing include: constructing a feature extraction module to extract a multi-scale crack feature from the asphalt pavement image dataset, which serves to capture damage features of different sizes; constructing a feature fusion module to perform feature fusion on the multi-scale crack feature map, which serves to enhance the integrity of feature expression;
[0052] Finally, based on the confidence fusion prediction box clustering screening method, the asphalt pavement damage detection model processes the asphalt pavement image, outputs the coordinate information of the asphalt pavement damage area and the corresponding confidence score, which can accurately locate the damage area and evaluate the detection reliability.
[0053] In a specific embodiment, the vehicle-mounted image acquisition device is a vehicle-mounted camera parallel to the road surface, which acquires an asphalt pavement data set containing a damage area through the vehicle-mounted camera, covers all areas in front through a road horizontal view image, and displays all crack information.
[0054] Specifically, the feature extraction module is an improved Darknet53 network module.
[0055] As shown in Figure 2 The improved Darknet53 network module includes an initial convolutional layer, a pooling layer, and five groups of residual blocks (the number of residual blocks in each group is 1, 2, 8, 8, and 4, respectively). The specific description of the execution steps of the improved Darknet53 network module is as follows:
[0056] S211, input the image of the asphalt pavement image data set to the initial convolutional layer, preliminarily extract the feature information of the crack edge and crack texture in the image and generate an initial feature map, and output to the maximum pooling layer; the initial convolutional layer uses a convolutional kernel with a size of 3x3, a step of 1 and a padding of 1;
[0057] S212, perform down-sampling operation on the initial feature map through the maximum pooling layer, reduce the feature map size and the data amount while retaining the key feature information, and output the sampling feature map to the first residual block; the maximum pooling layer uses a maximum pooling kernel with a size of 2x2 and a step of 2;
[0058] S213, receive the input feature map through the channel adjustment layer in the first residual block to adjust the channel number of the sampling feature map; wherein the channel adjustment layer changes the number of channels according to the network design requirements to control the network complexity and the subsequent feature dimension to obtain an adjusted feature map and output to the corresponding feature refinement layer, wherein the input feature map in the first residual block is the sampling feature map; the channel adjustment layer is a 1x1 convolutional layer with a step of 1 and a padding of 0;
[0059] S214, receive the adjusted feature map through the feature refinement layer, extract local features with high-order nonlinearity and capture multi-directional feature information with enhanced spatial resolution to obtain a local refined feature map and output to the skip connection structure; the feature refinement layer is a 3x3 convolutional kernel with a step of 1 and a padding of 1;
[0060] S215, receiving the sample feature map and the local refined feature map through the cross-connection structure and performing addition operation, which aims to solve the gradient vanishing problem in deep network training, guaranteeing that the network can effectively train deep structure, and obtaining the crack feature map of the first residual block after addition;
[0061] S216, taking the output of the residual block of the previous stage as the input image of the next stage, based on the crack feature map extraction process corresponding to steps S213 to S215, using each subsequent residual block to iteratively process in turn to obtain the crack feature map corresponding to each residual block, and output to the feature fusion module, wherein the sizes of the crack feature maps corresponding to each residual block are different.
[0062] In specific embodiments, the input image of the improved Darknet53 network module is an image with a resolution of 640x640; the final output is four crack feature maps of the 5th layer, the 10th layer, the 27th layer and the 52nd layer;
[0063] As shown in Figure 3 The feature fusion module is a four-layer feature pyramid network with recursive connection; the four-layer feature pyramid network is built by additionally introducing an output with a larger scale from the backbone network on the original three-layer feature pyramid structure, then performing feature splicing with the feature information in the previous output scale, to obtain feature vectors with richer semantic information under four different image scales; using shallower layer features to detect smaller targets, using deeper layer features to detect larger targets, to enhance the damage detection effect of small targets on the road surface;
[0064] Secondly, considering the problems of mismatch and lack of relevance between each feature map after the feature pyramid layer and the path aggregation network and the backbone network features, the idea of recursion is introduced, and the feedback information output by each feature pyramid network is integrated into the backbone network, to further enhance the relevance of the feature maps of the previous and subsequent layers; Figure 3 The dashed line represents that the feedback information of the feature pyramid output is added to the backbone network, and the feature fusion process is performed on the four feature maps with different scales;
[0065] The execution process of the feature fusion module includes:
[0066] S221, first feature pyramid processing: receiving the crack feature maps with different scales output by the feature extraction module through the first feature pyramid network, performing upsampling and downsampling fusion operation on the crack feature maps, to generate the first feature information map; the resolution of the first feature information map is 20x20, 40x40, 80x80 and 160x160;
[0067] S222, first feature fusion processing: the one feature information graph is fused with the crack feature graph, and a fusion feature graph is generated;
[0068] S223, second feature pyramid processing: the fusion feature graph is received by a second feature pyramid network, and up-sampling and down-sampling fusion operations are performed on the fusion feature graph to generate a secondary feature information graph;
[0069] S224, second feature fusion processing: the one feature information graph is spliced with the secondary feature information graph to obtain a multi-scale fusion feature information graph.
[0070] Specifically, the prediction box clustering and screening method based on confidence fusion comprises the following steps:
[0071] S31, for the multi-scale fusion feature information graph output by the asphalt pavement damage detection model, all candidate prediction boxes in the multi-scale fusion feature information graph are sorted in descending order according to the confidence scores, and the prediction box with the highest score is selected;
[0072] S32, the prediction box with the highest score is calculated with the rest of the candidate prediction boxes, and the prediction boxes exceeding the set threshold are marked as the same object and removed from the candidate prediction boxes;
[0073] S33, the top-left corner coordinates of the candidate prediction boxes marked as the same object are taken as the clustering target, the candidate prediction boxes are divided into a first clustering sub-cluster and a second clustering sub-cluster by K-Means, the sizes of the two clustering sub-clusters are compared, and the clustering sub-cluster containing fewer candidate prediction boxes is deleted, and the candidate prediction boxes in the remaining clustering sub-cluster are taken as the candidate prediction boxes to be fused;
[0074] S34, as shown in the formula, the confidence of the candidate prediction box to be fused is taken as the weight and the fusion candidate prediction box is performed to obtain the final screened result. Figure 4
[0075] Specifically, the expression of the fusion candidate prediction box is defined as:
[0076]
[0077] In the formula, Mi represents the confidence of the i-th prediction box to be fused; A is the number of boxes to be fused; S represents the coordinate vector of the boundary box after fusion; Si represents the coordinate vector of the i-th boundary box to be fused; the maximum confidence is taken as the final prediction score, and the score and S coordinate information are output as the final result.
[0078] Figure 5 The detection result of the application is shown, and it can be seen from the figure that the cracks on the road are marked by different color marking boxes.
[0079] The application has the following beneficial effects:
[0080] The improved Darknet53 network module combined with the recursive feature pyramid network module constructs the asphalt pavement damage detection model, realizes the automatic detection of the asphalt pavement damage, and can completely, quickly and accurately detect six damage areas in the asphalt pavement image, i.e., longitudinal cracks, transverse cracks, cracks, block cracks, pits and loose.
[0081] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the application.
Claims
1. A method for detecting damage to an asphalt pavement, characterized in that: The specific steps include: S1. Acquire an asphalt pavement image dataset using a vehicle-mounted image acquisition device; the asphalt pavement image includes a damaged area; S2. Construct an asphalt pavement damage detection model and process the asphalt pavement image dataset. The processing process includes: S21, feature extraction step: constructing a feature extraction module to extract a multi-scale crack feature map from the asphalt pavement image dataset; S22, feature fusion step: constructing a feature fusion module to perform feature fusion on the multi-scale crack feature map; S3. A prediction box clustering screening method based on confidence fusion is used to detect the asphalt pavement image processed by the asphalt pavement damage detection model, and output coordinate information of the asphalt pavement damage area and the corresponding confidence score.
2. The asphalt pavement damage detection method according to claim 1, characterized in that: The feature extraction module in step S21 is an improved Darknet53 network module, which includes an initial convolutional layer, a maximum pooling layer, and at least three residual blocks connected in series. The residual block consists of a channel adjustment layer, a feature refinement layer, and a spanning connection structure. The process performed by the feature extraction module includes: S211, inputting the image of the asphalt pavement image dataset into the initial convolution layer, preliminarily extracting feature information of crack edges and crack textures in the image and generating an initial feature map, which is output to the maximum pooling layer; S212, performing a downsampling operation on the initial feature map through a maximum pooling layer to obtain a sampled feature map and output it to a first residual block; S213. Receive an input feature map through a channel adjustment layer in the first residual block, adjust the number of channels of the sampled feature map, obtain an adjusted feature map, and output the adjusted feature map to a corresponding feature refinement layer, wherein the input feature map in the first residual block is the sampled feature map; S214, receiving the adjusted feature map through the feature refinement layer, extracting local features with high-order nonlinearity and capturing multi-directional feature information with spatial resolution enhancement, obtaining a local refined feature map, and outputting it to the spanning connection structure; S215: receiving the sampling feature map and the local refined feature map through the cross connection structure and performing an addition operation to obtain a crack feature map of the first residual block; outputting the crack feature map output by the first residual block to the feature fusion module and the next-level residual block; S216. Use the output of the residual block of the previous level as the input image of the next level. Based on the crack feature map extraction process corresponding to steps S213 to S215, use the subsequent residual blocks to perform iterative processing in sequence to obtain the crack feature maps corresponding to each residual block, and output them to the feature fusion module, where the sizes of the crack feature maps corresponding to each residual block are different.
3. The asphalt pavement damage detection method according to claim 1, characterized in that: The feature fusion module in step S22 is a four-layer feature pyramid network with recursive connections; The execution process of the feature fusion module includes: S221, first feature pyramid processing: receiving the crack feature maps of different scales output by the feature extraction module through a first feature pyramid network, performing upsampling and downsampling fusion operations on the crack feature maps, and generating a primary feature information map; S222, first feature fusion processing: cascade-fusing the primary feature information map with the crack feature map to generate a fused feature map; S223, second feature pyramid processing: receiving the fused feature map through a second feature pyramid network, performing upsampling and downsampling fusion operations on the fused feature map, and generating a secondary feature information map; S224, second feature fusion processing: performing a splicing operation on the first feature information map and the second feature information map to obtain a multi-scale fused feature information map.
4. The asphalt pavement damage detection method according to claim 1, characterized in that: The prediction box clustering screening method based on confidence fusion in step S3 includes the following steps: S31. For the multi-scale fusion feature information graph output by the asphalt pavement damage detection model, sort all the candidate prediction frames preset therein according to the confidence scores from large to small, and select the prediction frame with the highest score; S32, calculate the IoU between the prediction box with the highest score and the remaining candidate prediction boxes, mark the prediction boxes that exceed the set threshold as the same object, and remove them from the candidate prediction boxes; S33, taking candidate prediction boxes marked as the same object as clustering targets, dividing the candidate prediction boxes into a first cluster sub-cluster and a second cluster sub-cluster by K-Means, comparing the sizes of the two cluster sub-clusters and deleting the cluster sub-cluster containing fewer candidate prediction boxes, and taking the candidate prediction boxes in the remaining cluster sub-clusters as candidate prediction boxes to be fused; S34: Taking the confidence of the candidate prediction frame to be fused as a weight and fusing the candidate prediction frame to obtain the final screening result.
5. The asphalt pavement damage detection method according to claim 4, characterized in that: The expression for defining the fused candidate prediction box is: Where Mi represents the confidence of the i-th prediction box to be fused; A is the number of boxes to be fused; S represents the coordinate vector of the bounding box after fusion; Si represents the coordinate vector of the i-th bounding box to be fused; the maximum confidence is used as the final prediction score, and this score and S coordinate information are output as the final result.