A method for identifying internal road surface defects images

Abstract

This application discloses a method for identifying images of internal pavement defects. Based on the principles of the random forest algorithm and a data set of internal pavement defects, multiple sets of identification weight coefficients for internal pavement defects are generated, and rules for distinguishing and filtering defects are established. In addition, based on the results of internal pavement defect identification using a deep learning network, a canny edge algorithm is added to highlight the differences between the defect location and its surroundings on the radar map, thereby improving the accuracy of internal pavement defect identification. Combined with analysis of actual identification results, this method can further improve the accuracy of internal pavement defect identification, with an accuracy rate of approximately 95%.

Description

Technical Field

[0001] The present application relates to the technical field of disease identification and processing, and in particular to a method for identifying images of internal road surface diseases. Background Art

[0002] The detection, analysis and evaluation of the internal health status of the pavement can provide reliable and effective data support for scientific decision-making on pavement maintenance. The identification of internal pavement diseases, as an important part of the detection and evaluation of the internal health status of the pavement, plays a very important role in road maintenance and repair.

[0003] However, in the existing technology, when identifying internal road surface defects based on ground-penetrating radar images, the judgment is mainly made by comparing the feature differences between the defect location and its surrounding locations. However, the existing deep learning network model lacks the ability to judge feature differences, which can easily lead to misjudgment of internal defects. In addition, a single weight model is used for target identification of internal road surface defects. This model has certain limitations in judgment and can easily lead to misjudgment and missed judgments. Summary of the Invention

[0004] In order to solve the above technical problems, an embodiment of the present application provides a method for identifying images of internal road defects. Based on the existing deep learning convolutional network, an edge algorithm is introduced to highlight the differences in the locations of internal defects. On this basis, combined with the technical principles of random forests, multiple sets of road internal defect data sets and related weight models are established to identify the detection data, thereby improving the identification effect of internal road defects.

[0005] The present invention provides a method for identifying road surface internal disease images, including the following steps:

[0006] Step S1) obtaining radar data of the detected road section and preprocessing the radar data;

[0007] Step S2) constructing a dataset of internal road surface defects and classifying them based on the types of defects to form a dataset classified by defect type;

[0008] Step S3) The obtained data sets are fed into different deep learning network models, and the manual labeling results of various diseases in the data sets are analyzed in turn to form a disease discrimination weight model;

[0009] Step S4) obtaining the edge amplitude of the marked frame position in the data set based on the Canny edge algorithm, calculating the edge amplitude of each pixel point in the marked frame, and determining the edge amplitude threshold of each type of disease;

[0010] Step S5) pre-processes the radar data to be identified and respectively inputs them into at least two weighted discrimination models for discrimination, and filters the road surface internal disease discrimination results based on the model discrimination results and disease filtering rules.

[0011] Furthermore, the pre-processing in step S1) includes at least one of zero bias removal, zero point adjustment, and data gain, and the gain coefficient of the waveform is adjusted to enhance the characteristics of crack damage inside the pavement.

[0012] Furthermore, when the pre-processing adopts zero point adjustment, it includes:

[0013] Select a bridge slab or repair section location on the road section to be inspected;

[0014] Count the number of points at each channel of the bridge slab or repair section of the inspected road section;

[0015] The zero point adjustment of each channel is achieved by subtracting a preset fixed value from the number of points of each channel in turn; the fixed value is selected based on empirical values.

[0016] Furthermore, in the data set construction process of step S2), a grayscale spectrum is uniformly adopted, wherein when marking defects on the slice image of the radar data, the marking box must cover all the defect features.

[0017] Furthermore, the defects in step S2) include three types: poor interlayer adhesion defects, loose and broken defects within the structure, and crack defects. Among them, poor interlayer adhesion defects and loose and broken defects within the structure are judged using radar profiles, and crack defects are judged using radar slices.

[0018] When the disease dataset was established, the ratio of various disease data in each dataset was 1:1:1, and there was no difference in the characteristics and marking rules of the same disease.

[0019] Furthermore, the deep learning network model in step S3) includes a YOLO network model, a Faster-RCNN artificial neural network model and an SSD artificial neural network model.

[0020] Furthermore, the CSP-based Backbone network in the YOLO network model is replaced with the EfficientViT-M0 module;

[0021] The ShuffleAttention mechanism is added to the Faster-RCNN network model;

[0022] The loss function in the SSD network model selects the EIOU function.

[0023] Furthermore, the discrimination in step S5) is determined by combining the discrimination results under each weight model and the edge amplitude.

[0024] Furthermore, the edge amplitude includes a first edge amplitude boundary and a second edge amplitude boundary; the value of the first edge amplitude boundary is greater than the value of the second edge amplitude boundary;

[0025] Get the data of each weight model at the same position on the detection section,

[0026] If there are at least two weight model data with the same disease type and the edge amplitude exceeding the first edge amplitude boundary, then the disease at that location on the detected road section is considered to be of that type;

[0027] If there is no data from at least two weight models, and the damage edge amplitude does not exceed the first edge amplitude boundary, then the damage at that location on the detected road section is considered a misjudgment;

[0028] If there are at least two weight model data with different damage types but both exceeding the first edge amplitude boundary, it is determined that there is a damage at that location on the detected road section, but the type requires manual assistance to determine;

[0029] If there are data from at least two weighted models, and the damage edge amplitudes do not exceed the second edge amplitude boundary, regardless of whether the results of the models are the same, the damage at that location on the detected road section is considered a misjudgment;

[0030] If there are data from at least two weighted models, and the damage edge amplitudes of both models only exceed the second edge amplitude boundary, regardless of whether the results of the models are the same, it is determined that there is a damage at that location on the detected road section and the type of damage requires manual assistance.

[0031] If there is no data for at least two weight models, and the damage edge amplitude exceeds the first edge amplitude boundary, it is determined that whether there is a damage at that location on the detected road section and the type require manual assistance to determine.

[0032] The present invention demonstrates the following benefits: Based on the principles of a random forest algorithm, this method generates multiple sets of pavement defect identification weight coefficients from a pavement defect dataset and establishes rules for identifying and filtering defects. Furthermore, based on the results of a deep learning network identifying pavement defects, a canny edge algorithm is added to highlight the differences between the defect location and its surroundings on radar maps, thereby improving the accuracy of pavement defect identification. Combined with analysis of actual identification results, this method can further improve the accuracy of pavement defect identification, achieving an accuracy rate of approximately 95%. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1 is a flow chart of this application;

[0035] Figure 2 This is a flowchart of radar data zero point adjustment of this application;

[0036] Figure 3 yes Figure 2 The effect diagram of radar file A is generated based on the original radar detection data;

[0037] Figure 4 yes Figure 2 The rendering of the bridge slab and repair section position in the inspection section is selected;

[0038] Figure 5 yes Figure 2 Generate the effect diagram of radar file B;

[0039] Figure 6 It is the characteristic map of internal road surface defects in this application. DETAILED DESCRIPTION

[0040] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0042] In the description of this application, it should be understood that the terms "upper", "lower", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on this application.

[0043] The present application is described below with reference to specific embodiments:

[0044] The present invention proposes a method for identifying internal road surface defects. Figure 1 As shown, specifically including:

[0045] Step S1) obtaining radar data of the road section to be inspected and preprocessing the radar data. The processing methods include: zero bias removal, zero point adjustment, and data gain. The gain coefficient of the waveform is appropriately adjusted to make the characteristics of the crack disease inside the road surface more obvious.

[0046] As a specific embodiment, the radar detection data obtained is a 14-channel three-dimensional array ground penetrating radar, and the antenna type is a 1.2 GHz ground-coupled radar antenna.

[0047] As a specific embodiment, the specific parameters of the preprocessing method here are:

[0048] When performing radar data preprocessing, the zero bias removal parameter is set to 90%, the data gain selects the compensation gain, the gain coefficient is generally controlled between 1.0-2.0, and the data chromatogram uniformly uses the grayscale chromatogram.

[0049] As a specific embodiment, in order to make the structural layer positions of each channel basically close and without obvious fractures, the zero point adjustment of the radar data in this scheme adopts a new zero point adjustment method, that is, select a bridge slab or repair section position on the detection section, and then count the number of points of each channel at the bridge slab or repair section position on the detection section, and subtract a fixed value in turn according to the number of points of each channel. The fixed value here should be as large as possible, but not exceed the "minimum value of the counted points of each channel - 5", so as to achieve zero point adjustment of each channel.

[0050] like Figure 2-5 As shown, Figure 2 This is a flowchart of radar data zero point adjustment for this application; Figure 3 for Figure 2 The effect diagram of radar file A is generated based on the original radar detection data, where the horizontal axis represents the number of electromagnetic waves emitted by the radar transmitting antenna (the number of electromagnetic waves), and the vertical axis represents the time when the echo of this electromagnetic wave is received (starting from the start of the electromagnetic wave emission). The time can be converted into the depth information of the target. The specific formula is: H=vt / 2 (where v represents the propagation speed of the electromagnetic wave, which is 1.0*10 in this patent) 8 m / s); Figure 4 for Figure 2 The rendering of the bridge slab and repair section position in the inspection section is selected; Figure 5 yes Figure 2 The effect diagram of radar file B is generated in the figure. Point A in the figure shows that the structural layer position characteristics of each channel are relatively continuous, without any disconnection or continuous fluctuation (mainly shallow depth features).

[0051] Step S2) constructs a dataset of internal road surface defects and classifies them based on the types of defects to form a dataset classified by defect type.

[0052] As a specific embodiment, this solution divides the types of defects into three categories: poor interlayer adhesion, looseness and cracks. When establishing a defect dataset, the dataset is established in the following manner:

[0053] i) The dataset uniformly adopts grayscale color spectrum, with the slice image size of 400*100 and the cross-section image size of 320*320. When marking diseases, the marking box should cover all disease features as much as possible and should not be too large.

[0054] ii) When establishing the internal pavement disease dataset, internal pavement diseases were categorized into three types: poor interlayer bonding, loose and broken structures, and cracks. Poor interlayer bonding and loose and broken structures were primarily diagnosed using radar profiles, while cracks were diagnosed using radar slices.

[0055] iii) After obtaining the data, the dataset should be divided in a way that ensures that the ratio of data of different types of diseases in the dataset is 1:1:1, and there are no significant differences in the characteristics and marking rules of similar diseases.

[0056] In step S3), the data sets obtained in step S2) are respectively introduced into a plurality of different deep learning network models for feature extraction, thereby obtaining a plurality of different pavement internal disease discrimination weight models.

[0057] As a specific embodiment, the present invention adopts three deep learning network models, namely, YOLO, Faster-RCNN and SSD, three artificial neural network algorithms, to analyze the manual labeling results of three types of internal pavement diseases in the data set in turn, and form a disease discrimination weight model.

[0058] Because the targets for identification are internal road surface defects detected by ground-penetrating radar (GPR), characterized by highlights, discontinuities in the same direction, and a certain degree of hyperbolic characteristics, this patent innovatively replaces the CSP-based Backbone network in the YOLO network model with EfficientViT-M0, adds ShuffleAttention (an attention mechanism) to the Faster-RCNN network model, optimizes the loss function in the SSD network model, and selects the EIOU function, which better matches radar defect characteristics, thereby optimizing the three automatic identification network models.

[0059] Furthermore, when there are defects inside the road surface, the electromagnetic waves emitted by the ground-penetrating radar will be reflected when encountering the location of the defect inside the road surface, thereby generating features such as highlights, coaxial discontinuities, and hyperbolas. Therefore, when identifying the ground-penetrating radar map, the identification of the internal road surface defects is mainly achieved by comparing whether there are obvious differences in features such as highlights, discontinuities, and hyperbolas at the selected location and the surrounding locations. Therefore, in the means of this application:

[0060] 1. Using EfficientViT-M0 to replace the CSP-based Backbone network in the YOLO network model can improve the algorithm's ability to capture long-range features, achieve global interaction, and identify differences in disease locations;

[0061] 2. Adding the ShuffleAttention mechanism to the Faster-RCNN network model can improve the comparison of global features when the network model labels candidate boxes, highlight the differences in diseases, and improve the quality of candidate boxes;

[0062] 3. Since the damage features in radar images are mostly long strips, and the format of the marker boxes generated by the SDD algorithm is relatively fixed, changing the loss function in the SSD network model to EIOU can improve the regression of targets with abnormal aspect ratios.

[0063] As a specific embodiment, by debugging the parameters of the three neural network models, the optimal feature extraction parameters of the three network models are obtained in turn, as shown below.

[0064] YOLO model: convolution kernel size is 3, stride is 2, epoch is 300, patience is 300, batch size is 16, initial learning rate is 0.01, cyclic learning rate is 0.001, optimizer uses SGD, its momenrum is 0.937, weight decay is 0.0005, IOU training threshold is 0.2, loss function is cross entropy loss function, and class is 3.

[0065] Faster-RCNN model: convolution kernel size is 3, stride is 2, epoch is 300, patience is 300, batch size is 16, initial learning rate is 0.01, cyclic learning rate is 0.001, optimizer uses SGD, its momenrum is 0.937, weight decay is 0.0005, IOU training threshold is 0.2, loss function is cross entropy loss function, and class is 3.

[0066] SSD model: convolution kernel size is 3, stride is 2, epoch is 400, patience is 400, batch size is 16, initial learning rate is 0.01, cyclic learning rate is 0.01, optimizer uses SGD, its momenrum is 0.937, weight decay is 0.0005, IOU training threshold is 0.2, loss function is cross entropy loss function, and class is 3.

[0067] Step S4) Calculate the edge amplitude of the marked box position in the data set based on the Canny edge algorithm, thereby determining the edge amplitude threshold for each type of disease. When using the edge algorithm to calculate the edge amplitude of each type of disease in the data set to determine the threshold, the edge amplitude of each pixel in the marked box should be calculated to determine the edge threshold for each type of disease.

[0068] Step S5) The radar data to be identified is pre-processed and brought into three weighted discrimination models for discrimination. The road surface internal disease discrimination results are filtered based on the model discrimination results and disease filtering rules.

[0069] As a specific embodiment, the radar data to be identified is preprocessed using the preprocessing method in step S1). When exporting the preprocessed radar map data, the cross-section size is 320*320, the number of electromagnetic signal channels is set to 320, the slice size is 400*100, the number of electromagnetic signal channels is set to 400, and the overlap of adjacent map data is 5%.

[0070] As a specific embodiment, when filtering the discrimination results, a first edge amplitude boundary and a second edge amplitude boundary are first established based on empirical values ​​and actual road surface objects, and numerically, the first edge amplitude boundary is greater than the second edge amplitude boundary. The discrimination rules mainly include:

[0071] (1) When detecting defects at the same location on a road section, if two or more of the three defect discrimination weight models have the same discrimination results and their edge amplitudes exceed the first edge amplitude boundary (in this embodiment, the first edge amplitude boundary = 0.8), then the location can be identified as a defect of this type;

[0072] (2) If a defect at the same location on a road section is detected by two or more of the three defect discrimination weight models and is identified as a defect, but the defect types are different and the edge amplitude exceeds the first edge amplitude boundary, then it can be determined that there is a defect at that location, but the determination of the defect type requires manual assistance;

[0073] (3) If defects at the same location on the road section are detected by two or more of the three defect discrimination weight models and the edge amplitude only exceeds the second edge amplitude boundary (the second edge amplitude boundary = 0.6 in this embodiment), regardless of whether the defect type is the same, then the defect at this location requires manual assistance for discrimination;

[0074] (4) When detecting a defect at the same location on a road section, if only one of the three sets of defect discrimination weight models identifies it as a defect, but the edge amplitude of the defect exceeds the first edge amplitude boundary, then the defect at that location requires manual assistance in identification;

[0075] (5) When detecting a defect at the same location on a road section, if only one of the three sets of defect discrimination weight models identifies it as a defect, but the edge amplitude of the defect is lower than the first edge amplitude boundary, the defect at that location is considered to be misjudged;

[0076] (6) When detecting defects at the same location on a road section, if two or more of the three defect discrimination weight models identify the defect as a defect, and the edge amplitude of the defect does not exceed the second edge amplitude boundary, the defect at that location is considered to be misjudged.

[0077] It should be noted that in this application, when calculating the edge amplitude of the radar spectrum, the amplitude calculation direction is mainly horizontal, and the calculation direction can be calculated from the outside of the marking frame to the inside. This can also achieve a comparison of the differences in characteristics of the surrounding areas of the internal disease location of the road surface, but it can reduce the amount of calculation to a certain extent.

[0078] Compared with the existing technology, this application considers the importance of zero-point adjustment to radar data and proposes a method for zero-point adjustment of radar data based on bridge slabs or repair sections. This method can make the radar data of each channel better correspond in the depth direction, making the characteristics of the disease, especially the crack characteristics, more obvious;

[0079] Based on the principle of random forest, this application proposes to use multiple artificial neural network models to extract the characteristics of internal pavement diseases based on the pavement internal disease dataset. The results are as follows Figure 6 As shown in the figure, corresponding disease discrimination and filtering rules are established to improve the recognition accuracy of internal road surface diseases. Moreover, based on the existing deep learning neural network, the canny edge algorithm is introduced to determine the edge amplitude threshold of various diseases, highlighting the difference between the disease and its surrounding features, thereby improving the recognition accuracy of internal road surface diseases.

[0080] This patent compares the effects of different algorithm optimizations and algorithm combinations in turn, as shown in Table 1. It can be seen that the optimization of several algorithms can improve the accuracy (P) of disease identification. The combined algorithm can further improve the accuracy (P) of disease identification and can greatly improve the detection rate (R) of diseases.

[0081] Table 1 Comparison of recognition results of different algorithms

[0082]

[0083] Table 1 clearly shows that, judging solely from the efficiency of the optimized algorithms, the optimization of each model significantly improves overall efficiency. The optimized YOLO, Faster-RCNN, and SSD models all show significant improvements in the four key evaluation parameters compared to pre-optimization results. In the final algorithm combination, utilizing the weighted model formed in step S5, combining these three optimized modules yields a substantial improvement in overall evaluation parameters.

[0084] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0085] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations (such as quantity, shape, position, etc.) can be made to the technical solution of the present invention, and these equivalent transformations are all protected by the present invention.

Claims

1. A method for identifying internal road surface defects, characterized in that: The steps include: Step S1) obtaining radar data of the detected road section and preprocessing the radar data; Step S2) constructing a dataset of internal road surface defects and classifying them based on the types of defects to form a dataset classified by defect type; Step S3) The obtained data sets are fed into different deep learning network models, and the manual labeling results of various diseases in the data sets are analyzed in turn to form a disease discrimination weight model; Step S4) obtaining the edge amplitude of the marked frame position in the data set based on the Canny edge algorithm, calculating the edge amplitude of each pixel point in the marked frame, and determining the edge amplitude threshold of each type of disease; Step S5) pre-processing the radar data to be identified, respectively entering them into at least two weighted discrimination models for discrimination, and filtering the road surface internal disease discrimination results based on the model discrimination results and disease filtering rules; The defects in step S2) include three types: poor interlayer bonding defects, loose and broken defects within the structure, and crack defects. Among them, poor interlayer bonding defects and loose and broken defects within the structure are judged using radar profiles, and crack defects are judged using radar slices. When the disease dataset was established, the ratio of various disease data in each dataset was 1:1:1, and the characteristics and marking rules of the same disease were the same; The deep learning network model in step S3) includes a YOLO network model, a Faster-RCNN artificial neural network model and an SSD artificial neural network model; The CSP-based Backbone network in the YOLO network model is replaced by the EfficientViT-M0 module; An attention mechanism is added to the Faster-RCNN artificial neural network model; The loss function in the SSD artificial neural network model selects the EIOU function.

2. The method for identifying internal road surface defects according to claim 1, characterized in that: The pre-processing in step S1) includes at least one of zero bias removal, zero point adjustment, and data gain, and the gain coefficient of the waveform is adjusted to enhance the characteristics of crack damage inside the pavement.

3. The method for identifying internal road surface defects according to claim 2, characterized in that: When the pretreatment adopts zero point adjustment, it includes: Select a bridge slab or repair section location on the road section to be inspected; Count the number of points at each channel of the bridge slab or repair section of the inspected road section; The zero point adjustment of each channel is achieved by subtracting a preset fixed value from the number of points of each channel in turn; the fixed value is selected based on empirical values.

4. The method for identifying internal road surface defects according to claim 1, characterized in that: During the data set construction process of step S2), a grayscale spectrum is uniformly adopted. When marking defects on the slices of radar data, the marking frame must cover all defect features.

5. The method for identifying road surface internal disease images according to claim 1, characterized in that: The determination in step S5) is made by combining the discrimination results under each weight model and the edge amplitude.

6. The method for identifying internal road surface defects according to claim 5, characterized in that: The edge amplitude includes a first edge amplitude boundary and a second edge amplitude boundary; the value of the first edge amplitude boundary is greater than the value of the second edge amplitude boundary; Get the data of each weight model at the same position on the detection section: If there are at least two weight model data with the same disease type and the edge amplitude exceeding the first edge amplitude boundary, then the disease at that location on the detected road section is considered to be of that type; If there is no data from at least two weight models, and the damage edge amplitude does not exceed the first edge amplitude boundary, then the damage at that location on the detected road section is considered a misjudgment; If there are at least two weight model data with different damage types but both exceeding the first edge amplitude boundary, it is determined that there is a damage at that location on the detected road section, but the type requires manual assistance to determine; If there are data from at least two weighted models, and the damage edge amplitudes do not exceed the second edge amplitude boundary, regardless of whether the results of the models are the same, the damage at that location on the detected road section is considered a misjudgment; If there are data from at least two weighted models, and the damage edge amplitudes of both models only exceed the second edge amplitude boundary, regardless of whether the results of the models are the same, it is determined that there is a damage at that location on the detected road section and the type of damage requires manual assistance. If there is no data for at least two weight models, and the damage edge amplitude exceeds the first edge amplitude boundary, it is determined that whether there is a damage at that location on the detected road section and the type require manual assistance to determine.

Images