A method and system for intelligently identifying road surface defects

Through the method of combining the target detection model and the precise positioning of small frames, the problem of slow and inaccurate detection of apparent diseases on roads in the prior art is solved, automated identification and quantitative calculation are realized, and detection efficiency and accuracy are improved.

CN114529522BActive Publication Date: 2025-03-21SOUTHEAST UNIV +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210101302.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-27
Publication Date
2025-03-21
Estimated Expiration
2042-01-27

AI Technical Summary

Technical Problem

In the prior art, road apparent disease detection relies on manual visual inspection, which is slow, dangerous and inaccurate. Even after using vehicle-mounted collection equipment, there is still a problem of time-consuming, labor-intensive and low accuracy.

Method used

An intelligent identification method for apparent diseases on the road is proposed, which uses the target detection model to perform rough positioning of diseases, combines small frames to accurately locate them, and removes isolated boxes and the same items through post-processing steps to achieve automated disease identification.

Benefits of technology

It realizes automated disease identification, reduces detection false alarm rate, improves detection efficiency, and can quantitatively calculate the length or area of ​​apparent road diseases.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114529522B_ABST
    Figure CN114529522B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for intelligently identifying road surface defects. The method detects defects on an input road surface image based on a target detection model, and returns the defect area position coordinate information, defect type, and defect confidence of the road defect. The defect area is accurately located using a small frame of a fixed size, which is convenient for calculating the length or area of ​​a specific defect. The result of the accurate location is post-processed, including removing isolated frames and identical items, to obtain the final recognition result. The present invention can automatically identify defects, reduce the false alarm rate of detection, and improve detection efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and intelligent detection, and in particular relates to intelligent recognition technology for road surface defects. Background Art

[0002] Asphalt pavement is the most commonly used type of road construction in China. With the increase in highway mileage and the extension of service time, the inspection and maintenance of highways have become more and more arduous, and road surface disease detection is an important part of road inspection and maintenance.

[0003] The original road surface disease detection is mainly based on manual visual inspection, which is slow, dangerous, affects traffic and is inaccurate, especially on highways, where the risk factor is higher. This method is gradually being eliminated. With the continuous advancement of science and technology, high-speed and high-precision vehicle-mounted acquisition equipment is widely used. The equipment is used to patrol the road and collect road surface data. Professionals screen the image data one by one, mark the disease area, and then calculate the disease length or area uniformly, so as to calculate the damage rate. Although the existence of road inspection vehicles avoids the problems of slow speed and danger in manual inspection, it still has the disadvantages of time-consuming, labor-intensive and low accuracy through manual screening and annotation of huge image databases. Therefore, visual-based disease detection technology came into being. However, the complexity and diversity of road images and the weak signal of disease information make it difficult to identify road disease algorithms based on vision, and quantitative calculation of weak signal disease areas is required, which is always a difficulty. Summary of the invention

[0004] In order to solve the technical problems mentioned in the above background technology, the present invention proposes a method and system for intelligently identifying road surface defects, which automates defect identification, reduces the false alarm rate of detection, and improves detection efficiency.

[0005] In order to achieve the above technical objectives, the technical solution of the present invention is:

[0006] A method for intelligently identifying road surface defects comprises the following steps:

[0007] (1) Rough positioning of road surface defects: Defect detection is performed on the input road surface image based on the target detection model, and the coordinate information of the defect area, defect type, and confidence level of the road defect are returned;

[0008] (2) Accurate location of road surface defects: Use small grids of fixed size to accurately locate the defect area, making it easier to calculate the length or area of ​​a specific defect;

[0009] (3) Post-processing of positioning results: Post-process the precise positioning results, including removing isolated boxes and removing identical items, to obtain the final recognition results.

[0010] Furthermore, the specific process of step (1) is as follows:

[0011] (1a) Obtain images collected by a multifunctional road condition detection vehicle, filter the data and annotate the images according to the disease category, construct a road surface disease annotation dataset, and expand some disease-free images as background images as the total dataset, which is divided into a training set and a validation set;

[0012] (1b) constructing a road surface disease detection model based on the target detection method, using the training set to train the road surface disease detection model, using the validation set for the trained road surface disease detection model, and saving the road surface disease detection model with the best performance on the validation set as the disease rough positioning model;

[0013] (1c) The road surface image to be detected is passed into the defect coarse positioning model to obtain the defect detection box detected_Box:

[0014] detected_Box=[Box,confidence,class]

[0015] Among them, Box = [x1, y1, x2, y2], (x1, y1), (x2, y2) represent the coordinates of the upper left vertex and the lower right vertex of the disease detection box respectively, confidence represents the probability value of the existence of the disease, and class represents the specific disease category.

[0016] Furthermore, in step (1a), the damage categories include but are not limited to cracks, fissures, potholes and repairs; in step (1b), the road surface damage detection model adopts the YOLO series model architecture.

[0017] Furthermore, the specific process of step (2) is as follows:

[0018] (2a) Small frame division:

[0019] Divide the Box area into a series of small frame images, denoted as R_Images:

[0020] RW=T W ·W

[0021] RH=T H ·H

[0022] Among them, RW and RH are the length and width of the small frame, W and H are the length and width of the road detection image,

[0023] (2b) Through the precise positioning algorithm, the frames that do not contain the diseased area in R_Images are removed to obtain a set of valid small frames R_Boxes, where each small frame R_Box contains the following information:

[0024] R_Box=[u1,v1,u2,v2,class]

[0025] Among them, (u1, v1) and (u2, v2) are the coordinates of the upper left vertex and the lower right vertex of the small frame.

[0026] Furthermore, in step (2b), the precise positioning algorithm includes adopting a feature-map calculation strategy, and the specific process is as follows:

[0027] (2b101) Based on the coordinate information and category information of detected_Box, a specific area is cropped from the original image as a dataset for building a bBox classification model, and the dataset is divided into a training set and a validation set;

[0028] (2b102) Construct a bBox classification model based on a convolutional neural network, train the bBox classification model using the training set, and select the model parameters with the highest accuracy on the validation set as the final bBox classification model parameters;

[0029] (2b103) Based on the coordinate information of detected_Box, a specific area is cut from the original image and sent to the bBox classification model to calculate the thermal map Feature-map of the area;

[0030] (2b104) For the transverse and longitudinal defects on the road, find the box with the highest heat map in the vertical direction as the valid box, otherwise it is an invalid box. Keep the valid boxes and filter the invalid boxes to get R_Boxes.

[0031] Furthermore, in step (2b102), the convolutional neural network adopts a Resnet series model; in step (2b103), a CAM method or a Grad-CAM method is used to calculate a thermal map Feature-map.

[0032] Furthermore, the specific process of step (2b104) is as follows:

[0033] ① Determine the shape of the disease based on the Box in detected_Box: horizontal or vertical;

[0034] ② Group the R_Images vertically according to the disease morphology. For horizontal diseases, distribute all the Boxes vertically, that is, R_Boxes in the same vertical direction are grouped together; for vertical diseases, R_Boxes in the same horizontal direction are grouped together;

[0035] ③According to the Feature-map obtained by the bBox classification model, the thermal value statistics of the R_Box in the same group are performed, and the frame with the largest thermal value in the group is taken as the valid frame, and the category of the detected_Box is taken as the disease category of the valid frame. The coordinate data and disease category of the valid frame are retained, and the others are regarded as invalid frames;

[0036] ④ Repeat step ③ until all groups are calculated and R_Boxes is obtained.

[0037] Furthermore, in step (2b), the precise positioning algorithm includes adopting a small grid classification strategy, and the specific process is as follows:

[0038] (2b201) Based on R_Images, use its coordinate information to cut from the original image to obtain the small frame image of the corresponding area, classify it according to the corresponding category, and use it as the data set for building the small frame classification model, and divide it into a training set and a validation set;

[0039] (2b202) Construct a small frame classification model based on a convolutional neural network, use the training set to train the small frame classification model, and use the weight model with the highest accuracy on the validation set as the final small frame classification model RC_model;

[0040] (2b203) For R_Images, the corresponding image area is cropped from the original image according to its coordinate information and sent to RC_model. If the classification result of the model is inconsistent with the category of the Box, it is regarded as an invalid frame. All invalid frames are deleted and valid frames are retained to obtain R_Boxes.

[0041] Furthermore, in step (3), the method for removing the isolated frame is as follows:

[0042] If there is no box in the 8-field of a box, the box is regarded as an isolated box and deleted; the method of removing the same items is as follows:

[0043] First, delete the boxes with the same coordinates and the same disease category to ensure that no small boxes with the same information appear;

[0044] The category information of small frames with the same coordinates is counted. According to the priority of the category, the small frames with high priority are retained and the small frames with low priority are deleted to ensure that the number of small frames at the same location is unique and the disease type is unique.

[0045] A road surface disease intelligent identification system, characterized by comprising:

[0046] The defect coarse positioning module is used to detect defects on the input road surface image based on the target detection model, and return the defect area location coordinate information, defect type and confidence level of the road defect;

[0047] The disease precise positioning module is used to accurately locate the disease area using small frames of fixed size, which is convenient for calculating the length or area of ​​specific diseases;

[0048] The subdivision frame post-processing module is used to post-process the results of precise positioning, including removing isolated frames and removing identical items to obtain the final recognition results.

[0049] The beneficial effects brought by adopting the above technical solution are:

[0050] 1. The present invention completes the recognition and location of road surface defects based on deep learning (target detection algorithm and classification algorithm), and can quantitatively calculate the length of road surface defects, such as cracks and repair strips. In the quantitative calculation that meets the same accuracy, it is a technical solution that saves the most recognition time and the most annotation cost;

[0051] 2. The present invention innovatively adopts a combination strategy of a disease detection model and a feature-map, effectively displaying the location of cracks or repair strips in a high-thermal manner, and then uses a preset small frame to locate and frame them. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of the method of the present invention;

[0053] Figure 2 This is the inference result diagram of the apparent disease detection model in the embodiment. Hcrack and 0.71 in the upper left corner represent the class and confidence in detected_Box respectively;

[0054] Figure 3 The small frame shows the result of the crack recognition algorithm in the embodiment;

[0055] Figure 4 This is a diagram showing the high thermal area of ​​the crack in the embodiment. DETAILED DESCRIPTION

[0056] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0057] The present invention designs a method for intelligently identifying road surface defects, which mainly includes the following modules:

[0058] A: Disease coarse positioning module

[0059] B: Disease positioning module

[0060] C: Segmentation box post-processing module

[0061] The specific technical processes of the above modules are as follows: Figure 1 The specific technical solution includes the following steps:

[0062] Module A aims to achieve rough positioning of road surface defects based on the target detection model. This module detects defects on the input road surface image based on the target detection model and returns the defect area location coordinate information, defect type, and defect confidence of the road defect.

[0063] The module steps are as follows:

[0064] (A11) Obtain images collected by a multifunctional road condition detection vehicle, filter the data and annotate the images according to the disease categories to which they belong, construct a road surface disease annotated dataset, and expand some disease-free images (the number is 1%-5% of the total annotated dataset) as background images, which are used as the total dataset, and the total dataset is divided into a training set and a validation set.

[0065] The road damage includes various types of damage, including but not limited to cracks, fissures, potholes, and repairs.

[0066] (A12) Construct a road surface damage detection model based on target detection method.

[0067] Perform data enhancement on the model input data, including but not limited to image color adjustment, image cropping, image flipping, image rotation, image scaling, and image translation; perform image mosaic operations on the model data, that is, randomly splicing 4 images into one image for calculation, which can greatly increase data diversity and is conducive to learning small targets.

[0068] A disease detection model is built based on the yolov5x model. The detection model is trained using the training set. The validation set is used for the trained detection model. The detection model with the highest mAP on the validation set is saved as the disease coarse positioning model.

[0069] The mAP (mean Average Precision) refers to the average value of AP of each category; wherein AP refers to the area under the PR curve, which is generally used as an evaluation indicator for target detection tasks.

[0070] (A13) The surface image of the road to be detected is passed into the rough defect positioning model to obtain the rough positioning result of the defect detected_Box.

[0071] detected_Box=[Box,confidence,class]

[0072] Box = [x1, y1, x2, y2], (x1, y1), (x2, y2) represent the coordinates of the upper left vertex and the lower right vertex of the disease area, confidence represents the probability value of the disease, and class represents the specific disease category. The visualization effect of the detection box is as follows Figure 2 shown.

[0073] Module B aims to accurately locate the damaged area, using small grids of fixed size to accurately locate the damaged area (such as along the direction of cracks or strip repairs) so as to calculate the length or area of ​​specific damage. Figure 3 shown.

[0074] The module steps include:

[0075] Step B1: Divide the Box area into a series of small frames with length and width [RW, RH], denoted as R_Images.

[0076] RW=T W ·W

[0077] RH=T H ·H

[0078] Among them, RW and RH are the length and width of the small frame, W and H are the length and width of the road detection image, In this embodiment, RW is RH is

[0079] This step specifically includes the following methods:

[0080] Method a: Divide the area Box into small frames based on the step size [RW, RH]. When the multiple is not divisible by the length and width of the coordinate area, use the paading method for the length and width of the Box to obtain R_Images.

[0081] Method b: Divide the entire image into small boxes based on the step size [RW, RH]. When the multiple is not divisible by the length and width of the coordinate area, padding is used for the length and width of the coordinate area. Then the small boxes that fall within the Box area are retained to form R_Images.

[0082] The padding means that when dividing small grids, when the multiple is not divisible by the length and width of the coordinate area, the length and width of the coordinate area are padded with 0. For example, if the image width is 490 and RW is 20, the image is padded with 0 to 500 along the width direction, so that the entire image can be divided into 25 grids in width.

[0083] Step B2: Through the precise positioning algorithm, remove the frames that do not contain the diseased area in R_Images, and the obtained valid small frame set R_Boxes is the disease recognition result of this module, where each small frame R_Box contains the following information:

[0084] R_Box=[u1,v1,u2,v2,class]

[0085] Among them, u1, v1, u2, v2 are the coordinates of the upper left vertex and the lower right vertex of the small frame, and class represents the specific category of the disease in the small frame.

[0086] The algorithm integrates two strategies, namely Feature-map calculation strategy and small frame classification strategy.

[0087] B21: Feature-map calculation strategy. Based on the result R_Images of Box-divided small boxes, we can obtain the set of valid small boxes: R_Boxes by building a bBox classification model and removing invalid small boxes. The details are as follows:

[0088] (B2101) Based on the coordinate information and category information of detected_Box, a specific area is cropped from the original image as a dataset for building a bBox classification model. The dataset is divided into a training set and a validation set.

[0089] (B2102) A bBox classification model is constructed based on a convolutional neural network. The present invention uses a Resnet series model to train the input data set and selects the model parameters with the highest accuracy on the validation set as the final bBox classification model parameters.

[0090] (B2103) In the recognition stage, the specific area is cut out from the original image based on the coordinate information of detected_Box and sent to the bBox classification model to calculate the thermal map feature map of the area. The high thermal area of ​​the crack is displayed as follows Figure 4 shown.

[0091] The calculation of the heat map refers to the CAM method, extracting the output matrix and corresponding weights of the middle layer of the model, so as to calculate the heat map Feature-map of the area. Furthermore, the calculation of the heat map can also use other equivalent methods for calculating Feature-map, such as the Grad-CAM method. The CAM method and the Grad-CAM method are both public general algorithms and will not be described here.

[0092] (B2104) Eliminate invalid small frames. For the horizontal and vertical defects on the road (both horizontal and vertical directions are based on the relative position in the image), find the frame with the highest heat map in the vertical direction as the valid frame, otherwise it is an invalid frame. Keep the valid frames and filter the invalid frames. The specific implementation steps are as follows:

[0093] ① Determine the shape of the disease (horizontal or vertical) according to the Box in detected_Box.

[0094] ② The result R_Images of dividing the Box into small frames is grouped vertically according to the disease morphology. For example, for horizontal diseases, all the Boxes are distributed vertically, that is, the R_Boxes in the same vertical direction are grouped together; conversely, for vertical diseases, those in the same horizontal direction are grouped together.

[0095] ③ Based on the Feature-map obtained by the bBox classification model, the thermal value statistics of the R_Box in the same group are performed, and the frame with the largest thermal value in the group is taken as the valid frame, and the category of the detected_Box is taken as the disease category of the valid frame. The coordinate data and disease category of the valid frame are retained, and the others are regarded as invalid frames.

[0096] ④ Repeat step ③ until all groups are calculated, and a series of valid boxes R_Boxes are obtained.

[0097] B22: Small frame classification strategy. Based on the result R_Images of Box dividing small frames, we can obtain the set R_Boxes of valid small frames by building a small frame classification model and eliminating invalid small frames. The specific implementation process is as follows:

[0098] (B2201) Based on the result R_Images of dividing the Box into small frames, use its coordinate information to cut it from the original image to obtain the small frame images of the corresponding area, classify them according to the corresponding categories, and use them as the data set for building the small frame classification model, and divide them into training set and validation set.

[0099] (B2202) A small frame classification model is constructed based on a convolutional neural network. The present invention adopts a Resnet series model structure and uses a weight model with the highest accuracy on a validation set as the final small frame classification prediction model RC_model.

[0100] (B2203) In the recognition stage, for the result R_Images of the Box small frame division, the corresponding image area is cropped from the original image according to its coordinate information and sent to RC_model. If the classification result of the model is inconsistent with the category of the Box, it is regarded as an invalid frame. All invalid frames are deleted and valid frames are retained to obtain the final R_Boxes.

[0101] Furthermore, both strategies can be used separately as small frame positioning algorithms. The two algorithms can be reasonably combined according to the data conditions of the specific embodiment, and one or a combination of the two algorithms can be used.

[0102] According to the damage situation of this embodiment, the experiment found that the B21 Feature-map strategy performs better when it comes to single damage such as a single crack or a single repair, and the preset small grid can be used to accurately cover the location of the damage. When the crack and the repair location are close or even overlap, the B22 small grid classification strategy is more objective. Therefore, Figure 1 The specific steps of classify are as follows:

[0103] 1. For the detected_Box obtained by the coarse positioning model, traverse all the Boxes and the corresponding defect categories in the same image to determine whether both repair and crack defects exist.

[0104] 2. If both repair and crack defects exist, then:

[0105] (1) Calculate ratio_IOU for the two boxes Box1 and Box2:

[0106]

[0107] Among them, inter_area represents the area of ​​the intersection of the two boxes; Box1_area represents the area of ​​Box1; Box2_area represents the area of ​​Box2;

[0108] (2) When ratio_IOU is greater than the threshold ratio_IOU, the classify condition is considered to be met, and the small box classification strategy is used for Box1 and Box2, otherwise the Feature-map calculation strategy is used.

[0109] 3. If both repair and crack defects do not exist at the same time, the Feature-map calculation strategy is used.

[0110] Module C: Small box post-processing module. Post-process the output result R_Boxes of module B, including removing isolated boxes and removing identical items, to obtain the final small box result R_Result. The specific implementation steps are as follows:

[0111] C1: Remove isolated boxes. If there are isolated small boxes in the obtained small box list, it is likely to be a false positive of the target detection model or noise caused by the small box algorithm. In order to make the result smoother, the isolated small boxes are deleted.

[0112] Furthermore, the criterion for determining isolation is that if there is no frame in the 8-region of the frame, the frame is regarded as an isolated frame and deleted.

[0113] C2: Remove identical items. Considering the specific implementation scenario, when multiple detected_Boxes are nested, it is difficult to predict whether they are for the same disease or different diseases. The above algorithms are all based on detected_Box, so there will be two or more small boxes at the same coordinate position, and the disease type may not be unique. In this case, R_Boxes can be further post-processed based on the coordinate information of the small boxes and their disease types, so that the number of small boxes under the same coordinate is unique and the disease type is unique. The specific implementation steps are as follows:

[0114] C21, first delete the boxes with the same coordinates and the same disease category to ensure that no small boxes with the same information appear;

[0115] C22, count the category information of small frames with the same coordinates, retain the high priority and delete the low priority small frames according to the category priority, to ensure that the number of small frames in the same place is unique and the disease type is unique.

[0116] The embodiments are only for illustrating the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for intelligently identifying road surface defects, characterized in that: The following steps are involved: (1) Rough positioning of road surface defects: Defect detection is performed on the input road surface image based on the target detection model, and the defect area location coordinate information, defect type, and defect confidence level of the road defect are returned; (2) Accurate location of road surface defects: Use small grids of fixed size to accurately locate the defect area, making it easier to calculate the length or area of ​​a specific defect; (3) Post-processing of positioning results: Post-process the precise positioning results, including removing isolated boxes and removing identical items, to obtain the final recognition results; The specific process of step (1) is as follows: (1a) Obtain images collected by a multifunctional road condition detection vehicle, filter the data and annotate the images according to the disease category, construct a road surface disease annotation dataset, and expand some disease-free images as background images as the total dataset, which is divided into a training set and a validation set; (1b) constructing a road surface disease detection model based on the target detection method, using the training set to train the road surface disease detection model, using the validation set for the trained road surface disease detection model, and saving the road surface disease detection model with the best performance on the validation set as the disease rough positioning model; (1c) The road surface image to be detected is passed into the defect coarse positioning model to obtain the defect detection box detected_Box: detected_Box=[Box,confidence,class] Among them, Box = [x1, y1, x2, y2], (x1, y1), (x2, y2) represent the coordinates of the upper left vertex and the lower right vertex of the disease detection box respectively, confidence represents the probability value of the existence of the disease, and class represents the specific disease category; The specific process of step (2) is as follows: (2a) Small frame division: Divide the Box area into a series of small frame images, denoted as R_Images: RW=T W ·W HR=T H ·H Among them, RW and RH are the length and width of the small frame, W and H are the length and width of the road detection image, (2b) Through the precise positioning algorithm, the frames that do not contain the diseased area in R_Images are removed to obtain a set of valid small frames R_Boxes, where each small frame R_Box contains the following information: R_Box=[u1,v1,u2,v2,class] Among them, (u1, v1) and (u2, v2) are the coordinates of the upper left vertex and the lower right vertex of the small frame; In step (2b), the precise positioning algorithm includes adopting a feature-map calculation strategy, and the specific process is as follows: (2b101) Based on the coordinate information and category information of detected_Box, a specific area is cropped from the original image as a dataset for building a bBox classification model, and the dataset is divided into a training set and a validation set; (2b102) Construct a bBox classification model based on a convolutional neural network, train the bBox classification model using the training set, and select the model parameters with the highest accuracy on the validation set as the final bBox classification model parameters; (2b103) Based on the coordinate information of detected_Box, a specific area is cut from the original image and sent to the bBox classification model to calculate the thermal map Feature-map of the area; (2b104) For the transverse and longitudinal defects on the road, find the box with the highest heat map in the vertical direction as the valid box, otherwise it is an invalid box. Keep the valid boxes and filter the invalid boxes to get R_Boxes.

2. The method for intelligently identifying road surface defects according to claim 1, characterized in that: In step (1a), the damage categories include but are not limited to cracks, fissures, potholes and repairs; in step (1b), the road surface damage detection model adopts the YOLO series model architecture.

3. The method for intelligently identifying road surface defects according to claim 1, characterized in that: In step (2b102), the convolutional neural network adopts the Resnet series model; in step (2b103), the CAM method or the Grad-CAM method is used to calculate the thermal map Feature-map.

4. The method for intelligently identifying road surface defects according to claim 1, characterized in that: The specific process of step (2b104) is as follows: ① Determine the shape of the disease based on the Box in detected_Box: horizontal or vertical; ② Group the R_Images vertically according to the disease morphology. For horizontal diseases, distribute all the Boxes vertically, that is, R_Boxes in the same vertical direction are grouped together; for vertical diseases, R_Boxes in the same horizontal direction are grouped together; ③According to the Feature-map obtained by the bBox classification model, the thermal value statistics of the R_Box in the same group are performed, and the frame with the largest thermal value in the group is taken as the valid frame, and the category of the detected_Box is taken as the disease category of the valid frame. The coordinate data and disease category of the valid frame are retained, and the others are regarded as invalid frames; ④ Repeat step ③ until all groups are calculated and R_Boxes is obtained.

5. The method for intelligently identifying road surface defects according to claim 1, characterized in that: In step (2b), the precise positioning algorithm includes adopting a small grid classification strategy, and the specific process is as follows: (2b201) Based on R_Images, use its coordinate information to cut from the original image to obtain the small frame image of the corresponding area, classify it according to the corresponding category, and use it as the data set for building the small frame classification model, and divide it into a training set and a validation set; (2b202) Construct a small frame classification model based on a convolutional neural network, use the training set to train the small frame classification model, and use the weight model with the highest accuracy on the validation set as the final small frame classification model RC_model; (2b203) For R_Images, the corresponding image area is cropped from the original image according to its coordinate information and sent to RC_model. If the classification result of the model is inconsistent with the category of the Box, it is regarded as an invalid frame. All invalid frames are deleted and valid frames are retained to obtain R_Boxes.

6. The method for intelligently identifying road surface defects according to claim 1, characterized in that: In step (3), the method for removing isolated boxes is as follows: If there is no box in the 8-field of a box, the box is regarded as an isolated box and deleted; the method of removing the same items is as follows: First, delete the boxes with the same coordinates and the same disease category to ensure that no small boxes with the same information appear; The category information of small frames with the same coordinates is counted. According to the priority of the category, the small frames with high priority are retained and the small frames with low priority are deleted to ensure that the number of small frames at the same location is unique and the disease type is unique.

7. An intelligent recognition system for implementing the method for intelligently identifying road surface defects according to claim 1, characterized in that: include: The defect coarse positioning module is used to detect defects on the input road surface image based on the target detection model, and return the defect area location coordinate information, defect type and confidence level of the road defect; The disease precise positioning module is used to accurately locate the disease area using small frames of fixed size, which is convenient for calculating the length or area of ​​specific diseases; The subdivision frame post-processing module is used to post-process the results of precise positioning, including removing isolated frames and removing identical items to obtain the final recognition results.

Citation Information

Patent Citations

  • Road disease detection method based on deep learning

    CN111080620A

  • Road surface disease target detection and instance segmentation method based on road foreground image

    CN111553236A