Highway anti-dazzle plate health state automatic detection method

Abstract

The present application provides a highway anti-glare plate health state automatic detection method, comprising the following steps: step S1: collecting anti-glare plate images by using a binocular stereo road intelligent inspection system; step S2: training an anti-glare plate detection model based on YOLOv5; step S3: identifying the anti-glare plate images based on the detection model trained in step S2 to obtain coordinate information data of the anti-glare plate detection frame in the anti-glare plate images; step S4: according to prior knowledge, classifying the anti-glare plates in the detection frame into four categories: missing anti-glare plates, damaged anti-glare plates, inclined anti-glare plates and intact anti-glare plates; step S5: based on the pre-classification result of step S4, using the coordinate information data in step S3, calculating the height value of the anti-glare plate prediction frame, the front and back distance between the anti-glare plate prediction frames and the length-width ratio information of the anti-glare plate prediction frame, and determining the health state of the anti-glare plate according to the calculation result; the present application can automatically detect the state of the highway anti-glare plate based on the damage of the anti-glare plate position and geometric information.

Description

TECHNICAL FIELD

[0001] The present application relates to the image recognition technology field of road inspection, in particular to a highway anti-glare plate health state automatic detection method. BACKGROUND

[0002] At present, in the daily road inspection work, the anti-glare plate damage detection is mainly based on manual detection. However, due to the long mileage of the highway and the fast driving speed, this method has the problems of slow detection speed, low detection accuracy and low detection efficiency.

[0003] In the related research field, the research on anti-glare plates at home and abroad is basically concentrated in the aspects of anti-glare plate size design, anti-glare plate installation optimization design and anti-glare plate performance detection, and the research on anti-glare plate damage detection is less. In recent years, the deep learning technology has developed rapidly, from the initial two-stage model R-CNN, Faster-RCNN to the current single-stage model CenterNet, YOLO series, which has been widely used in target detection field. However, the detection accuracy of deep learning is greatly dependent on the number and quality of samples, and due to the lack of damaged anti-glare plate data and the various forms of damage, it is difficult to achieve high-precision anti-glare plate health state detection only by using the method of deep learning.

[0004] In view of the above problems, the present application provides an automatic damage detection method based on anti-glare plate position and geometric information. SUMMARY

[0005] The present application provides a highway anti-glare plate health state automatic detection method, which can automatically detect the state of the highway anti-glare plate based on the damage of the anti-glare plate position and geometric information.

[0006] The present application adopts the following technical scheme.

[0007] The highway anti-glare plate health state automatic detection method comprises the following steps:

[0008] Step S1: collecting anti-glare plate images by using a binocular stereo road intelligent inspection system;

[0009] Step S2: training an anti-glare plate detection model based on YOLOv5;

[0010] Step S3: identifying the anti-glare plate images based on the detection model trained in step S2 to obtain coordinate information data of the anti-glare plate detection frame in the anti-glare plate images;

[0011] Step S4: classifying the anti-glare plates in the detection frame into four categories of missing anti-glare plates, damaged anti-glare plates, inclined anti-glare plates and intact anti-glare plates according to prior knowledge;

[0012] Step S5: Based on the pre-classification result of step S4, the height value of the anti-glare plate prediction frame, the front and rear spacing between the anti-glare plate prediction frames, and the aspect ratio information of the anti-glare plate prediction frame are calculated using the coordinate information data in step S3, and the health status of the anti-glare plate body or the anti-glare plate distribution is determined based on the calculation results.

[0013] In step S3, the coordinate information of the anti-glare plate detection frame is obtained by using the anti-glare plate recognition model obtained by training to recognize the anti-glare plate in the anti-glare plate image, so as to extract the detection area of the anti-glare plate in the anti-glare plate image, that is, the detection frame corresponding to the anti-glare plate.

[0014] In step S4, the classification is based on the absence of anti-glare plates, the presence of broken anti-glare plates, and the presence of skewed anti-glare plates.

[0015] The prior knowledge in step S4 is the coordinate information characteristics of the health status and non-health status of the anti-glare plate, specifically,

[0016] Feature A: The spacing between the detection frames corresponding to the anti-glare plates gradually decreases in the anti-glare plate image, and the spacing between the detection frame corresponding to the first anti-glare plate and the detection frame corresponding to the second anti-glare plate is obviously greater than the spacing between the detection frames corresponding to the anti-glare plates in the rear of the anti-glare plate image.

[0017] Feature B: When the anti-glare plate is damaged, the height value of the detection frame corresponding to the anti-glare plate decreases in the anti-glare plate image, and the height of the detection frame corresponding to each anti-glare plate in the health state gradually decreases in the anti-glare plate image, and the height difference between the detection frame corresponding to the first anti-glare plate and the detection frame corresponding to the second anti-glare plate is obviously greater than the height difference between the detection frames corresponding to the anti-glare plates in the rear of the anti-glare plate image.

[0018] Feature C: The aspect ratio of the detection frame corresponding to the skewed anti-glare plate is significantly different from the aspect ratio of the detection frame corresponding to other types of anti-glare plates.

[0019] Step S5 specifically includes the following steps:

[0020] Step S51: Determine whether there is a missing anti-glare plate phenomenon according to feature A, specifically: judge whether there is a missing anti-glare plate between the first and second anti-glare plates, and the formula is

[0021] X 2(min) -X 1(max) > 2 * (X 3(min) -X 2(max) ) Formula One;

[0022] Determine whether there is a missing anti-glare plate between the nth and (n-1)th anti-glare plates, and the formula is

[0023] X n(min) -X n-1(max) >K*(X n-1(min) -X n-2(max) ) Equation Two;

[0024] wherein X n-1(max) represents the maximum horizontal coordinate of the (n-1)th anti-glare panel detection frame; X n-1(min) represents the minimum horizontal coordinate of the (n-1)th anti-glare panel detection frame, and K is a to-be-determined coefficient related to the detection effect of the present step;

[0025] Step S52: determining whether there is an anti-glare panel damage phenomenon according to feature B, and the specific method is: judging whether the first anti-glare panel is damaged according to the extracted anti-glare panel detection frame coordinate information, and the formula is

[0026] Y 1(max) -Y 1(min) <0.7*(Y 2(max) -Y 2(min) ) Equation Three;

[0027] judging whether the nth anti-glare panel is damaged, and the formula is

[0028] Y n(max) -Y n(min) <K1*(Y n-1(max) -Y n-1(min) ) Equation Four;

[0029] wherein Y n(max) represents the maximum vertical coordinate of the nth anti-glare panel detection frame; Y n(min) represents the minimum vertical coordinate of the nth anti-glare panel detection frame, and K1 is a to-be-determined coefficient related to the detection effect of the present step;

[0030] Step S53: determining whether there is an anti-glare panel tilt phenomenon according to feature C, and the formula is:

[0031] Y n(max) -Y n(min) *K2<(X n(max) -X n(min) ) Equation Five;

[0032] wherein K2 is a to-be-determined coefficient related to the detection effect of the present step;

[0033] Step S54: the anti-glare panel that does not satisfy the determination conditions of steps S51, S52 and S53 is determined to be in a healthy state of intact anti-glare panel.

[0034] In step S51, the detection effect is best when K is 1.1; in step S52, the detection effect is best when K1 is 0.6; in step S53, the detection effect is best when K2 is 0.42.

[0035] The binocular stereo road intelligent inspection system comprises a binocular road intelligent perception system, the binocular road intelligent perception system comprising a binocular camera, a GPS, an inertial measurement unit (IMU) and an artificial intelligence development board; the binocular road intelligent perception system is installed on the roof or window of a vehicle through a strong suction cup, and positioning information is obtained through the GPS and the IMU, and data is analyzed to trigger the binocular camera at a fixed distance to realize image acquisition.

[0036] In step S1, the binocular road intelligent perception system is used to collect relevant images of expressways and urban roads, and after preliminary data screening to remove invalid image data, a data set is formed, data samples are manually labeled through LabelImg data labeling software, anti-dazzle plate label information is generated, and an anti-dazzle plate recognition database for training an anti-dazzle plate detection model based on YOLOv5 is constructed.

[0037] In step S2, the YOLOv5 model is composed of four parts, namely, an input end, a Backbone, a Neck and a Prediction;

[0038] The input end is used to input original picture data, and a Mosaic data enhancement method is used to splice four randomly selected pictures through random scaling and random cropping operations to enrich the data set and improve the generalization of the anti-dazzle plate detection model;

[0039] The Backbone layer is composed of a Focus structure and a CSP structure, and is a convolutional neural network for extracting image features;

[0040] The Neck layer uses a path aggregation network (PANet) as an architecture to realize feature fusion;

[0041] The Prediction layer is used to output the regression of categories and bounding boxes;

[0042] When training the anti-dazzle plate detection model, first, the anti-dazzle plate recognition database is divided into a training set and a validation set in a ratio of 5:1; then the training set is imported into the YOLOv5 network for model training, the validation set is used to realize output verification of the model, the number of training rounds is set to 300, the batch-size is set to 16, and the training picture size is set to 640.

[0043] The application combines a binocular road intelligent perception system and proposes a method for efficiently and accurately detecting defective anti-dazzle plates. The method can automatically identify defective anti-dazzle plates and replace manual inspection of anti-dazzle plates. Moreover, the present example fully considers the influence of complex actual road environments on the robustness of the model and establishes anti-dazzle plate training databases under different complex road scenes. The diversity of the databases makes the generalization ability of the anti-dazzle plate model stronger. BRIEF DESCRIPTION OF DRAWINGS

[0044] The application will be further described in detail below in combination with the drawings and specific embodiments:

[0045] Attached Figure 1 is a flowchart of the application. DETAILED DESCRIPTION

[0046] The application will be further described in detail below in combination with the drawings and specific embodiments:

[0047] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise indicated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0048] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, they refer to the presence of a feature, step, operation, device, component, and / or combination thereof.

[0049] As shown in the figure, the highway anti-dazzle plate health state automatic detection method comprises the following steps:

[0050] Step S1: Collecting anti-dazzle plate images using a binocular stereo road intelligent inspection system;

[0051] Step S2: Training a YOLOv5-based anti-dazzle plate detection model;

[0052] Step S3: Identifying the anti-dazzle plate image based on the detection model trained in step S2 to obtain coordinate information data of the anti-dazzle plate detection frame in the anti-dazzle plate image;

[0053] Step S4: According to prior knowledge, classifying the anti-dazzle plate in the detection frame into four categories: missing anti-dazzle plate, damaged anti-dazzle plate, tilted anti-dazzle plate, and intact anti-dazzle plate;

[0054] Step S5: Based on the pre-classification result of step S4, the height value of the anti-glare plate prediction frame, the front and rear spacing between the anti-glare plate prediction frames, and the aspect ratio information of the anti-glare plate prediction frame are calculated using the coordinate information data in step S3, and the calculation result is used to determine the health status of the anti-glare plate body or the anti-glare plate distribution.

[0055] In step S3, the coordinate information of the anti-glare plate detection frame is obtained by using the anti-glare plate recognition model obtained by training to recognize the anti-glare plate in the anti-glare plate image, so as to extract the detection area of the anti-glare plate in the anti-glare plate image, that is, the detection frame corresponding to the anti-glare plate.

[0056] In step S4, the classification is based on the absence of anti-glare plates, the presence of broken anti-glare plates, and the presence of skewed anti-glare plates.

[0057] The prior knowledge in step S4 is the coordinate information characteristics of the health status and non-health status of the anti-glare plate, specifically,

[0058] Feature A: The spacing between the detection frames corresponding to the anti-glare plates gradually decreases in the anti-glare plate image, and the spacing between the detection frame corresponding to the first anti-glare plate and the detection frame corresponding to the second anti-glare plate is obviously greater than the spacing between the detection frames corresponding to the anti-glare plates in the rear of the anti-glare plate image.

[0059] Feature B: When the anti-glare plate is damaged, the height value of the detection frame corresponding to the anti-glare plate decreases in the anti-glare plate image, and the height of the detection frame corresponding to each anti-glare plate in the health state gradually decreases in the anti-glare plate image, and the height difference between the detection frame corresponding to the first anti-glare plate and the detection frame corresponding to the second anti-glare plate is obviously greater than the height difference between the detection frames corresponding to the anti-glare plates in the rear of the anti-glare plate image.

[0060] Feature C: The aspect ratio of the detection frame corresponding to the skewed anti-glare plate is significantly different from the aspect ratio of the detection frame corresponding to other types of anti-glare plates.

[0061] Step S5 specifically includes the following steps:

[0062] Step S51: Determine whether there is a missing anti-glare plate phenomenon according to feature A, specifically: judge whether there is a missing anti-glare plate between the first and second anti-glare plates, and the formula is

[0063] X 2(min) -X 1(max) >2*(X 3(min) -X 2(max) ) Formula One;

[0064] Determine whether there is a missing anti-glare plate between the nth and (n-1)th anti-glare plates, and the formula is

[0065] X n(min) -X n-1(max) >K*(X n-1(min) -X n-2(max) ) Formula Two;

[0066] wherein X n-1(max) represents the maximum horizontal coordinate of the (n-1)th anti-glare panel detection frame; X n-1(min) represents the minimum horizontal coordinate of the (n-1)th anti-glare panel detection frame, and K is a to-be-determined coefficient related to the detection effect of the present step.

[0067] Step S52: determining whether there is an anti-glare panel damage phenomenon according to feature B, and the specific method is: according to the extracted anti-glare panel detection frame coordinate information, judging whether the first anti-glare panel is damaged, and the formula is

[0068] Y 1(max) -Y 1(min) <0.7*(Y 2(max) -Y 2(min) ) Formula Three;

[0069] judging whether the nth anti-glare panel is damaged, and the formula is

[0070] Y n(max) -Y n(min) <K1*(Y n-1(max) -Y n-1(min) ) Formula Four;

[0071] wherein Y n(max) represents the maximum vertical coordinate of the nth anti-glare panel detection frame; Y n(min) represents the minimum vertical coordinate of the nth anti-glare panel detection frame, and K1 is a to-be-determined coefficient related to the detection effect of the present step.

[0072] Step S53: determining whether there is an anti-glare panel tilt phenomenon according to feature C, and the formula is:

[0073] Y n(max) -Y n(min) *K2<(X n(max) -X n(min) ) Formula Five;

[0074] wherein K2 is a to-be-determined coefficient related to the detection effect of the present step.

[0075] Step S54: the anti-glare panel that does not meet the determination conditions of steps S51, S52 and S53 is determined to be in a healthy state of intact anti-glare panel.

[0076] In step S51, the detection effect is best when K is 1.1; in step S52, the detection effect is best when K1 is 0.6; in step S53, the detection effect is best when K2 is 0.42.

[0077] The binocular stereo road intelligent inspection system comprises a binocular road intelligent perception system, the binocular road intelligent perception system comprising a binocular camera, a GPS, an inertial measurement unit (IMU) and an artificial intelligence development board; the binocular road intelligent perception system is installed on the roof or window of a vehicle through a strong suction cup, and positioning information is obtained through the GPS and the IMU, and data is analyzed to trigger the binocular camera at a fixed distance to realize image acquisition.

[0078] In step S1, the binocular road intelligent perception system is used to collect relevant images of expressways and urban roads, and after preliminary data screening to remove invalid image data, a data set is formed, data samples are manually labeled through LabelImg data labeling software, anti-dazzle plate label information is generated, and an anti-dazzle plate recognition database for training an anti-dazzle plate detection model based on YOLOv5 is constructed.

[0079] In step S2, the YOLOv5 model is composed of four parts, namely, an input end, a Backbone, a Neck and a Prediction;

[0080] The input end is used to input original picture data, and a Mosaic data enhancement method is used to splice four randomly selected pictures through random scaling and random cropping operations to enrich the data set and improve the generalization of the anti-dazzle plate detection model;

[0081] The Backbone layer is composed of a Focus structure and a CSP structure, and is a convolutional neural network for extracting image features;

[0082] The Neck layer uses a path aggregation network (PANet) as an architecture to realize feature fusion;

[0083] The Prediction layer is used to output the regression of categories and bounding boxes;

[0084] When training the anti-dazzle plate detection model, the anti-dazzle plate recognition database is divided into a training set and a validation set in a ratio of 5:1; the training set is imported into the YOLOv5 network for model training, the validation set is used to realize output validation of the model, the number of training rounds is set to 300, the batch-size is set to 16, and the training picture size is set to 640.

[0085] In the construction of the anti-glare board training library, the example utilizes 100,000 images of expressways and urban roads collected by the binocular road intelligent perception system independently developed by the laboratory. After preliminary data screening to remove invalid image data, the dataset contains 2,300 valid images, and the image resolution is 2208x1242. The data samples are manually labeled by the LabelImg data labeling software to generate anti-glare board label information and construct an anti-glare board recognition database.

[0086] Preferably, the example combines the binocular road intelligent perception system and proposes a method for efficiently and accurately detecting defective anti-glare boards. This method can automatically identify defective anti-glare boards and replace manual inspection of anti-glare boards. Furthermore, the example fully considers the influence of complex actual road environments on the robustness of the model and establishes anti-glare board training databases for different complex road scenarios. The diversity of the database makes the generalization ability of the anti-glare board model stronger.

[0087] In particular, the example proposes a defective anti-glare board detection method combining YOLOv5 and position information to address the difficulty of accurately detecting and identifying anti-glare boards due to the small amount of defective anti-glare board data and the diverse forms of defects. The specific method is as follows: anti-glare board images are collected using the binocular road intelligent perception system independently developed by the laboratory to construct an anti-glare board training library; a YOLOv5 deep learning algorithm is used to train an anti-glare board target recognition model; based on prior knowledge, anti-glare boards are divided into four categories: missing anti-glare boards, damaged anti-glare boards, tilted anti-glare boards, and intact anti-glare boards; based on the deep learning recognition results, the coordinate information of the anti-glare board detection frame is used to detect whether the anti-glare board is defective by calculating the distance, height, and aspect ratio, and finally achieve automatic detection of the health status of the anti-glare board.

[0088] The above description is only a preferred embodiment of the present application, and any equivalent changes and modifications made within the scope of the present application should be included in the scope of the present application.

Claims

1. An automatic detection method for the health status of highway anti-glare panels, characterized in that: The detection method includes the following steps: Step S1: Use a binocular stereo intelligent road inspection system to acquire images of the anti-glare panel; Step S2: Train an anti-glare panel detection model based on YOLOv5; Step S3: Based on the detection model trained in step S2, identify the anti-glare panel image to obtain the coordinate information data of the anti-glare panel detection box within the anti-glare panel image; Step S4: Based on prior knowledge, classify the anti-glare panels in the detection frame into four categories: missing anti-glare panels, damaged anti-glare panels, tilted anti-glare panels, and intact anti-glare panels. Step S5: Based on the pre-classification results of step S4, use the coordinate information data in step S3 to calculate the height value of the anti-glare panel prediction frame, the front-to-back spacing between the anti-glare panel prediction frames, and the aspect ratio information of the anti-glare panel prediction frames. Use the calculation results to determine the health status of the anti-glare panel body or the distribution of anti-glare panels. The specific method for obtaining the coordinate information of the anti-glare panel detection box in step S3 is as follows: using the trained anti-glare panel recognition model, the anti-glare panel in the anti-glare panel image is identified, so as to extract the detection area of ​​the anti-glare panel in the anti-glare panel image, that is, the detection box corresponding to the anti-glare panel. In step S4, the classification criteria are as follows: missing anti-glare panels are determined by the presence of gaps between two anti-glare panels; damaged anti-glare panels are determined by the presence of breaks or damage; and tilted anti-glare panels are determined by the presence of distortion or tilting. The prior knowledge in step S4 is the coordinate information features of the anti-glare panel in its healthy and unhealthy states, specifically: Feature A: For anti-glare panels in a healthy state, the spacing between the detection boxes corresponding to the anti-glare panels gradually decreases in the anti-glare panel image, and the spacing between the detection boxes corresponding to the first and second anti-glare panels is significantly greater than the spacing between the detection boxes corresponding to the last few anti-glare panels in the anti-glare panel image. Feature B: When the anti-glare panel is damaged, the height value of its corresponding detection box becomes lower in the anti-glare panel image. The height of the detection boxes corresponding to each anti-glare panel in the healthy state gradually decreases in the anti-glare panel image. Moreover, the height difference between the detection boxes corresponding to the first anti-glare panel and the second anti-glare panel is significantly greater than the height difference between the detection boxes corresponding to the last few anti-glare panels in the anti-glare panel image. Feature C: The aspect ratio of the detection frame corresponding to the tilted anti-glare panel is significantly different from that of the detection frames corresponding to other types of anti-glare panels. The binocular intelligent road inspection system includes a binocular intelligent road perception system, which includes a binocular camera, GPS, an inertial measurement unit (IMU), and an artificial intelligence development board. The binocular intelligent road perception system is mounted on the roof or window of the vehicle using a powerful suction cup. It obtains positioning information and analyzes the data through GPS and IMU, and triggers the binocular camera at a fixed distance to achieve image acquisition. In step S1, a binocular road intelligent perception system is used to collect relevant images of highways and urban roads. After preliminary data screening to remove invalid image data, a dataset is formed. Data samples are manually labeled using LabelImg data annotation software to generate anti-glare panel label information and construct an anti-glare panel recognition database for training a YOLOv5-based anti-glare panel detection model.

2. The automatic health status detection method for highway anti-glare panels according to claim 1, characterized in that: Step S5 specifically includes the following steps; Step S51: Determine whether there is a missing anti-glare panel based on feature A. Specifically, determine if there is a missing panel between the first and second anti-glare panels, using the formula: X 2(min) -X 1(max) >2*(X 3(min) -X 2(max) Formula 1; The formula to determine whether there is a missing anti-glare panel between the nth and (n-1th)th anti-glare panels is: X n(min) -X n-1(max) >K*(X n-1(min) -X n-2(max) Formula 2; In the formula, X n-1(max) X represents the maximum x-coordinate of the (n-1)th anti-glare panel detection frame; n-1(min) The x-coordinate of the (n-1)th anti-glare panel detection frame is the minimum value, and K is an undetermined coefficient related to the detection effect of this step. Step S52: Determine whether there is damage to the anti-glare panel based on feature B. Specifically, based on the extracted coordinate information of the anti-glare panel detection frame, determine whether the first anti-glare panel is damaged. The formula is as follows: Y 1(max) -Y 1(min) <0.7*(Y 2(max) -Y 2(min) ) Formula 3; The formula for determining whether the nth anti-glare panel is damaged is: Y n(max) -Y n(min) <K1*(Y n-1(max) -Y n-1(min) ) Formula 4; In the formula, Y n(max) Y represents the maximum value of the ordinate of the nth anti-glare panel detection frame; n(min) K1 represents the minimum value of the ordinate of the nth anti-glare panel detection frame, and K1 is an undetermined coefficient related to the detection effect of this step. Step S53: Determine whether the anti-glare panel is tilted based on feature C. The formula is: Y n(max) -Y n(min) *K2<(X n(max) -X n(min) Formula 5; In the formula, K2 is an undetermined coefficient related to the detection effect of this step; Step S54: Anti-glare panels that do not meet the judgment conditions of steps S51, S52, and S53 are judged to be intact anti-glare panels in a healthy state.

3. The automatic health status detection method for highway anti-glare panels according to claim 2, characterized in that: In step S51, the detection effect is best when K is 1.1; in step S52, the detection effect is best when K1 is 0.6; in step S53, the detection effect is best when K2 is 0.

42.

4. The automatic health status detection method for highway anti-glare panels according to claim 1, characterized in that: In step S2, the YOLOv5 model consists of four parts: input, backbone, neck, and prediction. The input end is used to input the original image data. The Mosaic data augmentation method is used to stitch together four randomly selected images through random scaling and random cropping operations to enrich the dataset and improve the generalization of the anti-glare panel detection model. The Backbone layer consists of a Focus structure and a CSP structure, which is a convolutional neural network used to extract image features; the Neck layer uses the PANet path aggregation network as its architecture to achieve feature fusion. The Prediction layer is used to output the regression of categories and bounding boxes; When training the anti-glare panel detection model, the anti-glare panel recognition database is first divided into two categories, training set and validation set, in a 5:1 ratio. Then, the training set is imported into the YOLOv5 network for model training, and the validation set is used to verify the output of the model. The number of training rounds is set to 300, the batch size is 16, and the training image size is 640.

Images