Abnormal traffic sign diagnosis method based on image processing

Abstract

The invention discloses an abnormal traffic sign diagnosis method based on image processing, and the method comprises the steps: carrying out the preprocessing and data expansion of a traffic sign image through employing a Gaussian filtering algorithm, carrying out the manual marking of the traffic sign image, predicting the exact position of a traffic sign through employing a rotating target detection algorithm with a model main body being R3det, and carrying out the recognition of the traffic sign. Calculating a rotation angle of the traffic sign, cutting an output result by utilizing feature mapping, carrying out color clustering on the cutting result, outputting a color proportion and color feature information, and judging whether the traffic sign has chromatic aberration by comparing the color proportion with the color feature information; the traffic sign can be automatically detected and identified, traffic sign judgment including inclination diagnosis and chromatic aberration diagnosis can be completed in real time, the inclination diagnosis can judge whether the traffic sign is perpendicular to the horizontal plane or not, the chromatic aberration diagnosis can well quantify the color characteristics of the traffic sign, and a set of standards is provided for chromatic aberration problem diagnosis of the traffic sign.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an abnormal traffic sign diagnosis method based on image processing. Background Art

[0002] Traffic signs are one of the most important safety facilities on highways. Abnormal traffic signs cannot convey correct traffic information, which reduces road safety. Therefore, the diagnosis and maintenance of traffic signs are crucial. Existing diagnostic methods are too simple and there is no uniform standard. In contrast, the strictly designed automated abnormal traffic sign diagnosis algorithm can accurately detect and diagnose traffic signs and eliminate subjective influences. We use deep learning algorithms to perform tilt diagnosis on traffic signs and construct a set of diseased traffic sign datasets for training. For the color difference problem, we use color space conversion, clustering and other methods to diagnose and analyze the color of traffic signs. The experimental results show that the diagnostic model performs well on the constructed dataset. The mAP values ​​of prohibition traffic signs, warning traffic labels, instruction traffic signs, and no-stop traffic labels are 90.4%, 90.7%, 90.5%, and 90.8%, respectively. The results show that the abnormal traffic sign diagnosis model is effective in the application of actual scenarios.

[0003] In recent years, my country's highways have been developing rapidly. According to statistics, in 2012, the mileage of highways in my country was 4.23 million km. By the end of 2022, the mileage of highways in my country was 5.35 million km, and the mileage of highways in China increased by more than 1 million km in ten years. It can be seen that a large part of my country's highways were built during this period. According to the relevant provisions of "Road Traffic Reflective Film" (GB / T18833-2012), the recommended service life of reflective film, a key material for traffic signs, is generally 10 years. This means that in the next ten years (2021-2031), the traffic signs on more than 1 million km of roads in my country need to be maintained or replaced. If the mileage of highways built in the early stage that have reached the service life but have not been maintained and replaced in time is added, this number will be even larger. This will bring huge manpower and material resources consumption to the road management and maintenance departments. Traffic signs are important basic safety facilities that convey specific information to drivers through graphic symbols and text content on the road, thereby assisting vehicles to pass safely. Their layout content, setting, physical appearance, technical indicators, etc. are closely related to road traffic safety. At the same time, due to the characteristics of sign maintenance operations that are different from other operations, during the maintenance and replacement of traffic signs, maintenance work has a certain impact on vehicle driving. Therefore, it is necessary to conduct traffic sign inspections and maintenance operations in a scientific, standardized and strict manner to ensure that traffic signs on the road are in good working condition.

[0004] Road maintenance and management departments conduct daily inspections based on the road grade and traffic volume within their jurisdiction. Daily inspections refer to regular, fixed-line, and fixed-item inspections of exclusive routes. Daily inspections of expressways should be conducted at least twice during the day and once at night; daily inspections of national and provincial trunk roads should be conducted at least once a day, and night inspections should be conducted every two to three days; daily inspections of county and township roads should be conducted every three to five days, and night inspections should be conducted every five to ten days. Daily inspections mainly use subjective observation methods to check the integrity and appearance quality of traffic signs. Problems found during daily inspections should be recorded and reported immediately.

[0005] As one of the most important safety facilities for highways and urban roads, traffic signs are exposed to the natural environment and are subjected to long-term erosion by the natural environment, including the influence of natural conditions such as sunlight, rain erosion, temperature changes, humidity, altitude, dust adhesion, corrosive components in the air, and obstruction caused by dense plant growth, as well as the influence of damage caused by human conditions such as human damage, vehicle collisions, automobile exhaust, and skewness and stains caused by unstable foundations on the use function of traffic signs; at the same time, due to changes in road conditions after the signs are set up, the use function of traffic signs may not meet the functional goals when they were set up; especially for warning signs, prohibition signs, and instruction signs, once the traffic signs fail to meet the functional requirements of traffic signs, it will have a greater impact on driving safety.

[0006] However, due to the long-term influence of the idea of ​​"rebuilding instead of maintaining", the relevant national authorities have not yet issued industry standards specifically for regulating sign maintenance operations, especially for the updating of traffic signs, which lacks the necessary work standards as a basis. This directly leads to the excessive reliance of on-site staff on personal experience in the maintenance of traffic signs, and the obvious differences between people and time. This will cause two adverse effects. On the one hand, a large number of traffic signs that cannot meet driving safety are still in use, which is very harmful; on the other hand, a large number of signs that can meet safety needs are replaced, resulting in a huge waste of manpower, material and financial resources.

[0007] There are a lot of studies on automatic traffic sign detection, but almost no research on automatic traffic sign diagnosis. For automatic traffic sign diagnosis, it is difficult to determine which method can achieve good results, mainly due to the lack of a standard dataset for traffic sign diagnosis, which should contain traffic signs of various health conditions. Some representative traffic sign datasets are as follows: German Traffic Sign Recognition Benchmark (GTSRB): The dataset is very large, containing more than 50,000 German traffic sign images, with a total of 43 types of traffic signs; German Traffic Sign Detection Benchmark (GTSDB): This dataset contains 900 German traffic sign images. Traffic signs are divided into three categories: prohibition signs, danger signs, and mandatory signs; Swedish Traffic Sign Dataset (STSD): There are more than 3,000 Swedish traffic sign images in 10 categories. LISA Dataset: contains more than 6,000 US traffic sign images, including 49 categories; CSUST Chinese Traffic Sign Detection Benchmark (CCTSDB): Chinese traffic sign dataset, with a total of more than 10,000 traffic sign images, divided into three major categories: instruction signs, prohibition signs, and warning signs; The Tsinghua-Tencent 100K dataset: China traffic sign dataset, with more than 100,000 images and 30,000 traffic sign instances; Despite the existence of so many data sets, most of the traffic sign images contained in the above data sets are normal traffic signs, and there are fewer defective traffic sign images. In addition, in some data sets, there are a large number of images that only contain backgrounds but no traffic signs. Based on this, the traffic signs constructed in this paper are divided into four categories: instruction signs, prohibition signs, warning signs, and no-parking signs. The reason for separating no-parking signs from prohibition signs is for subsequent color difference diagnosis. This data set not only contains defective traffic sign images, but also differs from all the above data sets in that the angle information is annotated, which can enable the traffic sign diagnosis model to achieve better training results.

[0008] Traffic sign recognition generally includes two parts: traffic sign detection and traffic sign recognition. The task of traffic sign detection is to extract traffic signs from the background and pass the color, shape and other information of traffic signs to the traffic sign recognition module. Common methods for traffic sign detection include: color feature-based methods, shape feature-based methods and multi-feature fusion-based methods. Color is the basic attribute of traffic signs. Scholars initially used color spaces such as RGB, HSV / HIS to realize traffic sign detection. Shape information is also an important feature that distinguishes traffic signs from other objects. The most commonly used method for shape detection is Hough transform to infer the location of traffic signs. However, relying on only one feature cannot well extract traffic signs from the background, so scholars began to use multi-feature fusion to realize traffic sign detection. After detecting the traffic sign, the traffic sign is recognized by analyzing the features of the traffic sign. Common traffic sign recognition methods include template matching, traditional machine learning methods and deep learning methods. Template matching is widely used in the field of pattern recognition. This method is to translate the template in the image to be recognized to search for traffic signs, and find the traffic sign that best matches the template through a certain algorithm. This type of method is difficult to achieve the goal of automatic recognition in a complex natural background, so traffic sign recognition based on machine learning is a more popular method. For example, SVM, AdaBoost, tree classifier, etc. Traditional machine learning methods use manually designed features and conventional classifiers to achieve recognition. However, it takes a lot of time to design features, and as the amount of data increases, it is difficult for such methods to have sufficient judgment. Many countries have issued relevant guidelines and regulations for the diagnosis and maintenance of traffic signs, but the diagnosis method is mainly based on traditional manual naked eye detection, and no scholar has ever studied the method of automated diagnosis. Traditional diagnostic methods are simple to operate, but time-consuming and labor-intensive, and require high experience from technicians. In addition, manual detection methods are easily affected by the subjective influence of technicians, which ultimately makes the accuracy of traffic sign diagnosis low. In addition, the judgment criteria for some traffic sign problems are different, such as color difference problems. Even the same technician may have different judgment results on color difference problems at different times. Summary of the invention

[0009] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an abnormal traffic sign diagnosis method based on image processing.

[0010] The objective of the present invention is achieved through the following technical solutions: The present invention provides an abnormal traffic sign diagnosis method based on image processing, comprising the following steps: S1. According to the characteristics of the traffic sign image, a filtering algorithm is used to perform denoising on the collected traffic sign image to obtain a first image set; S2. Performing data enhancement on the first image set to increase the amount of data for model training, thereby obtaining a second image set, wherein each image in the second image set includes a traffic sign; S3, manually labeling each image in the second image set; S4, using the R3det model to detect, identify and diagnose the tilt of the traffic sign in each image in the second image set, and output the detection result; S5, cutting out the detection results using feature mapping; S6, performing color clustering on the cropping results, and outputting color ratio and color feature information; S7. Determine whether there is color difference in the traffic sign by comparing the color ratio and the color feature information.

[0011] Furthermore, in step S1, the filtering algorithm is Gaussian filtering, and Gaussian noise z is processed by using Gaussian filter. The probability density function of Gaussian noise z is: ,in represents the mean of z, represents the standard deviation of z, Represents the variance of z; the specific operation of Gaussian filtering is: use a Gaussian kernel to traverse each pixel in the image, and replace the gray value of the pixel in the center of the Gaussian kernel with the weighted average gray value of the pixels in the neighborhood determined by the Gaussian kernel.

[0012] Furthermore, in step S2, the method of performing data enhancement on the first image set includes any one or more combinations of random translation, transposition, mirroring, rotation, scaling, mosaic or random cropping.

[0013] Furthermore, the output result in step S4 is represented by a coordinate representation method; The coordinate representation is ,in , , and Represent the coordinates of the four vertices on the rotation box.

[0014] Preferably, the representation of the output result is converted into opencv format: Output the results of R3det Convert to opencv format , , and then calculate the result Add 90°, add 90° and then , where x is the horizontal coordinate of the center point of the rotating box, and y is the vertical coordinate of the center point of the rotating box. It refers to the acute angle between the rotation box and the x-axis, w represents the horizontal size of the rotation box, and h represents the vertical size of the rotation box.

[0015] Further, step S5 includes: rotating each image in the second image set according to the output result of R3det to obtain a third image, then performing coordinate mapping on the coordinates of the output result to obtain a first coordinate, and finally cropping each image in the second image set using the first coordinate, wherein the formula for coordinate mapping is: ; in is the coordinate of any point on the third image, are the coordinates of the rotation center, is the rotation angle, is the new coordinate after coordinate mapping.

[0016] Furthermore, the step S6 includes using a K-means clustering algorithm to search and cluster the colors in the traffic sign image.

[0017] Further, the step S7 includes converting the RGB color space into the CIELab color space, including the following sub-steps: S11. First convert the RGB color space to the CIE1931Lab color space, and calculate the stimulus values ​​X, Y, and Z of the three primary colors. The calculation method is: ; S12, get the stimulus values ​​X, Y, Z of the three primary colors, which are used to calculate the brightness And the chromaticity coordinates , The value of is calculated as: ;in , and The calculation method is ,in , or , The value of is 95.047, The value of is 100.0, The value of is 108.883.

[0018] Preferably, including psychological chromaticity and psychological hue angle The calculation method is: ;in In CIELab color space The adjustment parameters of the axis, for The arithmetic mean of for The correction value of for The correction value of for The correction value of for The correction value of for Correction value for brightness difference , poor color and hue difference The calculation method is: ;in Represents the brightness of the set standard color. Represents the brightness of the color of the sample to be tested, The color representing the color of the sample to be detected, The color representing the set standard color, represents the hue angle difference; Also includes weight coefficient , and The calculation method is: ;in for The arithmetic mean of for The arithmetic mean of for The arithmetic mean of is the rotation angle determined by the change in hue, is the rotation amplitude determined by the chroma change, refers to the rotation function, T refers to the hue angle function, and finally the color difference is calculated by the CIEDE2000 color difference formula: ,in , and is the set correction factor.

[0019] The beneficial effects of the present invention are: 1) The present invention can automatically detect and identify traffic signs, and complete the diagnosis of traffic signs in real time. The diagnostic functions include tilt diagnosis and color difference diagnosis. The tilt diagnosis can determine whether the traffic sign is perpendicular to the horizontal plane. The color difference diagnosis can well quantify the color characteristics of the traffic sign, providing a set of standards for the diagnosis of color difference problems of traffic signs. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of an abnormal traffic sign diagnosis method based on image processing according to an embodiment of the present invention; Figure 2 Schematic diagram of using Gaussian filtering processing in an embodiment of the present invention, wherein (a) is before Gaussian filtering processing, and (b) is after Gaussian filtering processing; Figure 3 Schematic diagram of an angle representation method of an abnormal traffic sign diagnosis method based on image processing according to an embodiment of the present invention, wherein (a) is an opencv definition method, and (b) is a coordinate representation method; Figure 4 This is a comparison diagram of annotations of an abnormal traffic sign diagnosis method based on image processing according to an embodiment of the present invention, wherein (a) is an annotation of an existing traffic sign dataset, and (b) is an annotation of MOTS; Figure 5 This is a diagram showing the final diagnosis effect of an abnormal traffic sign diagnosis method based on image processing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] Exemplarily, the flow chart of the present invention is as follows Figure 1 As shown, the present invention provides an abnormal traffic sign diagnosis method based on image processing, comprising the following steps: S1. According to the characteristics of the traffic sign image, a filtering algorithm is used to perform denoising on the collected traffic sign image to obtain a first image set; S2. Performing data enhancement on the first image set to increase the amount of data for model training, thereby obtaining a second image set, wherein each image in the second image set includes a traffic sign; S3, manually labeling each image in the second image set; S4, using the R3det model to detect, identify and diagnose the tilt of the traffic sign in each image in the second image set, and output the detection result; S5, cutting out the detection results using feature mapping; S6, performing color clustering on the cropping results, and outputting color ratio and color feature information; S7. Determine whether there is color difference in the traffic sign by comparing the color ratio and the color feature information.

[0023] For example, first, according to the characteristics of traffic sign images, a filtering algorithm is used to denoise the collected traffic sign images. In order to increase the data volume of the deep learning model, the denoised images are enhanced. Finally, the traffic sign images are annotated to produce a data set for model training. Image denoising is an important part of image processing. When traffic sign images are collected, the difficulty of traffic sign recognition increases sharply due to problems such as different viewing angles, uneven lighting, and weak edge information. Therefore, before identifying and diagnosing traffic signs, it is necessary to denoise the images to reduce the interference of irrelevant factors on traffic sign recognition and diagnosis, and improve the accuracy of the model in traffic sign diagnosis. Gaussian filter is a linear smoothing filter, which is mainly used to process Gaussian noise. In image processing, noise refers to isolated pixels or pixel blocks that cause visual effects. Gaussian noise refers to a type of noise whose probability density function obeys Gaussian distribution. Gaussian filter is a linear smoothing filter that selects weights according to the shape of the Gaussian function. Gaussian filter is very effective in processing Gaussian noise. If Gaussian noise is a variable in the image, the function of its probability density z is: ,in represents the mean of z, represents the standard deviation of z, Represents the variance of z; the specific operation of Gaussian filtering is: use a Gaussian kernel to traverse each pixel in the image, and use the weighted average gray value of the pixels in the neighborhood determined by the Gaussian kernel to replace the gray value of the pixel at the center of the Gaussian kernel; the result of using Gaussian filtering is as follows Figure 2 As shown, (a) is the image before Gaussian filtering, and (b) is the image after Gaussian filtering. The denoised image is subjected to geometric transformations such as rotation, mosaic, random cropping, random translation, transposition, mirroring, and scaling to achieve the purpose of expanding the data set.

[0024] Exemplarily, Refined Single-Stage Detector with Feature Refinement (R3det) is used to realize the detection, recognition and tilt diagnosis of traffic signs. R3det uses ResNet as Backbone and FPN as Neck, and adds FeatureRefinement Module (FRM) in Head. R3det uses ResNet as feature extraction network to solve the problem that when the network depth increases to a certain level, the effect of deeper network stacking becomes worse. FPN as Neck simultaneously integrates low-level detail information and high-level semantic information, increases the receptive field of the low-level, and enables the low-level to obtain more contextual information when detecting small targets. FRM is added to realize the regression and classification of bounding boxes. By continuously refining the parameters of bounding boxes, the regression of bounding boxes is made more accurate. In response to the problem of feature misalignment in the existing rotation target detection network, R3det designed a Feature Refinement Module. The Feature Refinement Module first filters out the bounding boxes with low confidence through Bbox Filtering, retains only the bounding boxes with the highest confidence, and ensures that each feature point corresponds to a bounding box. Then, a new feature map is added through the feature maps output by the convolutional FPN network twice, and then feature interpolation is performed based on the feature vector of the retained bounding box and the feature vector of the feature map output in the second step to obtain a more accurate feature vector. The feature interpolation method is bilinear interpolation. By traversing all feature points, a new feature map is obtained, and finally the feature map is added to the original feature map to obtain the final feature map.

[0025] Specifically, Figure 3 The following is the correct output of the rotation frame angle information. The R3det output result Convert to opencv format , , and then calculate the result Add 90°, add 90° and then , this method is defined by opencv, It refers to the acute angle between the rotating box and the x-axis. x is the horizontal coordinate of the center point of the rotating box, y is the vertical coordinate of the center point of the rotating box, the side formed by the rotating box is width (w), and the other side is height (h), such as Figure 3 As shown in (a) in the figure, the coordinate notation is: , this representation is S 2The output method of A-Net is to start from the point in the upper left corner of the rotating box and arrange it clockwise, where: Represents the coordinates of the starting point of the upper left corner, Represents the coordinates of the second point arranged clockwise, Represents the coordinates of the third point arranged clockwise, Represents the coordinates of the fourth point arranged clockwise, such as Figure 3 As shown in (b) in .

[0026] Exemplarily, the color difference diagnosis of traffic signs is mainly based on two points: 1. The proportion of characteristic colors in traffic signs. 2. The color difference between the clustered main color and the standard color is calculated, and whether the calculated result is less than the set color threshold. Traffic signs that meet the above two conditions at the same time are judged as traffic signs with no obvious color difference; first, coordinate mapping is performed, and each image in the second image set is rotated according to the output result of R3det to obtain the third image, and then the coordinates of the output result are mapped to obtain the first coordinates, and finally each image in the second image set is cropped using the first coordinates, where the coordinate mapping formula is: Finally, each image in the second image set is cropped using the first coordinates, where the coordinate mapping formula is: ; in is the coordinate of any point on the third image, are the coordinates of the rotation center, is the rotation angle, is the new coordinate after coordinate mapping.

[0027] For example, the second step of color difference diagnosis requires color clustering, S 2 A-Net can output the detected traffic sign image and use K-means clustering algorithm to search and cluster the colors in the traffic sign image. N Samples { n 1 ,n 2 ,n 3 …n i}( i =1,2,3…N) and K Cluster centers { C 1 ,C 2 ,C 3 …C k} to achieve N The samples are divided into K Each sample is the extracted pixel r,g,bChannel value, the specific steps include: setting the cluster number K according to the color characteristics of traffic signs; randomly selecting from all data points K Points as initial cluster centers { C 1 ,…C K}; Calculate the distance from all data points to the cluster center, assign all data points to the cluster corresponding to the initial cluster center closest to them and mark {1, 2, 3 …K}, where the formula for calculating the Euclidean distance is ; Calculate the mean of each type of assigned data set and use it as the new cluster center; Repeat the two calculation steps until the cluster center of the image does not change.

[0028] Specifically, color difference diagnosis also requires color space conversion. The CIELab color space is a uniform color space model and a color model for visual perception. L ) and two color components a, b The color space is uniform in terms of perception. Two colors that people perceive as close visually are also close in position in the CIELab color space. And the chromaticity coordinates , Define CIE color space. To convert from RGB color space to CIELab color space, you must first convert to CIE1931Lab color space and calculate the stimulus values ​​of the three primary colors. X, Y, Z This transformation is a linear transformation, which is implemented by a 3×3 matrix. The matrix formula is as follows: ; Get the stimulus values ​​of the three primary colors X, Y, Z After that, it is used to calculate the brightness And the chromaticity coordinates , The value is calculated as follows: ;in , and The calculation method is ,in , or , The value of is 95.047, The value of is 100.0, The value of is 108.883.

[0029] Specifically, the last step of color difference diagnosis is to calculate the color difference using the CIEDE2000 color difference formula. The CIEDE2000 color difference formula is an image evaluation standard that includes the comprehensive color difference visual perception of brightness, hue, and saturation. The CIEDE2000 color difference formula is: ;in , and is the correction coefficient set; the calculation method of other parameters is: including psychological chromaticity and psychological hue angle The calculation method is: ;in In CIELab color space The adjustment parameters of the axis, for The arithmetic mean of for The correction value of for The correction value of for The correction value of for The correction value of for Correction value for brightness difference , poor color and hue difference The calculation method is: ;in Represents the brightness of the set standard color. Represents the brightness of the sample color to be tested, The color representing the color of the sample to be detected, The color representing the set standard color, represents the hue angle difference; Also includes weight coefficient , and The calculation method is: ;in for The arithmetic mean of for The arithmetic mean of for The arithmetic mean of is the rotation angle determined by the change in hue, is the rotation amplitude determined by the chroma change, refers to the rotation function and T refers to the hue angle function.

[0030] For example, a labeling comparison of an abnormal traffic sign diagnosis method based on image processing is as follows: Figure 4 As shown in the figure, (a) is the existing traffic sign dataset annotation, (b) is the MOTS annotation, using the Ubuntu22.04 system with CPU i7-11700 and GPU NVIDIA GTX-3080. The model is trained for a total of 12 epochs, with an initial learning rate of 0.0025, weight decay and momentum of 0.0001 and 0.9 respectively, and a minibatch of 2. First, we start the tilt detection evaluation. The results of R3det when using MOTS annotation are shown in Table 1, where R3Det_tiny represents the first-order FRM and R3Det_redfined represents the second-order FRM. R3det performs excellently on MOTS, with an mAP value of 90.6%. The introduction of FRM improves the detection performance of R3det. Table 1 shows the impact of the FRM order on the performance of R3det. The mAP value of 90.6% also means that the model can excellently complete the task of traffic sign tilt diagnosis in actual scenarios. Traffic signs with an inclination angle greater than 15 degrees are marked as WR, and traffic signs with an inclination angle less than 15 degrees are marked as WOR.

[0031] Table 1 Model performance comparison Exemplarily, the color difference diagnosis evaluation is performed after the tilt detection evaluation. The main task of the color difference diagnosis evaluation is to confirm the appropriate color ratio threshold and the color difference formula threshold. The determination of the color ratio threshold is relatively simple. The detection result output by R3det is converted into a color space and a mask is made according to the HSV basic color component range table (as shown in Table 2). The color ratio threshold of each type of traffic sign can be determined by the black and white ratio of the mask. The color ratio thresholds of jl, jg, zs, and jt are 15: Table 2 HSV basic color component range table The color standard RGB values ​​of the four categories of jl, jg, zs, and jt are set to (212, 26, 24), (198, 175, 97), (66, 118, 171), and (212, 26, 24). The traffic signs collected under different weather conditions that meet the human eye's judgment of no color difference are clustered to extract the main color and calculated with the CIEDE2000 color formula to finally obtain the color difference formula thresholds of various traffic signs. The calculation results of the sample colors and standard colors of various traffic signs are shown in Tables 3 to 6: Table 3 Calculation of color difference values ​​of prohibition signs Table 4 Calculation of color difference values ​​for warning signs Table 5 Calculation of color difference values ​​of indicator signs Table 6 Calculation of color difference value of no parking sign According to the above calculation results, the thresholds of various color difference formulas are determined to be 30, 25, 28 and 30. Traffic signs with color difference are marked as WCA, and traffic signs without color difference are marked as WOCA. The final diagnostic effect diagram is shown as follows: Figure 5 shown.

[0032] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for diagnosing abnormal traffic signs based on image processing, characterized in that: The following steps are involved: S1. According to the characteristics of the traffic sign image, a filtering algorithm is used to perform denoising on the collected traffic sign image to obtain a first image set; S2. Performing data enhancement on the first image set to increase the amount of data for model training, thereby obtaining a second image set, wherein each image in the second image set includes a traffic sign; S3, manually labeling each image in the second image set; S4, using the R3det model to detect, identify and diagnose the tilt of the traffic sign in each image in the second image set, and output the detection result; S5, cutting out the detection results using feature mapping; S6, performing color clustering on the cropping results, and outputting color ratio and color feature information; S7. Determine whether there is color difference in the traffic sign by comparing the color ratio and the color feature information.

2. The abnormal traffic sign diagnosis method based on image processing according to claim 1 is characterized by: The filtering algorithm in step S1 is Gaussian filtering, and Gaussian filter is used to process Gaussian noise z. The probability density function of Gaussian noise z is: ,in represents the mean of z, represents the standard deviation of z, Represents the variance of z; the specific operation of Gaussian filtering is: use a Gaussian kernel to traverse each pixel in the image, and replace the gray value of the pixel in the center of the Gaussian kernel with the weighted average gray value of the pixels in the neighborhood determined by the Gaussian kernel.

3. The abnormal traffic sign diagnosis method based on image processing according to claim 1 is characterized by: In step S2, the method of performing data enhancement on the first image set includes any one or more combinations of random translation, transposition, mirroring, rotation, scaling, mosaic or random cropping.

4. The abnormal traffic sign diagnosis method based on image processing according to claim 1 is characterized by: The output result in step S4 is represented by coordinate representation; The coordinate representation is ,in , , and Represent the coordinates of the four vertices on the rotation box.

5. The abnormal traffic sign diagnosis method based on image processing according to claim 4 is characterized in that: Convert the output result to opencv format: Output the results of R3det Convert to opencv format , , and then calculate the result Add 90°, add 90° and then , where x is the horizontal coordinate of the center point of the rotating box, and y is the vertical coordinate of the center point of the rotating box. It refers to the acute angle between the rotation box and the x-axis, w represents the horizontal size of the rotation box, and h represents the vertical size of the rotation box.

6. The abnormal traffic sign diagnosis method based on image processing according to claim 1 is characterized by: The step S5 includes: rotating each image in the second image set according to the output result of R3det to obtain a third image, then performing coordinate mapping on the coordinates of the output result to obtain a first coordinate, and finally cropping each image in the second image set using the first coordinate, wherein the coordinate mapping formula is: ； in is the coordinate of any point on the third image, are the coordinates of the rotation center, is the rotation angle, is the new coordinate after coordinate mapping.

7. The abnormal traffic sign diagnosis method based on image processing according to claim 1 is characterized by: The step S6 includes using a K-means clustering algorithm to search and cluster the colors in the traffic sign image.

8. The abnormal traffic sign diagnosis method based on image processing according to claim 1 is characterized in that: The step S7 includes converting the RGB color space into the CIELab color space, including the following sub-steps: S11. First convert the RGB color space to the CIE1931Lab color space, and calculate the stimulus values ​​X, Y, and Z of the three primary colors. The calculation method is: ； S12, get the stimulus values ​​X, Y, Z of the three primary colors, which are used to calculate the brightness And the chromaticity coordinates , The value of is calculated as: ;in , and The calculation method is ,in , or , The value of is 95.047, The value of is 100.0, The value of is 108.

883.

9. The abnormal traffic sign diagnosis method based on image processing according to claim 8 is characterized in that: Psychological chromaticity and psychological hue angle The calculation method is: ;in In CIELab color space The adjustment parameters of the axis, for The arithmetic mean of for The correction value of for The correction value of for The correction value of for The correction value of for Correction value for brightness difference , poor color and hue difference The calculation method is: ;in Represents the brightness of the set standard color. Represents the brightness of the sample color to be tested, The color representing the color of the sample to be detected, The color representing the set standard color, represents the hue angle difference; Also includes weight coefficient , and The calculation method is: ;in for The arithmetic mean of for The arithmetic mean of for The arithmetic mean of is the rotation angle determined by the change in hue, is the rotation amplitude determined by the chroma change, refers to the rotation function, T refers to the hue angle function, and finally the color difference is calculated by the CIEDE2000 color difference formula: ,in , and is the set correction factor.

Images