Traffic Light Color Recognition Method, Device, Electronic Device, and Storage Medium

Abstract

The present application discloses a traffic light color recognition method, apparatus, electronic device, and storage medium. The method includes: obtaining an original traffic light color recognition result, where the original traffic light color recognition result includes a traffic light detection frame; determining the brightness of the pixel points in the traffic light detection frame, and filtering the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points; determining the color integral of the filtered pixel points according to the channel values of the filtered pixel points in a preset color channel and the positions of the filtered pixel points in the traffic light detection frame; and determining the traffic light color corresponding to the traffic light detection frame based on the color integral of the filtered pixel points as the final traffic light color recognition result. The traffic light color recognition method of the present application reduces the influence of factors such as background and illumination in the traffic light detection frame on color recognition, and improves the accuracy of traffic light color recognition for autonomous driving vehicles.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular, to a traffic light color recognition method, device, electronic device, and storage medium. Background Art

[0002] Detecting and recognizing the color of the traffic light ahead through a camera and correctly understanding its meaning are essential capabilities for autonomous driving vehicles. Traffic light color recognition is an essential link among them. The result of traffic light color recognition is related to whether the autonomous driving vehicle should accelerate or decelerate, drive or stop. Therefore, autonomous driving vehicles have high requirements for the accuracy of traffic light color recognition.

[0003] Currently, autonomous driving vehicles recognize traffic lights through cameras, generally using mature object detection models such as yolov5 to achieve. By training the object detection model, it can detect the traffic light target box in the image and output its category at the same time, such as red light / yellow light / green light. In the case of high imaging quality, the object detection model can correctly distinguish the color of the traffic light.

[0004] However, in actual road conditions, when the traffic light occupies a small number of pixels in the image, the detection box output by the object detection model usually cannot properly enclose the traffic light, resulting in an increase in interference from the background part. When the natural light is strong, the color of the traffic light in the detection box will be greatly distorted, and at this time, the traffic light color recognition result output by the object detection model also becomes untrustworthy. Summary of the Invention

[0005] Embodiments of the present application provide a traffic light color recognition method, device, electronic device, and storage medium to improve the accuracy of traffic light color recognition.

[0006] Embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, embodiments of the present application provide a traffic light color recognition method, where the method includes:

[0008] Obtain an original traffic light color recognition result, where the original traffic light color recognition result includes a traffic light detection box;

[0009] Determine the brightness of the pixel points in the traffic light detection box, and filter the pixel points in the traffic light detection box according to the brightness of the pixel points in the traffic light detection box to obtain filtered pixel points;

[0010] Determine the color integral of the filtered pixel points according to the channel values of the filtered pixel points in a preset color channel and the positions of the filtered pixel points in the traffic light detection box;

[0011] Determine the traffic light color corresponding to the traffic light detection frame based on the color integral of the filtered pixel points as the final traffic light color recognition result.

[0012] Optionally, determining the brightness of the pixel points in the traffic light detection frame and filtering the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain the filtered pixel points includes:

[0013] Determine whether the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result;

[0014] In the case where the original traffic light color recognition result does not meet the credibility requirement of the preset traffic light recognition result, determine the brightness of the pixel points in the traffic light detection frame, and filter the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain the filtered pixel points.

[0015] Optionally, determining whether the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result includes:

[0016] Determine whether the size of the traffic light detection frame meets the preset size requirement, and determine whether the saturation of the pixel points in the traffic light detection frame meets the preset saturation requirement;

[0017] If the size of the traffic light detection frame meets the preset size requirement and the saturation of the pixel points in the traffic light detection frame meets the preset saturation requirement, determine that the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result;

[0018] If the size of the traffic light detection frame does not meet the preset size requirement, and / or the saturation of the pixel points in the traffic light detection frame does not meet the preset saturation requirement, determine that the original traffic light color recognition result does not meet the credibility requirement of the preset traffic light recognition result.

[0019] Optionally, determining the brightness of the pixel points in the traffic light detection frame includes:

[0020] Determine the pixel values of the pixel points in the traffic light detection frame in the RGB color space;

[0021] Determine the brightness of the pixel points in the traffic light detection frame according to the pixel values of the pixel points in the traffic light detection frame in the RGB color space.

[0022] Optionally, the preset color channels include the R channel and the G channel in the RGB color space. Determining the color integral of the filtered pixel points according to the channel values of the filtered pixel points in the preset color channels and the positions of the filtered pixel points in the traffic light detection frame includes:

[0023] Determining the channel value difference of the filtered pixel points according to the channel values of the filtered pixel points in the R channel and the G channel;

[0024] Determining the position weight of the filtered pixel points according to the positions of the filtered pixel points in the traffic light detection frame;

[0025] Determining the color integral of the filtered pixel points according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points.

[0026] Optionally, there are multiple filtered pixel points. Determining the position weight of the filtered pixel points according to the positions of the filtered pixel points in the traffic light detection frame includes:

[0027] Determining the position of the central pixel point of the traffic light detection frame;

[0028] Determining the position weight of the filtered pixel points according to the distances between the filtered pixel points and the central pixel point.

[0029] Optionally, determining the color integral of the filtered pixel points according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points includes:

[0030] If the channel value difference of the filtered pixel points is within the first interval range, determining the integral of the first traffic light color according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points;

[0031] If the channel value difference of the filtered pixel points is within the second interval range, determining the integral of the second traffic light color according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points;

[0032] If the channel value difference of the filtered pixel points is within the third interval range, determining the integral of the third traffic light color according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points.

[0033] Optionally, there are multiple filtered pixel points. Determining the traffic light color corresponding to the traffic light detection frame according to the color integrals of the filtered pixel points includes:

[0034] Assign the color integrals of each filtered pixel to the corresponding traffic light color respectively;

[0035] Determine the traffic light color corresponding to the traffic light detection frame according to the color integrals of each traffic light color.

[0036] In a second aspect, an embodiment of the present application further provides a traffic light color recognition device, where the device includes:

[0037] An acquisition unit, configured to acquire an original traffic light color recognition result, where the original traffic light color recognition result includes a traffic light detection frame;

[0038] A filtering unit, configured to determine the brightness of the pixels in the traffic light detection frame, and filter the pixels in the traffic light detection frame according to the brightness of the pixels in the traffic light detection frame to obtain filtered pixels;

[0039] A first determination unit, configured to determine the color integral of the filtered pixel according to the channel value of the filtered pixel in a preset color channel and the position of the filtered pixel in the traffic light detection frame;

[0040] A second determination unit, configured to determine the traffic light color corresponding to the traffic light detection frame according to the color integral of the filtered pixel as the final traffic light color recognition result.

[0041] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0042] A processor; and

[0043] A memory arranged to store computer-executable instructions, where the executable instructions, when executed, cause the processor to execute any one of the foregoing methods.

[0044] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device is caused to execute any one of the foregoing methods.

[0045] The above at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects: In the traffic light color recognition method of the embodiments of the present application, first obtain the original traffic light color recognition result, and the original traffic light color recognition result includes a traffic light detection frame; then determine the brightness of the pixel points in the traffic light detection frame, and filter the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain the filtered pixel points; then determine the color integral of the filtered pixel points according to the channel values of the filtered pixel points in the preset color channel and the positions of the filtered pixel points in the traffic light detection frame; finally, determine the traffic light color corresponding to the traffic light detection frame according to the color integral of the filtered pixel points as the final traffic light color recognition result. The traffic light color recognition method of the embodiments of the present application reduces the influence of factors such as the background part and illumination in the traffic light detection frame on color recognition, and improves the accuracy of traffic light color recognition for autonomous driving vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0047] Figure 1 is a schematic flow chart of a traffic light color recognition method in an embodiment of the present application;

[0048] Figure 2 is a schematic flow chart of a traffic light color recognition process in an embodiment of the present application;

[0049] Figure 3 is a schematic structural diagram of a traffic light recognition device in an embodiment of the present application;

[0050] Figure 4 is a schematic structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] In order to make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0052] The following will describe in detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.

[0053] The traffic light color recognition solutions in the prior art mainly detect the traffic light detection frame through a target detection model and output its color category at the same time. However, when the traffic light detection frame occupies a relatively small proportion in the image or the natural light is strong, the recognition accuracy will be greatly reduced.

[0054] There is another solution. First, a target detection model is used to detect the traffic light target frame, and at the same time, the original RGB image is converted into an HSV image. The HSV color space is designed from the perspective of the human visual sense and is convenient for humans to distinguish colors. The traffic light target frame is cropped out in the HSV image, and the proportion of the number of red pixels, yellow pixels, and green pixels is respectively counted. The color with the highest proportion of the number of pixels is used as the final traffic light color. However, the background part other than the traffic light in the traffic light detection frame obtained by this solution will affect the proportion of various color pixels, and color differences will occur when the natural light is strong, which will also affect the statistical values of various color pixels, resulting in a reduction in recognition accuracy.

[0055] In addition, when the S channel value in the HSV color space is too low, the saturation of the image pixels will be too low. Judging only from the threshold in the HSV color space, the traffic light part can no longer be accurately recognized, and the outside of the traffic light detection frame already has a tendency to show yellow or red due to light. In this case, it is very difficult to accurately judge the traffic light color by simply counting the proportion of red, yellow, and green pixels in the traffic light target frame.

[0056] Based on this, the embodiments of the present application provide a traffic light color recognition method. As Figure 1 shown, a flow diagram of a traffic light color recognition method in the embodiments of the present application is provided. The method at least includes the following steps S110 to step S140:

[0057] Step S110, obtaining the original traffic light color recognition result, where the original traffic light color recognition result includes a traffic light detection frame.

[0058] When the embodiments of the present application recognize the traffic light color, it is necessary to first obtain the original traffic light color recognition result. The original traffic light color recognition result can be obtained based on the existing target detection model. By training in advance a target detection model such as yolov5 that can detect traffic lights, and then inputting the image containing the traffic light into the trained target detection model, the traffic light detection frame and traffic light color type detected by the target detection model can be obtained.

[0059] Step S120, determining the brightness of the pixels in the traffic light detection frame, and filtering the pixels in the traffic light detection frame according to the brightness of the pixels in the traffic light detection frame to obtain the filtered pixels.

[0060] Since the original traffic light color recognition result obtained in the foregoing steps may be affected by the background or lighting, etc., resulting in inaccurate recognition results, the embodiments of the present application can further optimize the traffic light color recognition result based on the original traffic light color recognition result, thereby improving the accuracy of traffic light color recognition.

[0061] Specifically, the luminance value Luminance of each pixel point in the traffic light detection frame can be determined first. The level of the luminance value largely affects the recognition effect of the traffic light color. Therefore, the luminance value of each pixel point in the traffic light detection frame can be used here to filter the pixel points in the traffic light detection frame, so as to filter out the pixel points that do not meet the preset luminance value requirements.

[0062] Step S130, determine the color integral of the filtered pixel points according to the channel values of the filtered pixel points in the preset color channel and the positions of the filtered pixel points in the traffic light detection frame.

[0063] The color of a pixel point can be represented in different color spaces. For example, in the RGB color space, it corresponds to three channels: the R channel, the G channel, and the B channel. The channel values of each channel are all distributed in the range of 0 to 255. When the channel value of the R channel is relatively larger, the color of the pixel point appears red. When the channel value of the G channel is relatively larger, the color of the pixel point appears red. When the channel values of the R channel and the G channel are relatively close, the color of the pixel point appears yellow. Based on this, the embodiments of the present application can roughly distinguish the color of pixel points according to the relative magnitudes of the channel values of the R channel and the G channel.

[0064] In addition, the position of the pixel point in the traffic light detection frame also affects the accuracy of color recognition. Here, it is mainly considered that if the pixel points are distributed near the edge of the traffic light detection frame, the interference from the background part will be relatively large. Therefore, the position of the pixel point in the traffic light detection frame also needs to be taken into consideration.

[0065] Based on the above two dimensions, the color integral of the filtered pixel points can be determined. Here, the color integral can be regarded as the cumulative assignment for each traffic light color. Finally, the traffic light color that the traffic light detection frame is more inclined to can be determined according to the numerical value of the color integral.

[0066] Step S140, determine the traffic light color corresponding to the traffic light detection frame according to the color integral of the filtered pixel points, as the final traffic light color recognition result.

[0067] After determining the color integrals of all filtered pixel points, the color integral corresponding to each traffic light color can be used to determine which traffic light color has the largest color integral, and finally the traffic light color corresponding to the traffic light detection frame can be determined as the final traffic light color recognition result.

[0068] The traffic light color recognition method according to the embodiments of the present application reduces the influence of factors such as the background part and illumination in the traffic light detection frame on color recognition, and improves the accuracy of traffic light color recognition for autonomous driving vehicles.

[0069] In an embodiment of the present application, determining the brightness of the pixel points in the traffic light detection frame and filtering the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain the filtered pixel points includes: determining whether the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result; in the case where the original traffic light color recognition result does not meet the credibility requirement of the preset traffic light recognition result, determining the brightness of the pixel points in the traffic light detection frame, and filtering the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain the filtered pixel points.

[0070] As mentioned above, the traffic light color recognition result output by the object detection model for detecting traffic light colors is inaccurate in some cases, and then the traffic light color recognition process of the embodiments of the present application is executed. Therefore, after obtaining the original traffic light color recognition result, the embodiments of the present application can first determine whether the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result, for example, whether the traffic light color recognition result is interfered by factors such as the background part or illumination.

[0071] If the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result, it means that the original traffic light color recognition result is accurate and reliable, that is, it can be trusted by the autonomous driving vehicle. On the contrary, it means that it is not accurate and reliable enough, and then the subsequent processing process of the present application can be further executed.

[0072] It should be noted that the above judgment process can be an optional step. Those skilled in the art can determine whether to execute the subsequent process of the embodiments of the present application after judgment, or of course, can also omit the judgment link and directly apply the traffic light color recognition solution of the embodiments of the present application.

[0073] In one embodiment of the present application, determining whether the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result includes: determining whether the size of the traffic light detection frame meets the preset size requirement, and determining whether the saturation of the pixel points in the traffic light detection frame meets the preset saturation requirement; if the size of the traffic light detection frame meets the preset size requirement and the saturation of the pixel points in the traffic light detection frame meets the preset saturation requirement, it is determined that the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result; if the size of the traffic light detection frame does not meet the preset size requirement, and / or the saturation of the pixel points in the traffic light detection frame does not meet the preset saturation requirement, it is determined that the original traffic light color recognition result does not meet the credibility requirement of the preset traffic light recognition result.

[0074] When determining whether the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result in the embodiments of the present application, it can start from the following two aspects. One aspect is the interference of the background part in the traffic light detection frame, and the other aspect is the influence of light on the traffic light detection frame.

[0075] Based on the first aspect, the embodiments of the present application can first determine the size of the traffic light detection frame and judge whether the size of the traffic light detection frame meets the preset size threshold requirement. If the size of the traffic light detection frame is smaller than the preset size threshold, it means that the distance between the autonomous vehicle and the traffic light is relatively far, resulting in a small proportion of the traffic light detection frame in the entire image. At this time, the traffic light detection frame cannot properly enclose the position where the traffic light is located, that is, it exceeds the size of the traffic light, which leads to more background parts in the traffic light detection frame. When recognizing the color of the traffic light detection frame, more interference will be introduced, resulting in the untrustworthiness of the traffic light color predicted by the target detection model. On the contrary, if the size of the traffic light detection frame is not smaller than the preset size threshold, it means that the proportion of the traffic light detection frame in the entire image is relatively large, and the traffic light detection frame can properly enclose the position where the traffic light is located, with less background interference.

[0076] Therefore, by setting the size threshold, it can be determined whether the size of the traffic light detection frame meets the preset size threshold requirement, and further determine whether there is a problem of background interference. The size of this size threshold can be set to, for example, the size of 15 pixel points. By comparing the length of the minimum side of the traffic light detection frame with the size of 15 pixel points, it can be determined whether the preset size threshold requirement is met. Of course, those skilled in the art can also adjust it flexibly according to the actual situation, and no specific limitation is made here.

[0077] From a second perspective, embodiments of the present application can measure the influence of light by the saturation of pixel points. Specifically, the saturation Saturation of each pixel point can be calculated in the following manner:

[0078] Saturation = (max(R, G, B) - min(R, G, B)) / max(R, G, B)

[0079] After that, count the number of pixel points with saturation Saturation < 50%. When the proportion of the number of pixel points with saturation Saturation < 50% reaches 10%, it indicates that the imaging effect is greatly affected by light. Therefore, it can be considered that the traffic light color predicted by the target detection model at this time is not credible. Conversely, it indicates that the influence of light is small. For the settings of the above "50%" and "10%" values, those skilled in the art can flexibly adjust according to the actual situation and no specific limitation is made here.

[0080] It should be noted that the above two perspectives can be used together to determine whether the original traffic light color recognition result meets the credibility requirements of the preset traffic light recognition result. Of course, those skilled in the art can also choose any one of them for judgment.

[0081] In an embodiment of the present application, determining the brightness of pixel points in the traffic light detection frame includes: determining the pixel values of pixel points in the traffic light detection frame in the RGB color space; determining the brightness of pixel points in the traffic light detection frame according to the pixel values of pixel points in the traffic light detection frame in the RGB color space.

[0082] Embodiments of the present application can determine the brightness of each pixel point in the traffic light detection frame in the following manner:

[0083] Luminance = max(R, G, B) / 255 * 100%

[0084] Based on the brightness of each pixel point in the traffic light detection frame, pixel points in the traffic light detection frame can be filtered to filter out pixel points that do not meet the requirements of the preset brightness value. The requirements of the preset brightness value can be a preset brightness value threshold. For example, it can be set that the brightness value Luminance >= 50% to eliminate pixel points with lower brightness values. The brightness value threshold of 50% is an empirical threshold, and those skilled in the art can flexibly adjust according to actual needs and no specific limitation is made here.

[0085] In an embodiment of the present application, the preset color channels include the R channel and the G channel in the RGB color space. Determining the color integral of the filtered pixel points according to the channel values of the filtered pixel points in the preset color channels and the positions of the filtered pixel points in the traffic light detection frame includes: determining the channel value difference of the filtered pixel points according to the channel values of the filtered pixel points in the R channel and the G channel; determining the position weight of the filtered pixel points according to the positions of the filtered pixel points in the traffic light detection frame; and determining the color integral of the filtered pixel points according to the channel value difference and the position weight of the filtered pixel points.

[0086] In the embodiment of the present application, only a three-classification of "red / yellow / green" needs to be performed on the traffic light detection frame. In the RGB color space, the pixel values of the most typical red / yellow / green can be represented as (255, 0, 0) / (255, 255, 0) / (0, 255, 0) respectively. That is, the R channel represents the red component, the G channel represents the green component, and when R and G are close, it represents the yellow component. Therefore, in the embodiment of the present application, the color of the pixel points can be roughly distinguished based on the relative magnitudes of the channel values of the R channel and the G channel in the RGB color space.

[0087] Specifically, in the embodiment of the present application, the channel value difference between the two channels of the filtered pixel points can be calculated according to the channel values of the filtered pixel points in the R channel and the G channel, which can be expressed as delta = R - G, and this is used as a dimension for determining the color integral of the filtered pixel points.

[0088] In addition, the position of the pixel points in the traffic light detection frame also affects the accuracy of color recognition. If the pixel points are closer to the edge of the traffic light detection frame, the greater the interference from the background part. If the pixel points are farther from the edge of the traffic light detection frame, the smaller the interference from the background part. Based on this, in the embodiment of the present application, the position weight weight of each pixel point is set based on the position of the pixel point in the traffic light detection frame, and then the color integral of the pixel points can be adjusted by the magnitude of the position weight of each pixel point.

[0089] In an embodiment of the present application, there are multiple filtered pixel points. Determining the position weight of the filtered pixel points according to the positions of the filtered pixel points in the traffic light detection frame includes: determining the position of the central pixel point of the traffic light detection frame; and determining the position weight of the filtered pixel points according to the distances between each of the filtered pixel points and the central pixel point.

[0090] Based on the foregoing embodiments, if a pixel is closer to the edge of the traffic light detection frame, it means that the pixel is farther from the center point of the traffic light detection frame. Conversely, if a pixel is farther from the edge of the traffic light detection frame, it means that the pixel is closer to the center point of the traffic light detection frame. Based on this, when determining the position weight weight(j,i) of each pixel in the embodiments of the present application, the position coordinates (y,x) of the central pixel of the traffic light detection frame can be determined first, that is, y = h / 2, x = w / 2, where h represents the height of the traffic light detection frame and w represents the width of the traffic light detection frame. Then, calculate the distance distance between the position coordinates (j,i) of each pixel and the position coordinates (y,x) of the central pixel. The smaller the distance, the closer it is to the center point of the traffic light detection frame. Correspondingly, a larger position weight can be set. Conversely, a smaller position weight can be set. Specifically, it can be expressed in the following form:

[0091] distance_ratio = ((j - y) 2 + (i - x) 2 ) 1 / 2 / ((0.5h) 2 + (0.5w) 2 ) 1 / 2

[0092] Where distance_ratio represents the ratio of the distance between (j,i) and (y,x) to the maximum distance value.

[0093] weight(j,i) = (1.0 - distance_ratio) 2

[0094] Where the range of weight(j,i) is (0,1.0]. The closer to the central pixel point, the larger the value of weight(j,i), and the farther from the central pixel point, the smaller the value of weight(j,i).

[0095] In addition to calculating the position weight of each pixel in the way of the central pixel point, it can also be measured by the distance from each pixel to the edge of the traffic light detection frame closest to it. If the distance from a pixel to the edge of the traffic light detection frame closest to it is larger, it means that the pixel is farther from the edge of the traffic light detection frame. Then, a larger position weight can be set. Conversely, a smaller position weight can be set.

[0096] In one embodiment of the present application, determining the color integral of the filtered pixel points according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points includes: if the channel value difference of the filtered pixel points is within the range of the first interval, determining the integral of the first traffic light color according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points; if the channel value difference of the filtered pixel points is within the range of the second interval, determining the integral of the second traffic light color according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points; if the channel value difference of the filtered pixel points is within the range of the third interval, determining the integral of the third traffic light color according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points.

[0097] In the embodiment of the present application, the channel value difference corresponding to each pixel point can be first compared with multiple preset interval ranges, so as to determine which interval range the channel value difference corresponding to the pixel point falls into, and then calculate the color integral of the pixel point according to the color integral calculation method corresponding to the interval range and assign it to the corresponding traffic light color. Specifically, the following method can be used to implement it:

[0098] 1) When delta < -20:

[0099] green_score += weight(j, i) * (|delta| - 20) / 255

[0100] 2) When -20 < delta < 20:

[0101] yellow_score += weight(j, i) * (20 - |delta|) / 255

[0102] 3) When delta >= 20:

[0103] red_score += weight(j, i) * (|delta| - 20) / 255

[0104] For the first case above, it is stated that compared with the channel value of the R channel, the channel value of the G channel is larger, indicating that the color of the pixel point is more inclined to green. Then, an integration can be performed on the green light, green_score++; for the second case above, it is stated that the channel value of the R channel is relatively close to the channel value of the G channel, indicating that the color of the pixel point is more inclined to yellow. Then, an integration can be performed on the yellow light, yellow_score++; for the third case above, it is stated that compared with the channel value of the G channel, the channel value of the R channel is larger, indicating that the color of the pixel point is more inclined to red. Then, an integration can be performed on the red light, red_score+.

[0105] It should be noted that the magnitudes of the above interval thresholds are empirical values, and those skilled in the art can flexibly adjust them according to the actual situation, and no specific limitations are made here.

[0106] In an embodiment of the present application, the filtered pixel points include multiple. Determining the traffic light color corresponding to the traffic light detection box according to the color integration of the filtered pixel points includes: respectively assigning the color integrations of each filtered pixel point to the corresponding traffic light color; determining the traffic light color corresponding to the traffic light detection box according to the color integrations of each traffic light color.

[0107] The process of calculating the color integrations of each filtered pixel point in the embodiment of the present application can be regarded as a process of calculating pixel by pixel. The color integrations of all traversed pixel points will be accumulated according to different traffic light colors, so as to obtain the final color integration corresponding to each traffic light color. Finally, the final color integrations corresponding to each traffic light color are compared, and the traffic light color with the largest color integration value is taken as the final traffic light color.

[0108] In addition, it should be noted that in extremely special cases, the color integration values of two traffic light colors may be equal. At this time, it can be processed according to the priority order of red light > yellow light > green light. For example, if the color integrations of the red light and the yellow light are equal, it is recognized as the red light; if the color integrations of the yellow light and the green light are equal, it is recognized as the yellow light, so as to ensure the safety of the autonomous vehicle as much as possible.

[0109] For the convenience of understanding the embodiments of the present application, as Figure 2As shown in the figure, a schematic diagram of a traffic light color recognition process in an embodiment of the present application is provided. First, the image to be recognized is input into the target detection model to obtain the original traffic light color recognition result, including the traffic light detection box and the traffic light color type, etc. Then, it is judged whether the original traffic light color recognition result meets the credibility requirements of the preset traffic light recognition result. On the one hand, it can be judged whether the size of the traffic light detection box meets the requirements of the preset size threshold. On the other hand, it can be judged whether the saturation of each pixel point in the traffic light detection box meets the requirements of the preset saturation threshold. If the size of the traffic light detection box is smaller than the preset size threshold, or the number of pixel points with a saturation less than a certain saturation threshold in the traffic light detection box reaches a certain proportion, it is considered that the original traffic light color recognition result does not meet the credibility requirements of the preset traffic light recognition result, that is, it is necessary to re-determine the traffic light color recognition result.

[0110] When re-determining the traffic light color recognition result, first determine the brightness of each pixel point in the traffic light detection box. If the brightness of the pixel point is less than the preset brightness threshold, it means that the color recognition effect based on this pixel point will be affected. Therefore, these pixel points with a brightness less than the preset brightness threshold can be filtered out, and the remaining pixel points after filtering are processed subsequently.

[0111] After that, it is necessary to further determine the channel values of the remaining pixel points after filtering in the R channel and the G channel. Then, the difference between the channel values of the R channel and the G channel is used as one aspect to measure the color tendency of the pixel point. On the other hand, it is necessary to determine the position of the pixel point in the entire traffic light detection box, and then determine the color integral of each pixel point based on the above two aspects.

[0112] Finally, after traversing each pixel point, the total color integral corresponding to each traffic light color can be obtained, and the color with the largest color integral is used as the final traffic light color.

[0113] The traffic light color recognition method of the present application uses the size of the traffic light detection box and the saturation threshold of the pixel points to judge the credibility of the original traffic light color detection. In addition, by the brightness of the pixel points and the R / G channel difference, combined with the position weight of the pixel points, the color of the traffic light detection box is re-judged, reducing the influence of factors such as the background part and illumination in the traffic light detection box on color recognition, and improving the accuracy of traffic light color recognition of autonomous driving vehicles.

[0114] The embodiment of the present application also provides a traffic light color recognition device 300, as Figure 3As shown in the figure, a schematic structural diagram of a traffic light recognition device in an embodiment of the present application is provided. The device 300 includes: an acquisition unit 310, a filtering unit 320, a first determination unit 330, and a second determination unit 340, where:

[0115] The acquisition unit 310 is configured to acquire an original traffic light color recognition result, and the original traffic light color recognition result includes a traffic light detection frame.

[0116] The filtering unit 320 is configured to determine the brightness of the pixel points in the traffic light detection frame, and filter the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points.

[0117] The first determination unit 330 is configured to determine a color integral of the filtered pixel points according to the channel values of the filtered pixel points in a preset color channel and the positions of the filtered pixel points in the traffic light detection frame.

[0118] The second determination unit 340 is configured to determine the traffic light color corresponding to the traffic light detection frame according to the color integral of the filtered pixel points as a final traffic light color recognition result.

[0119] In an embodiment of the present application, the filtering unit 320 is specifically configured to: determine whether the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result; in the case where the original traffic light color recognition result does not meet the credibility requirement of the preset traffic light recognition result, determine the brightness of the pixel points in the traffic light detection frame, and filter the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points.

[0120] In an embodiment of the present application, the filtering unit 320 is specifically configured to: determine whether the size of the traffic light detection frame meets a preset size requirement, and determine whether the saturation of the pixel points in the traffic light detection frame meets a preset saturation requirement; if the size of the traffic light detection frame meets the preset size requirement, and the saturation of the pixel points in the traffic light detection frame meets the preset saturation requirement, it is determined that the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result; if the size of the traffic light detection frame does not meet the preset size requirement, and / or the saturation of the pixel points in the traffic light detection frame does not meet the preset saturation requirement, it is determined that the original traffic light color recognition result does not meet the credibility requirement of the preset traffic light recognition result.

[0121] In one embodiment of the present application, the filtering unit 320 is specifically configured to: determine the pixel values of the pixel points in the traffic light detection frame in the RGB color space; and determine the brightness of the pixel points in the traffic light detection frame according to the pixel values of the pixel points in the RGB color space.

[0122] In one embodiment of the present application, the preset color channels include the R channel and the G channel in the RGB color space. The first determination unit 330 is specifically configured to: determine the channel value difference of the filtered pixel points according to the channel values of the filtered pixel points in the R channel and the G channel; determine the position weight of the filtered pixel points according to the positions of the filtered pixel points in the traffic light detection frame; and determine the color integral of the filtered pixel points according to the channel value difference and the position weight of the filtered pixel points.

[0123] In one embodiment of the present application, there are multiple filtered pixel points. The first determination unit 330 is specifically configured to: determine the position of the central pixel point of the traffic light detection frame; and determine the position weight of the filtered pixel points according to the distances between the filtered pixel points and the central pixel point.

[0124] In one embodiment of the present application, the first determination unit 330 is specifically configured to: if the channel value difference of the filtered pixel points is within the first interval range, determine the integral of the first traffic light color according to the channel value difference and the position weight of the filtered pixel points; if the channel value difference of the filtered pixel points is within the second interval range, determine the integral of the second traffic light color according to the channel value difference and the position weight of the filtered pixel points; if the channel value difference of the filtered pixel points is within the third interval range, determine the integral of the third traffic light color according to the channel value difference and the position weight of the filtered pixel points.

[0125] In one embodiment of the present application, there are multiple filtered pixel points. The second determination unit 340 is specifically configured to: assign the color integrals of the respective filtered pixel points to the corresponding traffic light colors; and determine the traffic light color corresponding to the traffic light detection frame according to the color integrals of the respective traffic light colors.

[0126] It can be understood that the above traffic light color recognition device can implement each step of the traffic light color recognition method provided in the foregoing embodiment. The relevant explanations regarding the traffic light color recognition method are applicable to the traffic light color recognition device and will not be elaborated herein.

[0127] Figure 4It is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 4 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0128] The processor, network interface, and memory can be interconnected through an internal bus, and the internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 4 only a bidirectional arrow is used in

[0129] The memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0130] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a traffic light color recognition device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0131] Obtain the original traffic light color recognition result, and the original traffic light color recognition result includes a traffic light detection frame;

[0132] Determine the brightness of the pixel points in the traffic light detection frame, and filter the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points;

[0133] Determine the color integral of the filtered pixel points according to the channel values of the filtered pixel points in a preset color channel and the positions of the filtered pixel points in the traffic light detection frame;

[0134] Determine the traffic light color corresponding to the traffic light detection frame based on the color integral of the filtered pixel points as the final traffic light color recognition result.

[0135] The method performed by the traffic light color recognition device disclosed in the foregoing embodiments of the present application Figure 1 can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the foregoing method can be completed by the integrated logic circuit in the hardware of the processor or the instructions in the form of software. The foregoing processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the foregoing method.

[0136] The electronic device can also execute Figure 1 the method performed by the traffic light color recognition device in Figure 1 and implement the functions of the traffic light color recognition device in the embodiments shown. The embodiments of the present application will not be elaborated herein.

[0137] The embodiments of the present application also propose a computer-readable storage medium that stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including multiple application programs, can enable the electronic device to execute Figure 1 the method performed by the traffic light color recognition device in the embodiments shown, and specifically used to execute:

[0138] Obtain the original traffic light color recognition result, where the original traffic light color recognition result includes a traffic light detection frame;

[0139] Determine the brightness of the pixel points in the traffic light detection frame, and filter the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points;

[0140] Determine the color integral of the filtered pixel points according to the channel values of the filtered pixel points in a preset color channel and the positions of the filtered pixel points in the traffic light detection frame;

[0141] Determine the traffic light color corresponding to the traffic light detection frame according to the color integral of the filtered pixel points as the final traffic light color recognition result.

[0142] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0143] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0144] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 steps of the functions specified in one block or multiple blocks.

[0146] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0147] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0148] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0149] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the element.

[0150] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A method for traffic light color recognition, wherein, The method includes: Obtaining an original traffic light color recognition result, where the original traffic light color recognition result includes a traffic light detection frame; Determining the brightness of the pixel points in the traffic light detection frame, and filtering the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points; Determining the color integral of the filtered pixel points according to the channel values of the filtered pixel points in a preset color channel and the positions of the filtered pixel points in the traffic light detection frame; Determining the traffic light color corresponding to the traffic light detection frame according to the color integral of the filtered pixel points as the final traffic light color recognition result; The preset color channel includes the R channel and the G channel in the RGB color space, and the determining the color integral of the filtered pixel points according to the channel values of the filtered pixel points in the preset color channel and the positions of the filtered pixel points in the traffic light detection frame includes: Determining the channel value difference of the filtered pixel points according to the channel values of the filtered pixel points in the R channel and the G channel; Determining the position weight of the filtered pixel points according to the positions of the filtered pixel points in the traffic light detection frame; Determining the color integral of the filtered pixel points according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points; The determining the color integral of the filtered pixel points according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points includes: Determining the range of the interval into which the channel value difference of the filtered pixel points falls; According to the range of the interval into which the channel value difference of the filtered pixel points falls, the channel value difference of the filtered pixel points, and the position weight of the filtered pixel points, using the color integral calculation method corresponding to the interval range to determine the integral of the corresponding traffic light color.

2. The method according to claim 1, wherein The determining the brightness of the pixel points in the traffic light detection frame, and filtering the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points includes: Determining whether the original traffic light color recognition result meets the credibility requirements of the preset traffic light recognition result; In the case where the original traffic light color recognition result does not meet the credibility requirements of the preset traffic light recognition result, determining the brightness of the pixel points in the traffic light detection frame, and filtering the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points.

3. The method according to claim 2, wherein, The determining whether the original traffic light color recognition result meets the credibility requirements of the preset traffic light recognition result includes: Determining whether the size of the traffic light detection frame meets the preset size requirements, and determining whether the saturation of the pixel points in the traffic light detection frame meets the preset saturation requirements; If the size of the traffic light detection frame meets the preset size requirement, and the saturation of the pixel points in the traffic light detection frame meets the preset saturation requirement, it is determined that the original traffic light color recognition result meets the credibility requirement of the preset traffic light recognition result; If the size of the traffic light detection frame does not meet the preset size requirement, and / or the saturation of the pixel points in the traffic light detection frame does not meet the preset saturation requirement, it is determined that the original traffic light color recognition result does not meet the credibility requirement of the preset traffic light recognition result.

4. The method according to claim 1, wherein The determining the brightness of the pixel points in the traffic light detection frame includes: Determining the pixel values of the pixel points in the traffic light detection frame in the RGB color space; Determining the brightness of the pixel points in the traffic light detection frame according to the pixel values of the pixel points in the traffic light detection frame in the RGB color space.

5. The method according to claim 1, wherein, There are multiple filtered pixel points, and the determining the position weights of the filtered pixel points according to the positions of the filtered pixel points in the traffic light detection frame includes: Determining the position of the central pixel point of the traffic light detection frame; Determining the position weights of the filtered pixel points according to the distances between the respective filtered pixel points and the central pixel point.

6. The method according to claim 1, wherein The determining the color integral of the filtered pixel points according to the channel value differences of the filtered pixel points and the position weights of the filtered pixel points includes: If the channel value difference of the filtered pixel point is within the first interval range, determining the integral of the first traffic light color according to the channel value difference of the filtered pixel point and the position weight of the filtered pixel point; If the channel value difference of the filtered pixel point is within the second interval range, determining the integral of the second traffic light color according to the channel value difference of the filtered pixel point and the position weight of the filtered pixel point; If the channel value difference of the filtered pixel point is within the third interval range, determining the integral of the third traffic light color according to the channel value difference of the filtered pixel point and the position weight of the filtered pixel point.

7. The method according to claim 1, wherein There are multiple filtered pixel points, and the determining the traffic light color corresponding to the traffic light detection frame according to the color integrals of the filtered pixel points includes: Assigning the color integrals of the respective filtered pixel points to the corresponding traffic light colors; Determining the traffic light color corresponding to the traffic light detection frame according to the color integrals of the respective traffic light colors.

8. A traffic light color recognition device, wherein, The device includes: An acquisition unit, configured to acquire an original traffic light color recognition result, where the original traffic light color recognition result includes a traffic light detection frame; A filtering unit, configured to determine the brightness of the pixel points in the traffic light detection frame, and filter the pixel points in the traffic light detection frame according to the brightness of the pixel points in the traffic light detection frame to obtain filtered pixel points; A first determination unit, configured to determine the color integral of the filtered pixel points according to the channel values of the filtered pixel points in a preset color channel and the positions of the filtered pixel points in the traffic light detection frame; A second determination unit, configured to determine the traffic light color corresponding to the traffic light detection frame according to the color integration of the filtered pixel points, as the final traffic light color recognition result; The preset color channels include the R channel and the G channel in the RGB color space. Specifically, the first determination unit is configured to: Determine the channel value difference of the filtered pixel points according to the channel values of the filtered pixel points in the R channel and the G channel; Determine the position weight of the filtered pixel points according to the positions of the filtered pixel points in the traffic light detection frame; Determine the color integration of the filtered pixel points according to the channel value difference of the filtered pixel points and the position weight of the filtered pixel points; Specifically, the first determination unit is configured to: Determine the range of the interval into which the channel value difference of the filtered pixel points falls; According to the range of the interval into which the channel value difference of the filtered pixel points falls, the channel value difference of the filtered pixel points, and the position weight of the filtered pixel points, use the color integration calculation method corresponding to the interval range to determine the integration of the corresponding traffic light color.

9. An electronic device, comprising: A processor; And A memory arranged to store computer-executable instructions, which when executed cause the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, the computer-readable storage medium stores one or more programs, which when executed by an electronic device including a plurality of application programs, cause the electronic device to execute the method according to any one of claims 1 to 7.

Images