Traffic light state recognition method and related device

Abstract

The application aims to provide a traffic light state recognition method and related equipment, which combines traffic signal light color change rules to correct color recognition results and improve recognition accuracy. A traffic light state recognition method comprises the following steps: determining multiple images of a traffic light; performing color recognition on the traffic light in each image to obtain a color signal sequence of the traffic light; detecting whether a target color sequence exists in a subsequence, the subsequence being a part of the color signal sequence, and the color signals in the target color sequence meeting a color signal change rule; and if the subsequence contains the target color sequence, correcting the color signals of the target color sequence.

Description

Technical Field

[0001] This application relates to the field of identification, and more particularly to a method and related equipment for identifying the status of traffic lights. Background Technology

[0002] Traffic lights are typically located at intersections or other locations where traffic control is required. The presence of traffic lights plays a vital role in managing traffic flow, improving road capacity, and reducing traffic accidents.

[0003] With the rapid development of IoT and AI technologies in recent years, intelligent transportation is gradually replacing traditional traffic management systems, and traffic signal recognition is a crucial component. Currently, computer vision-based traffic signal recognition has become a viable solution, with mainstream approaches primarily based on deep learning-based object detection algorithms. This method trains a neural network model using a large amount of traffic monitoring video footage to directly predict traffic light colors from images. However, in practical applications, the accuracy of this method is difficult to guarantee due to extreme lighting conditions such as strong light and darkness, as well as abnormal conditions such as occlusion and dirt. Summary of the Invention

[0004] The purpose of this application is to provide a traffic light status recognition method and related equipment, which combines the color change pattern of traffic lights to correct the color recognition results and improve the recognition accuracy.

[0005] The aforementioned and other objectives will be achieved through the features described in the independent claims. Further implementations are illustrated in the dependent claims, the specification, and the drawings.

[0006] In a first aspect, embodiments of this disclosure provide a traffic light state recognition method, applied to recognizing the color of any individual light within a traffic light, the method comprising:

[0007] Identify multiple images of a traffic light;

[0008] Perform color recognition on the traffic lights in each of the images to obtain the color signal sequence of the traffic lights;

[0009] Detect whether a target color sequence exists in a sub-sequence, wherein the sub-sequence is a part of the color signal sequence, and the color signal in the target color sequence has a color signal variation pattern;

[0010] If the target color sequence exists in the subsequence, then the color signal of the target color sequence is calibrated.

[0011] In one possible implementation, in the traffic light status recognition method provided in this disclosure, the color signals in the color signal sequence are arranged according to the acquisition time of the image corresponding to each color signal;

[0012] The detection of whether the target color sequence exists in the sub-sequence includes:

[0013] The first detection sliding window is used to traverse the color signal sequence to detect the start and end times of the target color sequence;

[0014] If the start time and the end time are detected, and all color signals between the start time and the end time satisfy a preset condition, then it is determined that the color signals in the subsequence exist in the target color sequence.

[0015] In one possible implementation, the traffic light state recognition method provided in this disclosure includes one or more of the following preset conditions:

[0016] In all color signals between the start time and the end time, the proportion of each color signal is less than or equal to a proportion threshold.

[0017] The similarity between the color signal between the start time and the end time and the color signal in the most recently determined target color sequence is greater than or equal to the similarity threshold.

[0018] In one possible implementation, in the traffic light state recognition method provided in this embodiment, if two color signals are present for the first time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the starting time.

[0019] In one possible implementation, in the traffic light state recognition method provided in this embodiment, if the two color signals are present for the second time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the end time.

[0020] In one possible implementation, the traffic light state recognition method provided in this disclosure includes, as a step, calibrating the color signal of the target color sequence, comprising:

[0021] The color signals within the target color sequence are traversed using a second detection sliding window, and the switching frequency of the color signals within the second detection sliding window is detected.

[0022] If the number of color signal switching times in the second detection window is greater than or equal to the threshold, the color signal of the incorrectly identified image in the second detection window is changed to the color signal with the largest total number in the second detection window. The images corresponding to the color signals with the smallest total number in the second detection window are all incorrectly identified images.

[0023] In one possible implementation, the traffic light state recognition method provided in this disclosure, after calibrating the color signal of the target color sequence, further includes:

[0024] The color signal detection accuracy within the target color sequence is determined based on the total number of color signals within the target color sequence and the total number of incorrectly identified images.

[0025] If the color signal detection accuracy within the target color sequence is greater than or equal to the accuracy threshold, then the color signal of the corresponding image in the color signal sequence is corrected based on the calibrated target color sequence.

[0026] In one possible implementation, the traffic light state recognition method provided in this disclosure further includes:

[0027] Send the corrected color signal sequence.

[0028] Secondly, an electronic device according to embodiments of the present disclosure includes:

[0029] An image acquisition module is used to determine multiple images of a traffic light;

[0030] Processing module, used for:

[0031] Perform color recognition on the traffic lights in each of the images to obtain the color signal sequence of the traffic lights;

[0032] Detect whether a target color sequence exists in a sub-sequence, wherein the sub-sequence is a part of the color signal sequence, and the color signal in the target color sequence has a color signal variation pattern;

[0033] If the target color sequence exists in the subsequence, then the color signal of the target color sequence is calibrated.

[0034] In one possible implementation, in the electronic device provided by this disclosure, the color signals in the color signal sequence are arranged according to the acquisition time of the image corresponding to each color signal; the processing module is specifically used for:

[0035] The first detection sliding window is used to traverse the color signal sequence to detect the start and end times of the target color sequence;

[0036] If the start time and the end time are detected, and all color signals between the start time and the end time satisfy a preset condition, then it is determined that the color signals in the subsequence exist in the target color sequence.

[0037] In one possible implementation, the preset conditions include one or more of the following:

[0038] In all color signals between the start time and the end time, the proportion of each color signal is less than or equal to a proportion threshold.

[0039] The similarity between the color signal between the start time and the end time and the color signal in the most recently determined target color sequence is greater than or equal to the similarity threshold.

[0040] In one possible implementation, in the electronic device provided by this disclosure, if two color signals are present for the first time in the first detection sliding window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection sliding window is the starting time.

[0041] In one possible implementation, in the electronic device provided by this disclosure, if the two color signals are present for the second time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the end time.

[0042] In one possible implementation, in the electronic device provided by this disclosure, the processing module is specifically used for:

[0043] The color signals within the target color sequence are traversed using a second detection sliding window, and the switching frequency of the color signals within the second detection sliding window is detected.

[0044] If the number of color signal switching times in the second detection window is greater than or equal to the threshold, the color signal of the incorrectly identified image in the second detection window is changed to the color signal with the largest total number in the second detection window. The images corresponding to the color signals with the smallest total number in the second detection window are all incorrectly identified images.

[0045] In one possible implementation, in the electronic device provided by this disclosure, the processing module is further configured to:

[0046] The color signal detection accuracy within the target color sequence is determined based on the total number of color signals within the target color sequence and the total number of incorrectly identified images.

[0047] If the color signal detection accuracy within the target color sequence is greater than or equal to the accuracy threshold, then the color signal of the corresponding image in the color signal sequence is corrected based on the calibrated target color sequence.

[0048] Thirdly, embodiments of this disclosure provide a computer-readable storage medium storing computer instructions that, when executed on an electronic device, cause the electronic device to perform a traffic light state recognition method as described in the first aspect and any possible implementation thereof.

[0049] Fourthly, embodiments of this application provide a computer program product comprising a computer program that, when executed by an electronic device, implements the steps of the traffic light state recognition method described above.

[0050] The beneficial effects of the embodiments disclosed herein are as follows:

[0051] In this embodiment, color signals of small lights in each image are identified using color recognition to obtain a color signal sequence. The subsequence of the color signal sequence is then judged to contain one or more color change cycles by determining if a target color sequence conforms to the color signal change pattern. Utilizing the characteristic that the same color signal is continuous within a color change cycle, the color signals in the target color sequence are calibrated to assist in correcting or refining the recognition results obtained through color recognition. This improves the accuracy of traffic light status recognition and can be applied to scenarios where the traffic light color change cycle is variable.

[0052] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this disclosure;

[0055] Figure 2 A flowchart illustrating the traffic light status recognition method provided in this embodiment of the disclosure;

[0056] Figure 3a and Figure 3b A schematic diagram illustrating the color change cycle of a small light;

[0057] Figure 4A flowchart illustrating the traffic light status recognition method provided in this embodiment of the disclosure;

[0058] Figure 5 This is a schematic diagram of the first detection sliding window operation process;

[0059] Figure 6 A schematic diagram illustrating the working process of the second detection sliding window;

[0060] Figure 7 This is a schematic diagram illustrating another working process of the second detection sliding window;

[0061] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0062] Figure 9 This is a schematic diagram of the structure of a traffic light recognition device provided in an embodiment of the present disclosure. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this disclosure. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0064] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this disclosure. Traffic lights are typically installed at intersections. Identifying the colors of traffic lights helps in timely determining whether they are malfunctioning, or assists in judging whether traffic behavior is consistent with the color indications of the traffic lights. This disclosure provides an electronic device capable of executing the traffic light status recognition method provided in this disclosure. Please see also... Figure 1 Electronic devices can be placed near traffic lights and can capture images of the traffic lights in real time. In this application, traffic lights may include light groups. Each light group may include multiple individual lights. A traffic light may include one or more individual lights. The traffic light status recognition method provided in this application can be used to identify any individual light within a traffic light. In this disclosure, "traffic light" can refer to an individual light.

[0065] Figure 2 An exemplary embodiment illustrates a method for recognizing the state of a traffic light. The method may include the following steps:

[0066] S101, determine multiple images of the traffic light.

[0067] The electronic device can identify, capture, or acquire multiple images of traffic lights. Optionally, the electronic device can capture multiple images of traffic lights, thereby achieving the acquisition of multiple traffic light images. Alternatively, the electronic device can capture multiple images of traffic lights individually. In some application scenarios, the electronic device can acquire images in real time or periodically. Alternatively, the electronic device can respond to commands from a management terminal, such as a command to start recognizing traffic lights, and execute the traffic light state recognition method provided in this disclosure embodiment.

[0068] S102, perform color recognition on the traffic lights in each of the images to obtain the color signal sequence of the traffic lights.

[0069] Electronic devices can detect the color of traffic lights in each image, identifying the color signal of the traffic light in each image. In practical applications, the color signals of traffic lights can be divided into three categories: red, yellow, and green. Detecting the color signal of a traffic light (color) means detecting whether the traffic light is red, yellow, or green.

[0070] In this embodiment of the disclosure, the color signal sequence includes the color signal of the traffic light in each image. For example, the color signal sequence can be sorted according to the acquisition time of each image, and the elements in each order can represent the color of the traffic light. For instance, the detection result sequence can be "red-red…red-green-green…green-yellow-yellow…yellow-red-red…red-red".

[0071] S103, Detect whether the target color sequence exists in the subsequence. If yes, proceed to step S104; otherwise, proceed to step S105.

[0072] In practical applications, traffic light colors change periodically. Optionally, the traffic light color change can be "red—green—yellow (flashing yellow)—red—green—yellow…", in which case the color signal within one color change cycle can be "red—green—yellow". Alternatively, the traffic light color change can be "green—yellow—red—green—yellow—red…", in which case the color signal within one color change cycle can be "green—yellow—red". Or, the traffic light color change can be "yellow—red—green—yellow—red—green…", in which case the color signal within one color change cycle can be "yellow—red—green".

[0073] In this embodiment, the traffic light signals within a color change cycle exhibit a pattern of color signal variation; for example, the number of each color signal is less than or equal to a preset threshold. Furthermore, within a single color change cycle, the same color signal is continuous. Typically, the duration of each color is configured during traffic light operation. The durations of different colors may vary. It should be noted that the traffic light color change cycle is not fixed. In practical applications, the operation of traffic lights is controlled by relevant departments. The traffic light color change cycle may be adjusted. The traffic light state recognition method provided in this embodiment remains applicable in such cases.

[0074] The electronic device can acquire subsequences from the color signal sequence. Optionally, the subsequence length can be pre-configured. The subsequence can include the color signals of the traffic light in a preset number of images, arranged according to the acquisition time of each image.

[0075] Electronic devices can extract subsequences from color signal sequences. By determining whether one or more target color sequences exist within the subsequence, it can be determined whether the subsequence includes one or more potential color change periods. Utilizing the characteristic that the same color signal is continuous among all color signals within a color change period, the color signals within the potential color change period are calibrated. In some examples, the color signals in the target color sequence exhibit a color signal change pattern, and the proportion of each color signal in the target color sequence is less than or equal to a certain percentage threshold.

[0076] S104, The color signal of the target color sequence is calibrated.

[0077] like Figure 3a As shown, within a color change cycle, the same color signal is continuous. The number of times two colors switch is counted as one, the number of times a traffic light changes from green to yellow is counted as one, and the number of times a traffic light changes from yellow to red is counted as one. Please combine this with... Figure 3b Within a target color sequence, if the number of times two colors switch exceeds a preset threshold, it can be considered that there is an erroneous color signal within the target color sequence. An exemplary preset threshold can be configured as 2. The electronic device can correct the erroneous color signal within the target color sequence, thereby correcting the corresponding erroneous color signal in the color signal sequence identified in step S102.

[0078] S105, Reacquire the subsequence.

[0079] In the case where the target color sequence does not exist in the current subsequence, the electronic device can re-obtain a subsequence from the color signal sequence. The acquisition time of the first image in the re-obtained subsequence is after the acquisition time of the first image in the previously obtained subsequence. In other words, the re-obtained subsequence and the previously obtained subsequence can partially overlap.

[0080] Figure 4 According to an exemplary embodiment, a method for recognizing the state of a traffic light is provided, which may include the following steps:

[0081] Step S201, acquiring multiple images of the traffic light.

[0082] Step S202, performing target detection on each image, detecting the color signal of the traffic light in each image, and generating a color signal sequence of the traffic light.

[0083] In a possible implementation manner, the electronic device can use any existing method for recognizing the state of a traffic light to recognize the color of the traffic light.

[0084] In another possible implementation manner, the color recognition of traffic lights belongs to a typical small target detection task, and directly performing detection may not have good results. In the embodiments of the present disclosure, for any image of a traffic signal light, the electronic device can perform overlapping segmentation on the image of the traffic signal light to obtain multiple slices. The effect of enlarging the size can be achieved. The electronic device can sequentially detect each slice. Traffic signal lights generally appear in the form of a light group, and each light group contains multiple traffic lights. For the convenience of description, the set of predicted bounding boxes of the signal light group obtained here can be represented as G = {G

[0085] |0 ≤ i < M, i ∈ Z}, where M represents the number of light groups and i represents the light group number. The set of predicted bounding boxes of traffic lights can be represented as L = {L ij |0 ≤ i < M, 0 ≤ j < N(G i ), i, j ∈ Z}, where N(G i ) represents the number of traffic lights included in the所属灯组.

[0085] The electronic device can sequentially recognize all traffic light bounding boxes L ij in each light group. If N(G i ) is 0, then directly recognize the light group bounding box G i . The recognition results of traffic lights include red lights, yellow lights, and green lights. Optionally, the recognition results of traffic lights can include red lights, yellow lights, green lights, and lights out (not on). The electronic device can record the recognition results of traffic lights L ij in each image, so as to generate a color signal sequence of traffic lights L ij .

[0086] Step S203: Obtain a subsequence from the color signal sequence.

[0087] Electronic devices can access traffic lights L ij From the color signal sequence, obtain the subsequence R. ij Optionally, the subsequence can be part or all of the color signal sequence.

[0088] Step S204: Use the first detection sliding window to traverse the subsequence and detect whether there is a start time and end time of the candidate sequence. If yes, proceed to step S205. If no, re-acquire the subsequence.

[0089] like Figure 5 As shown, the electronic device can use a first detection sliding window W1 of length K to traverse the subsequence R. ij If two color signals are present for the first time in the first detection window W1, and the number of the two color signals is the same, the image acquisition time at the midpoint of the first detection window W1 is the start time. If the two color signals are present for the second time in the first detection window W1, and the number of the two color signals is the same, the image acquisition time at the midpoint of the first detection window W1 is the end time.

[0090] For example, the length of the first detection sliding window W1 can be K seconds. The first detection sliding window W1 initially contains only two color signals, and each color signal is continuous with a duration of K seconds. Seconds. At this time, the image acquisition time at the midpoint of the first detection sliding window W1 is the starting time t1. The first detection sliding window W1 contains only the aforementioned two color signals for the second time, and each color signal is continuous with a duration of [missing value]. Seconds. At this time, the image acquisition time at the midpoint of the first detection sliding window W1 is the end time t2. The signal sequence between the start time t1 and the end time t2 can be recorded as a candidate sequence.

[0091] Step S205: Determine whether the candidate sequence meets the preset conditions. If yes, proceed to step S206. If no, obtain the subsequence again.

[0092] Electronic devices determine whether a candidate sequence meets preset conditions, which is also a determination of whether the candidate sequence is the target color sequence. If the candidate sequence meets the preset conditions, then the candidate sequence is the target color sequence and represents a normal color change cycle. Conversely, if the candidate sequence does not meet the preset conditions, then the candidate sequence is not the target color sequence and does not represent a normal color change cycle.

[0093] In some examples, the preset conditions include one or more of the following:

[0094] All color signals between the start time and the end time, that is, in the candidate sequence, the proportion of each color signal is less than or equal to the proportion threshold.

[0095] The similarity between the color signal (i.e., the candidate sequence) between the start time and the end time and the color signal in the most recently determined target color sequence is greater than or equal to the similarity threshold.

[0096] In one possible design, if the proportion of each color signal in all color signals in the candidate sequence is less than or equal to a proportion threshold, and the similarity between the color signals in the candidate sequence and the color signals of the most recently determined target color sequence is greater than or equal to a similarity threshold, then the candidate sequence can be considered as the target color sequence and is also a normal cycle.

[0097] If the proportion of any color signal in the candidate sequence is greater than a certain threshold, then the target period is not a color change period. Alternatively, if the similarity between the color signals in the candidate sequence and the color signals in the most recently determined target color sequence is less than a certain threshold, then the candidate sequence is not the target color sequence.

[0098] In one possible design, the electronic device can determine the similarity between the color signals in the target period and the color signals in the most recently determined color change period after determining that the proportion of each color signal in all color signals within the target period is less than or equal to a proportion threshold.

[0099] Step S206: Use a second detection sliding window to traverse the color signals of the target color sequence and detect the color signal switching frequency within the second detection sliding window.

[0100] like Figure 6 As shown, the electronic device can use a second detection window W2 to traverse the color signal within the color change period. The length of the second detection window W2 is less than that of the first detection window W1. Optionally, the length of the second detection window W2 can be half that of the first detection window W1.

[0101] When all color signals within a target color sequence are accurately identified, at most one color switch will occur in the color signal of the second detection window W2. Therefore, by determining the number of color switches in the color signal within the second detection window W2, it can be determined whether there are any identification errors in the color signals within the current target color sequence.

[0102] Step S207: Check whether the number of color signal switching times in the second detection window is greater than or equal to the number threshold. If yes, proceed to step S208. If no, re-acquire the sub-sequence.

[0103] If the number of switching events in the color signal of the second detection window W2 is less than the threshold number, it can be determined that all color signals within the second detection window W2 are accurately identified. If the number of switching events in the color signal of the second detection window W2 is greater than or equal to the threshold number, it can be determined that there are identification errors in the color signals within the second detection window W2.

[0104] Step S208: The images corresponding to the color signals with the fewest total number in the second detection window are all identified as incorrectly identified images, and the color signals of the incorrectly identified images in the target color sequence are changed to the color signals with the most total number in the second detection window.

[0105] The electronic device can count the total number of color signals within the second detection window W2. For ease of explanation, examples will be provided, such as... Figure 7 As shown, among the two colors contained in the second detection window W2, the color signal with the largest total quantity is green, and the color signal with the smallest total quantity is yellow. The images corresponding to the elements with yellow color signals within the second detection window W2 are all incorrectly identified images. The electronic device changes all elements with yellow color signals within the second detection window W2 to green. That is, the electronic device changes the color signal of the incorrectly identified images to green within the second detection window.

[0106] In one possible implementation, after executing step S208, the electronic device can correct the color signals within the current target color sequence. The electronic device can correct the color signals of traffic lights in the corresponding images within the color signal sequence based on the color signals of traffic lights in each image within the current target color sequence.

[0107] In another possible implementation, the electronic device may also perform the operation in step S209.

[0108] Step S209: Determine the color signal detection accuracy within the target color sequence based on the total number of color signals within the target color sequence and the total number of incorrectly identified signals.

[0109] Electronic devices can statistically analyze the detection accuracy of color signals within the current target color sequence. Where N(·) represents the quantity, Diff c T represents the total number of misidentified images within the current target color sequence. c This indicates the total number of color signals within the current target color sequence.

[0110] Step S210: If the color signal detection accuracy in the target color sequence is greater than or equal to the accuracy threshold, the color signal of the corresponding image in the color signal sequence is corrected based on the color signal of the incorrectly identified image in the target color sequence after the color signal is changed.

[0111] If the color signal detection accuracy within the target color sequence is greater than or equal to an accuracy threshold, the electronic device can use the corrected target color sequence to correct the color signals of the corresponding images in the color signal sequence. The electronic device can then send the corrected color signal sequence to a server or management terminal. It should be understood that if there are no incorrectly identified images within each color change period, the electronic device can directly send the color signal sequence.

[0112] In practical applications, relevant departments can adjust the traffic light cycle based on traffic flow. If the color signal detection accuracy within the color change cycle is less than the accuracy threshold, it indicates that the current color change cycle may be adjusted. The electronic device can reacquire a subsequence of the color signal sequence and re-detect the start and end times of the target color sequence. Therefore, the traffic light state recognition method provided in this embodiment can operate stably even when the traffic light cycle changes.

[0113] Based on the same inventive concept, this disclosure provides an electronic device. Figure 8 An exemplary schematic diagram of an electronic device is shown. The electronic device may include an image acquisition module and a processing module. The electronic device can be used to identify the color of any traffic light.

[0114] The image acquisition module may have the function or capability to acquire images. The image acquisition module may include, but is not limited to, a camera. The image acquisition module can provide the acquired images to the processing module. The image acquisition module can be used to acquire multiple images of the traffic light.

[0115] The processing module may have data processing functions or capabilities. The processing module may include one or more chips. Optionally, the processing module may include one or more of a neural network processing unit (NPU) and a central processing unit (CPU). The processing module can be used for:

[0116] Target detection is performed on each of the images to generate a detection result sequence, the detection result sequence including the color signal of the traffic light in each of the images;

[0117] The color signal sequence of the traffic light is obtained from the detection result sequence. The color signal sequence includes the color signals of the traffic light in a preset number of images and is arranged according to the acquisition time of each image. The presence of a color change cycle in the color signal sequence is detected. If a color change cycle exists in the color signal sequence, the color signals in each color change cycle are calibrated to correct the detection result sequence.

[0118] In one possible implementation, when the processing module detects whether there is a color change period in the color signal sequence, it may perform the following operations:

[0119] The first detection sliding window is used to traverse the color signal sequence to detect the start and end times of the color change cycle;

[0120] If the start time and the end time are detected, and the proportion of each color signal in all color signals between the start time and the end time is less than or equal to a proportion threshold, then the similarity between the color signals between the start time and the end time and the color signals in the most recently determined color change cycle is determined.

[0121] If the similarity is greater than or equal to the similarity threshold, it is determined that the color signal sequence has a color change period, wherein all color signals between the start time and the end time are all color signals within a color change period.

[0122] In one possible implementation, if two color signals are present for the first time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the start time; if the two color signals are present for the second time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the end time.

[0123] In one possible implementation, when the processing module calibrates the color signal within each color change cycle, it may perform the following operations:

[0124] The color signal within the color change period is traversed using a second detection sliding window to detect the color signal switching frequency within the second detection sliding window.

[0125] If the number of color signal switching times in the second detection window is greater than or equal to the number threshold, the images corresponding to the color signals with the fewest total number in the second detection window are identified as incorrectly identified images, and the color signals of the incorrectly identified images are changed to the color signals with the most total number in the second detection window.

[0126] In one possible implementation, after calibrating the traffic light color signal for each color change cycle, the processing module may perform the following operations:

[0127] The color signal detection accuracy within the color change period is determined based on the total number of color signals within the color change period and the total number of incorrectly identified images.

[0128] If the color signal detection accuracy within the color change period is greater than or equal to the accuracy threshold, the color signal of the corresponding image in the detection result sequence is corrected based on the color signal of the incorrectly identified image after the color change period.

[0129] In one possible implementation, the processing module can also send a corrected sequence of detection results.

[0130] Based on the same inventive concept, such as Figure 9 As shown, this embodiment of the present disclosure provides a traffic light recognition device, which may include an image acquisition unit, a traffic light recognition unit, a color change period detection unit, and an analysis and processing unit.

[0131] An image acquisition unit is used to acquire multiple images of the traffic light.

[0132] A traffic light recognition unit is used to perform color recognition on the traffic lights in each of the images and obtain the color signal sequence of the traffic lights;

[0133] A color change period detection unit is used to detect whether a target color sequence exists in a sub-sequence, wherein the sub-sequence is a part of the color signal sequence, and the color signal in the target color sequence conforms to the color signal change rule.

[0134] An analysis and processing unit is used to calibrate the color signal of the target color sequence if the sub-sequence contains the target color sequence.

[0135] In one possible design, the color signals in the color signal sequence are arranged according to the acquisition time of the image corresponding to each color signal; the color change period detection unit is specifically used for:

[0136] The first detection sliding window is used to traverse the color signal sequence to detect the start and end times of the target color sequence;

[0137] If the start time and the end time are detected, and all color signals between the start time and the end time satisfy a preset condition, then it is determined that the color signals in the sub-sequence exist in the target color sequence. The preset condition includes one or more of the following:

[0138] In all color signals between the start time and the end time, the proportion of each color signal is less than or equal to a proportion threshold.

[0139] The similarity between the color signal between the start time and the end time and the color signal in the most recently determined target color sequence is greater than or equal to the similarity threshold.

[0140] In some examples, if two color signals are present for the first time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the starting time.

[0141] If the two color signals are present for the second time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the end time.

[0142] In one possible design, the analysis and processing unit is specifically used for:

[0143] The color signals within the target color sequence are traversed using a second detection sliding window, and the switching frequency of the color signals within the second detection sliding window is detected.

[0144] If the number of color signal switching times in the second detection window is greater than or equal to the threshold, the color signal of the incorrectly identified image in the second detection window is changed to the color signal with the largest total number in the second detection window. The images corresponding to the color signals with the smallest total number in the second detection window are all incorrectly identified images.

[0145] In one possible design, the analysis and processing unit is also used for:

[0146] After calibrating the color signal of the target color sequence, the method further includes:

[0147] The color signal detection accuracy within the target color sequence is determined based on the total number of color signals within the target color sequence and the total number of incorrectly identified images.

[0148] If the color signal detection accuracy within the target color sequence is greater than or equal to the accuracy threshold, then the color signal of the corresponding image in the color signal sequence is corrected based on the calibrated target color sequence.

[0149] In one possible design, the analysis and processing unit can send the corrected color signal sequence to the control unit, allowing the control unit to utilize the corrected color signal sequence. For example, the control unit can perform comprehensive analysis and processing based on the corrected color signal sequence and road vehicle driving data.

[0150] Based on the same inventive concept, this disclosure provides a computer storage medium comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the traffic light state recognition methods discussed above. Since the principle by which the above-described computer storage medium solves the problem is similar to that of the traffic light state recognition method, the implementation of the above-described computer storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0151] In specific implementation, computer storage media can include: Universal Serial Bus Flash Drive (USB), portable hard drive, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0152] Based on the same inventive concept, this disclosure also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the traffic light state recognition methods discussed above. Since the principle by which the above-described computer program product solves the problem is similar to that of the traffic light state recognition method, the implementation of the above-described computer program product can be referred to the implementation of the method, and repeated details will not be described again.

[0153] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for recognizing the state of a traffic light, characterized in that, The method includes: Identify multiple images of a traffic light; Perform color recognition on the traffic lights in each of the images to obtain the color signal sequence of the traffic lights; Detect whether a target color sequence exists in a sub-sequence, wherein the sub-sequence is a part of the color signal sequence, and the color signal in the target color sequence conforms to the color signal change pattern; If the target color sequence exists in the sub-sequence, then the color signal of the target color sequence is calibrated; The calibration of the color signal of the target color sequence includes: The color signals within the target color sequence are traversed using a second detection sliding window, and the switching frequency of the color signals within the second detection sliding window is detected. If the number of color signal switching times in the second detection window is greater than or equal to the number threshold, the color signal of the incorrectly identified image in the second detection window is changed to the color signal with the largest total number in the second detection window, wherein the images corresponding to the color signals with the smallest total number in the second detection window are all the incorrectly identified images; After calibrating the color signal of the target color sequence, the method further includes: The color signal detection accuracy within the target color sequence is determined based on the total number of color signals within the target color sequence and the total number of incorrectly identified images. If the color signal detection accuracy within the target color sequence is greater than or equal to the accuracy threshold, then the color signal of the corresponding image in the color signal sequence is corrected based on the calibrated target color sequence. The method further includes: Send the corrected color signal sequence.

2. The method as described in claim 1, characterized in that, The color signals in the color signal sequence are arranged according to the acquisition time of the image corresponding to each color signal; The detection of whether the target color sequence exists in the sub-sequence includes: The first detection sliding window is used to traverse the color signal sequence to detect the start and end times of the target color sequence; If the start time and the end time are detected, and all color signals between the start time and the end time satisfy a preset condition, then it is determined that the color signals in the subsequence exist in the target color sequence.

3. The method as described in claim 2, characterized in that, The preset conditions include one or more of the following: In all color signals between the start time and the end time, the proportion of each color signal is less than or equal to a proportion threshold. The similarity between the color signal between the start time and the end time and the color signal in the most recently determined target color sequence is greater than or equal to the similarity threshold.

4. The method as described in claim 2 or 3, characterized in that, If two color signals are present for the first time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the starting time.

5. The method as described in claim 4, characterized in that, If the two color signals are present for the second time in the first detection window, and the number of the two color signals is the same, the acquisition time of the image at the midpoint of the first detection window is the end time.

6. An electronic device, characterized in that, include: An image acquisition module is used to determine multiple images of a traffic light; Processing module, used for: Perform color recognition on the traffic lights in each of the images to obtain the color signal sequence of the traffic lights; Detect whether a target color sequence exists in a sub-sequence, wherein the sub-sequence is a part of the color signal sequence, and the color signal in the target color sequence has a color signal variation pattern; If the target color sequence exists in the sub-sequence, then the color signal of the target color sequence is calibrated; Wherein, the calibration of the color signal of the target color sequence includes: The color signals within the target color sequence are traversed using a second detection sliding window, and the switching frequency of the color signals within the second detection sliding window is detected. If the number of color signal switching times in the second detection window is greater than or equal to the number threshold, the color signal of the incorrectly identified image in the second detection window is changed to the color signal with the largest total number in the second detection window, wherein the images corresponding to the color signals with the smallest total number in the second detection window are all the incorrectly identified images; After the color signal of the target color sequence is calibrated, the processing module is further configured to: The color signal detection accuracy within the target color sequence is determined based on the total number of color signals within the target color sequence and the total number of incorrectly identified images. If the color signal detection accuracy within the target color sequence is greater than or equal to the accuracy threshold, then the color signal of the corresponding image in the color signal sequence is corrected based on the calibrated target color sequence. Send the corrected color signal sequence.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on an electronic device, cause the electronic device to perform any of the traffic light state recognition methods as described in claims 1-5.

Images