Traffic light state detection method, device and equipment, and computer program product

Abstract

The invention discloses a traffic light state detection method, apparatus and device, and a computer program product. The traffic light state detection method comprises the steps of obtaining an image of a traffic scene at a current moment; traffic light detection is carried out on the image of the traffic scene at the current moment, a traffic light detection result at the current moment is obtained, and the traffic light detection result at the current moment comprises an initial traffic light color detection result at the current moment; and determining an actual traffic light state at the current moment by using a predefined finite state machine and a preset smoothing mechanism according to the initial traffic light color detection result at the current moment. According to the traffic light state detection method provided by the embodiment of the invention, the traffic light state is identified by using the finite-state machine and the smoothing mechanism of the multi-frame data, which is different from a scheme which only depends on single-frame detection or a single decision model, so that the accuracy and robustness of traffic light state identification are remarkably improved; and identification challenges in a complex traffic environment can be effectively handled.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a traffic light status detection method, device and equipment, and computer program product. Background Technology

[0002] With the rapid advancement of autonomous driving technology, the requirements for vehicles' perception of their surroundings during autonomous driving are becoming increasingly stringent. Against this backdrop, accurate detection and recognition of traffic lights have become crucial factors in ensuring driving safety and improving the driving experience. Especially in the complex traffic environments of cities, the status of traffic lights directly impacts vehicle driving decisions, such as whether to stop or continue driving. Therefore, developing efficient and accurate traffic light status detection systems is of paramount importance.

[0003] However, existing traffic light detection technologies face numerous challenges in practical applications. First, relying on single-frame images for position detection and state recognition often leads to false positives or false negatives. Insufficient lighting or partial obstruction of traffic lights can cause the system to misidentify the traffic light's state, resulting in unnecessary emergency braking or delayed starts, seriously impacting driving safety. Second, during traffic light state changes, brief false positives (e.g., due to flashing traffic lights or environmental interference) make it difficult for the system to accurately track the true state of the traffic light, leading to inconsistent vehicle responses. This not only reduces the driving experience but may also cause traffic accidents.

[0004] Furthermore, most existing traffic light detection algorithms are based on single feature extraction and classification models. While these models perform well under ideal conditions, their robustness and adaptability are significantly insufficient in situations with large variations in lighting conditions and complex traffic environments. Especially in some extreme cases, such as traffic light malfunctions or abnormal flashing patterns, existing systems often fail to make correct judgments, leading to inaccurate identification results. For example, some solutions use Markov decision processes for state determination, but this method relies heavily on the prior probability of state transitions and has poor adaptability in atypical scenarios; other solutions only learn the state through simple statistical analysis of historical frames, lacking explicit modeling of the traffic light state transition logic, making it difficult to handle special states such as flashing.

[0005] Therefore, improving the stability and accuracy of traffic light status detection systems in complex environments has become an urgent technical problem to be solved. Summary of the Invention

[0006] This application provides a traffic light status detection method, apparatus, device, and computer program product to improve the stability and accuracy of traffic light status detection in complex environments.

[0007] The embodiments of this application adopt the following technical solutions:

[0008] In a first aspect, embodiments of this application provide a traffic light status detection method, the traffic light status detection method comprising:

[0009] Obtain an image of the traffic scene at the current moment;

[0010] Traffic light detection is performed on the image of the traffic scene at the current moment to obtain the traffic light detection result at the current moment, which includes the initial traffic light color detection result at the current moment;

[0011] Based on the initial traffic light color detection results at the current moment, the actual traffic light state at the current moment is determined using a predefined finite state machine and a preset smoothing mechanism.

[0012] Optionally, determining the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment, using a predefined finite state machine and a preset smoothing mechanism, includes:

[0013] The current state of a traffic light is determined using a predefined state queue of a finite state machine, where the current state is derived from the actual traffic light state of the previous moment recorded in the state queue.

[0014] Based on the initial traffic light color detection result at the current moment and the current state of the traffic light, the actual traffic light state at the current moment is determined using the predefined finite state machine's state transition rules and the preset smoothing mechanism. The actual traffic light state at the current moment is then stored in the state queue as the starting point for the state transition of the predefined finite state machine at the next moment.

[0015] Optionally, determining the actual traffic light state at the current moment based on the initial traffic light color detection result and the current state of the traffic light, using the predefined state transition rules of the finite state machine and the preset smoothing mechanism, includes:

[0016] Based on the initial traffic light color detection result at the current moment, the smoothed traffic light color detection result is determined using the preset smoothing mechanism;

[0017] Based on the smoothed traffic light color detection result and the current state of the traffic light, determine whether the current state of the traffic light triggers the state transition rule;

[0018] If triggered, the traffic light state after the transition is determined according to the triggered state transition rules, and this state is taken as the actual traffic light state at the current moment.

[0019] Optionally, determining the smoothed traffic light color detection result using the preset smoothing mechanism based on the initial traffic light color detection result at the current moment includes:

[0020] Based on the initial traffic light color detection result at the current moment, the preset smoothing mechanism is used to determine whether the initial traffic light color detection result at the current moment remains consistent across multiple consecutive frames.

[0021] If so, the initial traffic light color detection result at the current moment is used as the smoothed traffic light color detection result.

[0022] Optionally, determining the actual traffic light state at the current moment based on the initial traffic light color detection result and the current state of the traffic light, using the predefined state transition rules of the finite state machine and the preset smoothing mechanism, includes:

[0023] If the smoothed traffic light color detection result is off, and the current state of the traffic light is green, then it is determined that the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is flashing.

[0024] If the smoothed traffic light color detection result is yellow, and the current state of the traffic light is flashing, then it is determined that the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is yellow.

[0025] If the smoothed traffic light color detection result is red, and the current state of the traffic light is yellow, then the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is red.

[0026] If the smoothed traffic light color detection result is green, and the current state of the traffic light is red, then it is determined that the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is green.

[0027] If the traffic light state after the transition is any one of yellow, red, or green, reset the state queue of the predefined finite state machine.

[0028] If, based on the smoothed traffic light color detection result and the current state of the traffic light, it is determined that no state transition rule has been triggered, then the current state of the traffic light remains unchanged.

[0029] Optionally, determining the actual traffic light state at the current moment based on the initial traffic light color detection result and the current state of the traffic light, using the predefined state transition rules of the finite state machine and the preset smoothing mechanism, includes:

[0030] If the smoothed traffic light color detection result is off, and the current state of the traffic light is green, then the current state of the traffic light is determined using preset flashing state information to determine whether the current state of the traffic light triggers the state transition rule. The preset flashing state information includes at least one of flashing duration, flashing count, and flashing period.

[0031] Optionally, the traffic light status detection method further includes:

[0032] Update the status queue based on the actual traffic light status at the current moment;

[0033] The traffic light state prediction model is updated based on the state data stored in the updated state queue.

[0034] The updated traffic light state prediction model is used to predict the traffic light state, and the prediction result of the traffic light state is obtained. The prediction result of the traffic light state includes the state of the traffic light at the next moment and the state switching time of the state at the next moment.

[0035] Secondly, embodiments of this application also provide a traffic light status detection device, the traffic light status detection device comprising:

[0036] The acquisition unit is used to acquire images of the traffic scene at the current moment;

[0037] A traffic light detection unit is used to perform traffic light detection on the image of the traffic scene at the current moment to obtain the traffic light detection result at the current moment, which includes the initial traffic light color detection result at the current moment.

[0038] The determining unit is used to determine the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment, using a predefined finite state machine and a preset smoothing mechanism.

[0039] Thirdly, embodiments of this application also provide an apparatus, comprising:

[0040] A processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform any of the aforementioned traffic light state detection methods.

[0041] Fourthly, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement any of the aforementioned traffic light state detection methods.

[0042] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: The traffic light state detection method of this application embodiment first acquires an image of the traffic scene at the current moment; then, it performs traffic light detection on the image of the traffic scene at the current moment to obtain the traffic light detection result at the current moment, which includes the initial traffic light color detection result at the current moment; finally, based on the initial traffic light color detection result at the current moment, it determines the actual traffic light state at the current moment using a predefined finite state machine and a preset smoothing mechanism. The traffic light state detection method of this application embodiment significantly improves the accuracy and robustness of traffic light state recognition by using a finite state machine and a smoothing mechanism for multi-frame data, effectively addressing the recognition challenges in complex traffic environments. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0044] Figure 1 This is a flowchart illustrating a traffic light status detection method according to an embodiment of this application;

[0045] Figure 2 This is a schematic diagram of the structure of a traffic light status detection device according to an embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the structure of a device according to an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0049] Currently, traffic light status detection technology mainly relies on computer vision and deep learning methods. Common technical solutions include image classification models based on convolutional neural networks (CNNs), object detection algorithms (such as YOLO and SSD), and image processing methods based on traditional computer vision techniques. These solutions aim to achieve efficient traffic light detection by extracting features from images and identifying and classifying the status of traffic lights.

[0050] (1) Deep learning-based detection algorithms: These algorithms can provide high accuracy and real-time performance under relatively ideal conditions. For example, using CNN for image classification can effectively identify the red, yellow, and green states of traffic lights. However, these algorithms face challenges in complex and dynamic urban environments, such as changes in lighting and traffic light occlusion.

[0051] (2) Algorithms based on traditional visual processing: These methods mainly rely on edge detection, color segmentation and other techniques to identify traffic light status. Although they perform well under certain conditions, their robustness and accuracy are often poor in complex environments.

[0052] Although existing technical solutions are theoretically feasible, they still have many shortcomings in practical applications, mainly in the following aspects:

[0053] (1) Insufficient accuracy

[0054] Existing algorithms are susceptible to interference from factors such as changes in lighting, weather conditions, and obstructions when dealing with complex environments (such as urban roads). These factors can lead to false detections (misidentification of a certain state) or false detections (failure to recognize traffic lights), thereby affecting vehicle driving decisions and increasing traffic safety hazards.

[0055] (2) Discontinuous state tracking

[0056] Many detection systems rely solely on information from the current frame for their decisions, lacking consideration of context. This makes it difficult for the system to react consistently when traffic light states change rapidly.

[0057] (3) Poor robustness

[0058] Most existing traffic light detection algorithms are based on a single feature extraction and classification model, which makes the system less robust to changes in lighting and complex environments. Especially in extreme cases, such as traffic light malfunctions or abnormal flashing, existing systems often cannot cope effectively, leading to inaccurate recognition results.

[0059] (4) Insufficient ability to handle special situations

[0060] In certain specific situations (such as malfunctioning traffic lights or flashing patterns), existing detection algorithms lack effective processing mechanisms. The system often fails to accurately identify these special cases, leading to misleading vehicle driving decisions.

[0061] Therefore, existing traffic light state detection technologies have significant shortcomings in terms of accuracy, state tracking, robustness, and ability to handle special situations. These problems not only affect the safety and reliability of autonomous driving systems but also limit their widespread application in complex urban environments. Therefore, a more advanced technological solution is urgently needed to improve the accuracy of traffic light detection and the overall robustness of the system, in order to support a safer and more reliable autonomous driving experience.

[0062] Based on this, this application proposes a traffic light state detection method based on a finite state machine, aiming to solve the problems of false detection, missed detection, and adaptability in complex environments in existing traffic light detection systems. By introducing a state machine, the system can manage traffic light state transitions, record state change times, and reduce interference from false detections. Specifically, the state machine design not only improves the accuracy of system state recognition but also effectively addresses the impact of brief false detections and flickering phenomena, thereby ensuring the system's stability under conditions of changing light and occlusion. The state machine can also improve the system's consistency and real-time response capabilities by persistently storing historical state information and combining historical data for prediction. When a brief flickering of the traffic light is detected, the state machine can confirm whether it is a genuine state transition, avoiding misleading the vehicle into making incorrect reactions. Therefore, even in complex environments, this application can more accurately identify traffic light states, providing more reliable technical support for autonomous vehicles.

[0063] Specifically, embodiments of this application provide a traffic light status detection method, such as... Figure 1 The diagram shows a flowchart of a traffic light state detection method according to an embodiment of this application. The traffic light state detection method includes the following steps S110 to S130:

[0064] Step S110: Obtain an image of the traffic scene at the current moment.

[0065] When detecting traffic light status, it is necessary to first capture images of the current traffic scene using image acquisition equipment to obtain image data including the traffic lights, providing a foundation for subsequent traffic light detection. This image acquisition equipment can be cameras deployed on autonomous vehicles or cameras deployed on the roadside.

[0066] Step S120: Perform traffic light detection on the image of the traffic scene at the current moment to obtain the traffic light detection result at the current moment. The traffic light detection result at the current moment includes the initial traffic light color detection result at the current moment.

[0067] After acquiring the current scene image, it is necessary to further utilize object detection algorithms (such as the YOLOv5 algorithm based on deep learning) to perform object detection on the acquired image, detecting information such as the location and category of traffic lights (red, green, yellow, and off), which serves as the traffic light detection result for the current moment. Considering that object detection algorithms may produce false detections, the traffic light color status information detected in this step is only used as the initial traffic light color detection result, and further confirmation of the actual traffic light status is required.

[0068] Step S130: Based on the initial traffic light color detection result at the current moment, determine the actual traffic light state at the current moment using a predefined finite state machine and a preset smoothing mechanism.

[0069] For traffic light status detection, embodiments of this application predefine a finite state machine and a smoothing mechanism. The finite state machine predefines the various possible states of the traffic light (such as red, green, yellow, flashing green, etc.) and the transition rules between states. For example, a state set S = {G, Y, R, F} is defined, where G represents green, Y represents yellow, R represents red, and F represents flashing green. State transition rules are specified, including normal state transitions such as from green to flashing, from flashing to yellow, from yellow to red, and from red to green, as well as the duration conditions of different states.

[0070] The smoothing mechanism primarily addresses the potential for brief false detections in target detection algorithms by pre-setting smoothing rules. For example, when a change in a traffic light's state is detected, the state machine does not immediately recognize it as a genuine state transition. Instead, it makes a comprehensive judgment based on historical state information (achieved through persistent historical state information). If historical data indicates that the traffic light was previously in a stable state, and the brief change might be a false detection, the state machine does not confirm this change as a genuine state transition to avoid misleading the vehicle into making an incorrect response.

[0071] By combining the initial traffic light color detection results at the current moment, the predefined finite state machine, and the preset smoothing mechanism, the detection results are comprehensively analyzed and processed to finally determine the actual traffic light state at the current moment. This ensures that the system can accurately identify the traffic light state in complex environments (such as changes in light, occlusion, etc.) and provides reliable technical support for autonomous vehicles.

[0072] The final identification results are provided to the vehicle's driving decision module for decision-making, such as whether to stop or continue driving, and guide the vehicle to take corresponding actions.

[0073] The traffic light state detection method of this application significantly improves the accuracy and robustness of traffic light state recognition by utilizing a finite state machine and a smoothing mechanism for multi-frame data, effectively addressing the recognition challenges in complex traffic environments.

[0074] In some embodiments of this application, determining the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment using a predefined finite state machine and a preset smoothing mechanism includes: determining the current moment state of the traffic light using a predefined finite state machine's state queue, wherein the current moment state originates from the actual traffic light state of the previous moment recorded in the state queue; determining the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment and the current moment state of the traffic light using the predefined finite state machine's state transition rules and the preset smoothing mechanism, and storing the actual traffic light state at the current moment in the state queue as the starting point for the state transition of the predefined finite state machine at the next moment.

[0075] A predefined finite state machine maintains a state queue that records the actual states of traffic lights at various points in the past. When it is necessary to determine the current state of the traffic light, the machine first retrieves the actual traffic light state recorded at the previous moment from the state queue and uses it as the starting point for the finite state machine's state transition at the current moment. For example, if the traffic light was recorded as green in the state queue at the previous moment, then the green light state is used as the initial state for further judgment at the current moment.

[0076] Inputs for finite state machine state transitions: The current state of the finite state machine and the initial traffic light color detection result at the current moment (external real-time input) are used as inputs for the finite state machine state transitions.

[0077] The finite state machine state transition process: The smoothing mechanism defined in this application embodiment refers to performing multi-frame fusion processing only on the initial detection result of a single frame, and outputting a smoothed detection result. Considering that the target detection algorithm may experience transient false detections, the pre-defined smoothing mechanism can filter out instantaneous false detections and avoid incorrect state judgments due to false detections. The predefined finite state machine sets normal transition rules between different traffic light states. For example, it specifies normal state transition processes such as from green light to flashing, flashing to yellow light, yellow light to red light, and red light to green light. By comparing the initial traffic light color detection result at the current moment with the actual traffic light state at the previous moment, it is determined whether the state transition rules are met, which serves as a prerequisite for determining whether a state transition has occurred.

[0078] The result of the finite state machine's state transition: By combining state transition rules and a smoothing mechanism, the actual traffic light state at the current moment is determined. This result is also the next state of the finite state machine and is recorded in the state queue as the starting point for the next state transition. This process is repeated, using the finite state machine's state transition rules and smoothing mechanism to realize the flow of traffic light states.

[0079] The pre-defined smoothing mechanism effectively addresses the possibility of brief false detections in complex environments. In complex environments such as changing lighting or obstructions, traffic light detection results may be unstable. By combining the smoothing mechanism with historical state information for comprehensive judgment, false detection signals can be filtered out, ensuring the system can accurately identify traffic light status even in complex environments, thus enhancing the system's adaptability to complex conditions.

[0080] By utilizing the state queue of a finite state machine and taking the actual traffic light state of the previous moment as a reference for the current moment's judgment, combined with state transition rules, the legality and rationality of state transitions are ensured, further improving the accuracy of state judgment and reducing erroneous state judgments caused by accidental factors or false detections.

[0081] The state queue records the historical state information of the traffic lights, providing a comprehensive reference for determining the current state. Combined with a smoothing mechanism, it maintains the consistency of state judgments, avoiding frequent state changes due to false detections at individual moments, thereby improving system stability and providing more reliable technical support for systems such as autonomous vehicles that rely on traffic light state information.

[0082] Compared with the scheme that only relies on the statistical analysis of historical frames, the smoothing mechanism combined with the legality judgment of state machine transitions in the embodiments of this application can more effectively filter out instantaneous false detections, especially when dealing with detection jumps caused by rapid flickering or occlusion.

[0083] In some embodiments of this application, determining the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment and the current state of the traffic light using the predefined state transition rules of the finite state machine and the preset smoothing mechanism includes: determining a smoothed traffic light color detection result based on the initial traffic light color detection result at the current moment using the preset smoothing mechanism; determining whether the current state of the traffic light triggers the state transition rule based on the smoothed traffic light color detection result and the current state of the traffic light; if triggered, determining the transitioned traffic light state based on the triggered state transition rule as the actual traffic light state at the current moment.

[0084] Based on the smoothing mechanism defined in the aforementioned embodiments, the initial traffic light color detection result at the current moment is first processed by multi-frame fusion using the smoothing mechanism to output a smoothed traffic light color detection result, thereby filtering out instantaneous false detections. Then, based on the smoothed traffic light color detection result and the actual traffic light state at the previous moment (i.e., the current moment's state) obtained through the state queue, these two are compared with the state transition rules set in the predefined finite state machine. For example, the finite state machine specifies that the normal state transition paths are from green to flashing, from flashing to yellow, from yellow to red, and from red to green. If the smoothed traffic light color detection result detects red, and the previous moment's state was yellow, then it is determined that the current moment's state has triggered the state transition rule from yellow to red; if the smoothed traffic light color detection result detects green, and the previous moment's state was yellow, since there is no normal transition rule from yellow to green, the current moment's detection result may be a false detection result, and it is determined that the current moment's state has not triggered the state transition rule.

[0085] When it is determined that the current state triggers a state transition rule, the traffic light state after the transition is determined according to the specific rule triggered. For example, if the state transition rule from yellow to red is triggered, the traffic light state after the transition is determined to be red.

[0086] The pre-defined smoothing mechanism plays a crucial role in determining the actual traffic light status, effectively filtering out interference from transient false detections. In complex environments, such as changes in lighting or obstructions, traffic light detection results may be unstable. Through the comprehensive judgment of the smoothing mechanism, it can be ensured that the system will not erroneously change its state due to false detections at individual moments, thus improving the system's resistance to false detection signals.

[0087] By combining a preset smoothing mechanism and predefined state transition rules, normal state transitions can be accurately identified. State transitions are only confirmed when they meet the preset rules, avoiding erroneous state judgments caused by accidental false detections or irrelevant color changes. This improves the accuracy of state transition recognition and provides more accurate and stable technical support for systems such as autonomous vehicles that rely on traffic light state information.

[0088] In some embodiments of this application, determining the smoothed traffic light color detection result based on the initial traffic light color detection result at the current moment using the preset smoothing mechanism includes: determining whether the initial traffic light color detection result at the current moment remains consistent across multiple consecutive frames based on the initial traffic light color detection result at the current moment using the preset smoothing mechanism; if so, then using the initial traffic light color detection result at the current moment as the smoothed traffic light color detection result.

[0089] After obtaining the initial traffic light color detection result for the current moment, a preset smoothing mechanism is used to further verify this initial traffic light color detection result. For example, verification can be performed in the following way:

[0090]

[0091] Among them, S t The current state represents the stable state recorded in the state machine at the previous moment (based on the smoothed result of historical frames); S final The smoothed state represents the smoothed traffic light color detection result; N is the consecutive frame threshold, representing the minimum number of consecutive consistent frames required to confirm the detection result. For example, N=3 means that 3 consecutive frames need to be detected consistently.

[0092] By continuously acquiring multiple frames of traffic scene images and performing traffic light detection on each frame, an initial traffic light color detection result is obtained for each frame. Then, it is determined whether the initial traffic light color detection result remains consistent across these multiple consecutive frames. For example, if the initial traffic light color detection result is red, and the traffic light color detection result is red in multiple consecutive frames, then the smoothed traffic light color detection result is red.

[0093] By requiring that the traffic light color detection results remain consistent across multiple consecutive frames before being considered as the actual traffic light color detection results, it is possible to effectively filter out transient false detections or unstable states caused by accidental factors (such as sudden changes in light or occlusion), greatly improving the accuracy and reliability of state judgment and reducing erroneous state transitions caused by false detections.

[0094] In some embodiments of this application, determining the actual traffic light state at the current moment based on the initial traffic light color detection result and the current state of the traffic light, using the predefined state transition rules of the finite state machine and the preset smoothing mechanism, includes: if the smoothed traffic light color detection result is off, and the current state of the traffic light is green, then the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is flashing; if the smoothed traffic light color detection result is yellow, and the current state of the traffic light is flashing, then the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is yellow; if the smoothed traffic light color... If the detection result is red and the current state of the traffic light is yellow, then the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is red. If the smoothed traffic light color detection result is green and the current state of the traffic light is red, then the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is green. If the transitioned traffic light state is any one of yellow, red, or green, the state queue of the predefined finite state machine is reset. If, based on the smoothed traffic light color detection result and the current state of the traffic light, it is determined that no state transition rule has been triggered, then the current state of the traffic light remains unchanged.

[0095] The normal state transition process defined in this application mainly includes the following situations:

[0096] 1) Determining the state from off to flashing

[0097] If the smoothed traffic light color detection result is off, and the actual traffic light state at the previous moment was green, it is determined that the current state has triggered the state transition rule, and the transitioned traffic light state is determined to be flashing. This is because in the normal working logic of the traffic light, a brief flashing prompt may occur after the green light ends, which conforms to the transition rule from a normal on state to a flashing state.

[0098] 2) Determining the transition from flashing light to yellow light

[0099] If the smoothed traffic light color detection result is yellow, and the actual traffic light state at the previous moment was flashing, it is determined that the current state has triggered the state transition rule, and the transitioned traffic light state is determined to be yellow. The flashing state is usually used as a prompt state during the transition from green to yellow. When the yellow light color is detected, the state transitions to yellow according to the preset rules.

[0100] 3) Determining the transition from yellow to red light status

[0101] If the smoothed traffic light color detection result is red, and the actual traffic light state at the previous moment was yellow, it is determined that the current state has triggered the state transition rule, and the traffic light state after the transition is red. This reflects the logic that the traffic light will inevitably transition to red after the yellow light ends in the normal working process.

[0102] 4) Determining the transition from red to green light

[0103] If the smoothed traffic light color detection result is green, and the actual traffic light state at the previous moment was red, it is determined that the current state has triggered the state transition rule, and the traffic light state after the transition is green. This is a normal transition of the traffic light from a red light prohibiting passage to a green light allowing passage.

[0104] When the traffic light state after the transition is any of the yellow, red, or green light states, the predefined state queue of the finite state machine needs to be reset. Resetting the state queue means recording the currently determined actual traffic light state as the new starting state in the state queue, while clearing historical state information that may have been recorded due to false detections or unstable states. This ensures that the state queue accurately reflects the true state of the current traffic light, providing a more accurate reference for subsequent state judgments.

[0105] This application's embodiments establish clear judgment rules for several common traffic light state transition scenarios, enabling precise matching of the traffic light's state transition logic during actual operation. By combining the smoothed traffic light color detection result with the actual traffic light state at the previous moment and judging according to preset state transition rules, the accuracy and rationality of state transition judgment are ensured, avoiding errors caused by misjudgment or state transitions that do not conform to actual logic.

[0106] For special state transitions such as the transition from off to flashing, the system can accurately identify and handle them. Although the flashing state is not the regular on state of a traffic light, it has certain indication significance in actual traffic scenarios. By incorporating it into the state transition rules, the system can better understand the special behavior of traffic lights at different stages, improve its ability to identify special states, and enable the system to more comprehensively adapt to various operating conditions of traffic lights.

[0107] In some embodiments of this application, determining the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment and the current state of the traffic light using the predefined state transition rules of the finite state machine and the preset smoothing mechanism includes: if the smoothed traffic light color detection result is off and the current state of the traffic light is green, then determining whether the current state of the traffic light triggers the state transition rule using preset flashing state information, wherein the preset flashing state information includes at least one of flashing duration, flashing count, and flashing period.

[0108] When the smoothed traffic light color detection result is off, and the actual traffic light state at the previous moment was green, it indicates that the current state may have changed to a flashing state. In order to further improve the accuracy of the flashing state judgment, it can proceed to the subsequent judgment process.

[0109] To accurately identify the flashing pattern of traffic lights, this application embodiment pre-sets flashing state information as prior information for flashing state changes, thereby determining whether it is a genuine flashing state change and making an appropriate response. This information may include, for example, flashing duration (the total duration of the flashing process), flash count, and flashing cycle (the frequency of flashing). When determining whether the current state triggers a state transition rule, this preset information can be used for analysis.

[0110] In the aforementioned preset flashing state information, the number of flashes refers to the number of times the light alternates between off and green (e.g., "off → green → off" is 2 flashes); the flashing period refers to the duration of a single alternation (e.g., off light 0.25 seconds + green light 0.25 seconds = period 0.5 seconds); and the total duration is the cumulative time of multiple flashes (e.g., 2 flashes × 0.5 seconds / period + last off light 0.5 seconds = total duration 1.5 seconds). When all the above conditions are met, it is determined to be a real flash, triggering a state transition.

[0111] This process involves pre-setting prior information, such as the flicker duration, to predict the current and next states. Unlike schemes that rely on Markov decisions and statistical analysis of single historical frames, this mechanism combines state machine rules with multi-frame smoothing to more accurately identify the actual flickering pattern, preventing erroneous state transitions caused by relying solely on perception detection results, and improving system robustness.

[0112] By introducing preset flashing state information, it is possible to more accurately determine whether the current light-off state is a flashing state, thereby more accurately determining whether to trigger the state transition rule. Compared to simply relying on color detection results and the state at the previous moment, this method comprehensively considers the characteristics of the flashing state, reduces erroneous state judgments caused by brief false detections or light-off situations unrelated to flashing, and improves the accuracy of state judgment.

[0113] Flashing status provides a certain indication during traffic light operation, but it is difficult to accurately identify based solely on color detection. Pre-set flashing status information provides the system with specific criteria for identifying flashing states, enabling the system to better understand the special behavior of traffic lights at different stages, enhancing the system's ability to identify flashing states, and allowing the system to more comprehensively adapt to various traffic light operating conditions.

[0114] In some embodiments of this application, the traffic light state detection method further includes: updating the state queue according to the actual traffic light state at the current moment; updating the traffic light state prediction model according to the state data stored in the updated state queue; and using the updated traffic light state prediction model to predict the traffic light state to obtain a traffic light state prediction result, wherein the traffic light state prediction result includes the next moment state of the traffic light and the state switching time of the next moment state.

[0115] Dynamically adjust traffic light status prediction model:

[0116] 1) Real-time data updates: Each time a state transition actually occurs, the new state data is added to the state queue, the average state switching time and transition probability are recalculated, and the traffic light state prediction model is updated.

[0117] 2) Anomaly handling: If the actual switching time deviates significantly from the prediction, the data will be marked as an anomaly and its weight in historical data will be reduced to avoid anomalies interfering with the overall pattern; if anomalies occur frequently in a certain type of transfer, the scope of data statistics will be expanded or new features such as traffic flow will be introduced to optimize the model.

[0118] Predicting traffic light status:

[0119] 1) Determine the next state: Based on the current state and the transition probability table, select the state with the highest probability as the predicted next state. For example, if the current state is R, the probability of the next state is likely G.

[0120] 2) Calculate the remaining time: Subtract the current state's duration from the historical average duration to obtain the remaining time, thus estimating the state transition time. For example, if R's average duration is 120 seconds and it has been on for 80 seconds, then the transition is expected in 40 seconds (the specific time can be identified and estimated using a digital recognition algorithm).

[0121] It should be noted that the state transition in the aforementioned embodiments is a "real-time decision-making based on reality," ensuring the reliability of the current state through multi-frame verification and rule constraints. The prediction in this application's embodiments, however, is a "forward-looking planning based on history," utilizing statistical patterns to predict future changes and guide the optimization of detection resources. By predicting traffic light states based on historical data and dynamically adjusting the prediction time by calculating the average time of past state transitions, it can adapt to the traffic light switching patterns of different regions and time periods.

[0122] Therefore, the prediction model in this application is based on dynamic optimization of the historical state sequence of a state machine. By continuously learning the transition intervals and patterns in the state queue, the prediction parameters are dynamically adjusted. Unlike static rules or single-frame feature learning, and unlike fixed-period assumptions or static map information dependence, it can adapt to the traffic light timing patterns at different intersections and special scenarios such as temporary traffic control.

[0123] In summary, this application achieves at least the following technical effects compared to the prior art:

[0124] Compared to the Markov decision + state machine approach used in existing technologies, this application avoids misjudgments by the probabilistic model in atypical scenarios by explicitly defining state transition rules (such as the forced timing sequence of green light → flashing → yellow light). Compared to the historical frame memory learning approach used in existing technologies, this application combines the logical constraints of the state machine to reduce the dependence on a large amount of historical data, and is superior in terms of real-time performance and robustness.

[0125] This application embodiment also provides a traffic light status detection device 200, such as... Figure 2 The diagram shows a schematic representation of a traffic light status detection device according to an embodiment of this application. The traffic light status detection device 200 includes: an acquisition unit 210, a traffic light detection unit 220, and a determination unit 230, wherein:

[0126] Acquisition unit 210 is used to acquire an image of the traffic scene at the current time;

[0127] Traffic light detection unit 220 is used to perform traffic light detection on the image of the traffic scene at the current moment to obtain the traffic light detection result at the current moment, which includes the initial traffic light color detection result at the current moment;

[0128] The determining unit 230 is used to determine the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment, using a predefined finite state machine and a preset smoothing mechanism.

[0129] In some embodiments of this application, the determining unit 230 is specifically used to: determine the current state of the traffic light using a predefined finite state machine state queue, wherein the current state originates from the actual traffic light state of the previous moment recorded in the state queue; determine the actual traffic light state of the current moment using the predefined finite state machine state transition rules and the preset smoothing mechanism based on the initial traffic light color detection result of the current moment and the current state of the traffic light; and store the actual traffic light state of the current moment in the state queue as the starting point for the state transition of the predefined finite state machine at the next moment.

[0130] In some embodiments of this application, the determining unit 230 is specifically used to: determine a smoothed traffic light color detection result based on the initial traffic light color detection result at the current moment using the preset smoothing mechanism; determine whether the current moment state of the traffic light triggers the state transition rule based on the smoothed traffic light color detection result and the current moment state of the traffic light; if triggered, determine the transitioned traffic light state based on the triggered state transition rule, as the actual traffic light state at the current moment.

[0131] In some embodiments of this application, the determining unit 230 is specifically used to: determine, based on the initial traffic light color detection result at the current moment, whether the initial traffic light color detection result at the current moment remains consistent across multiple consecutive frames using the preset smoothing mechanism; if so, then use the initial traffic light color detection result at the current moment as the smoothed traffic light color detection result.

[0132] In some embodiments of this application, the determining unit 230 is specifically used for: if the smoothed traffic light color detection result is off, and the current state of the traffic light is green, then determining that the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is flashing; if the smoothed traffic light color detection result is yellow, and the current state of the traffic light is flashing, then determining that the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is yellow; if the smoothed traffic light color detection result is red, and the current state of the traffic light is yellow, then determining that the current state of the traffic light is off. If the previous state triggers the state transition rule and the traffic light state after the transition is red; if the smoothed traffic light color detection result is green and the current state of the traffic light is red, then the current state of the traffic light triggers the state transition rule and the traffic light state after the transition is green; if the traffic light state after the transition is yellow, red, or green, then the state queue of the predefined finite state machine is reset; if it is determined from the smoothed traffic light color detection result and the current state of the traffic light that no state transition rule has been triggered, then the current state of the traffic light remains unchanged.

[0133] In some embodiments of this application, the determining unit 230 is specifically used to: if the smoothed traffic light color detection result is off, and the current state of the traffic light is green, then determine whether the current state of the traffic light triggers the state transition rule using preset flashing state information, wherein the preset flashing state information includes at least one of flashing duration, flashing count, and flashing period.

[0134] In some embodiments of this application, the traffic light state detection device 200 further includes: a first update unit, configured to update the state queue according to the actual traffic light state at the current moment; a second update unit, configured to update the traffic light state prediction model according to the state data stored in the updated state queue; and a prediction unit, configured to predict the traffic light state using the updated traffic light state prediction model to obtain a traffic light state prediction result, wherein the traffic light state prediction result includes the next moment state of the traffic light and the state switching time of the next moment state.

[0135] It is understood that the traffic light status detection device described above can implement all the steps of the traffic light status detection method provided in the foregoing embodiments. The relevant explanations of the traffic light status detection method are applicable to the traffic light status detection device, and will not be repeated here.

[0136] Figure 3This is a schematic diagram of the structure of a device according to an embodiment of this application. For example... Figure 3 As shown, the device includes one or more processors (or processing units), and may also include one or more memories coupled to the processors, and may also include a communication module coupled to the processors.

[0137] A communication module can be used to communicate with other devices or apparatuses, such as sending or receiving data and / or signals. A communication module may have at least one communication module for communication. A communication module may include any interface necessary for communicating with other devices. Exemplarily, a communication module may be a transceiver, circuit, bus, module, or other type of communication module.

[0138] The processor may include, but is not limited to, one or more of the following: a general-purpose computer, a special-purpose computer, a microcontroller, a digital signal processor (DSP), or a controller-based multi-core controller architecture. The device may have multiple processors, such as application-specific integrated circuit (ASIC) chips, which are time-dependent on a clock synchronized with the main processor.

[0139] The memory may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, at least one of the following: read-only memory (ROM), electrically programmable read-only memory (EPROM), flash memory, hard disk, compact disc (CD), digital video disc (DVD), or other magnetic and / or optical storage. Examples of volatile memories include, but are not limited to, at least one of the following: random access memory (RAM), or other volatile memories that do not persist during the duration of a power outage.

[0140] A computer program consists of computer-executable instructions that are executed by an associated processor. Programs can be stored in ROM. A processor can perform any appropriate action and processing by loading the program into RAM.

[0141] Possible implementations of this application can be achieved through a program, enabling the communication device to execute any of the processes discussed in the foregoing embodiments. Possible implementations of this application can also be achieved through hardware or a combination of software and hardware.

[0142] In some implementations, the program may be tangibly contained in a computer-readable storage medium, which may include in a device (such as in memory) or other storage device accessible by the device. The program may be loaded from the computer-readable storage medium into RAM for execution. The computer-readable storage medium may include any type of tangible non-volatile memory, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc.

[0143] This application also provides a computer-readable storage medium storing computer instructions or program code thereon, which, when executed by a processor, causes the processor to perform the methods and functions involved in any of the above embodiments. A computer-readable medium can be any tangible medium that contains or stores a program for or relating to an instruction execution system, apparatus, or device. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. More detailed examples of computer-readable storage media include electrical connections with one or more wires, magnetic media (e.g., disks, floppy disks, hard disks, magnetic tapes, magnetic storage devices), optical media (e.g., optical storage devices, DVDs), semiconductor media (e.g., solid-state drives), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), or any suitable combination thereof.

[0144] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. Embodiments of this application also provide at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. This computer program product includes one or more computer-executable instructions, such as instructions included in a program module, which execute in a device on a target's real or virtual processor to perform the processes, methods, and functions involved in any of the above embodiments. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0145] This application also proposes a computer program product, including a computer program or instructions that, when run on a computer, cause the computer to perform the processes, methods, and functions described in the above embodiments. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided as needed. The machine-executable instructions for the program modules can be executed locally or in a distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0146] Generally, the various embodiments of this application can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software, which can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of this disclosure are shown and described as block diagrams, flowcharts, or represented using some other illustration, it should be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as, as non-limiting examples, in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0147] It should be noted that although embodiments of this application have been described above with reference to the accompanying drawings, these embodiments are not independent of each other, and they can be combined to obtain other embodiments. The methods, situations, categories, and classifications of embodiments in this application are only for the convenience of description and should not constitute a special limitation. Various methods, categories, situations, and features in embodiments can be combined with each other if logically consistent. The various embodiments of this application can be arbitrarily combined to achieve different technical effects. The embodiments of this application will not list various combinations.

[0148] Furthermore, although the operation of the methods of this disclosure is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps. It should also be noted that the features and functions of two or more devices according to this disclosure may be embodied in one device. Conversely, the features and functions of one device described above may be further divided and embodied by multiple devices.

[0149] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0150] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting the status of a traffic light, characterized in that, The traffic light status detection method includes: Obtain an image of the traffic scene at the current moment; Traffic light detection is performed on the image of the traffic scene at the current moment to obtain the traffic light detection result at the current moment, which includes the initial traffic light color detection result at the current moment; Based on the initial traffic light color detection results at the current moment, the actual traffic light state at the current moment is determined using a predefined finite state machine and a preset smoothing mechanism.

2. The traffic light status detection method according to claim 1, characterized in that, The step of determining the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment, using a predefined finite state machine and a preset smoothing mechanism, includes: The current state of a traffic light is determined using a predefined state queue of a finite state machine, where the current state is derived from the actual traffic light state of the previous moment recorded in the state queue. Based on the initial traffic light color detection result at the current moment and the current state of the traffic light, the actual traffic light state at the current moment is determined using the predefined finite state machine's state transition rules and the preset smoothing mechanism. The actual traffic light state at the current moment is then stored in the state queue as the starting point for the state transition of the predefined finite state machine at the next moment.

3. The traffic light status detection method according to claim 2, characterized in that, The step of determining the actual traffic light state at the current moment based on the initial traffic light color detection result and the current state of the traffic light, using the predefined state transition rules of the finite state machine and the preset smoothing mechanism, includes: Based on the initial traffic light color detection result at the current moment, the smoothed traffic light color detection result is determined using the preset smoothing mechanism; Based on the smoothed traffic light color detection result and the current state of the traffic light, determine whether the current state of the traffic light triggers the state transition rule; If triggered, the traffic light state after the transition is determined according to the triggered state transition rules, and this state is taken as the actual traffic light state at the current moment.

4. The traffic light status detection method according to claim 3, characterized in that, The step of determining the smoothed traffic light color detection result based on the initial traffic light color detection result at the current moment using the preset smoothing mechanism includes: Based on the initial traffic light color detection result at the current moment, the preset smoothing mechanism is used to determine whether the initial traffic light color detection result at the current moment remains consistent across multiple consecutive frames. If so, the initial traffic light color detection result at the current moment is used as the smoothed traffic light color detection result.

5. The traffic light status detection method according to claim 3, characterized in that, The step of determining the actual traffic light state at the current moment based on the initial traffic light color detection result and the current state of the traffic light, using the predefined state transition rules of the finite state machine and the preset smoothing mechanism, includes: If the smoothed traffic light color detection result is off, and the current state of the traffic light is green, then it is determined that the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is flashing. If the smoothed traffic light color detection result is yellow, and the current state of the traffic light is flashing, then it is determined that the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is yellow. If the smoothed traffic light color detection result is red, and the current state of the traffic light is yellow, then the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is red. If the smoothed traffic light color detection result is green, and the current state of the traffic light is red, then it is determined that the current state of the traffic light triggers the state transition rule and the transitioned traffic light state is green. If the traffic light state after the transition is any one of yellow, red, or green, reset the state queue of the predefined finite state machine. If, based on the smoothed traffic light color detection result and the current state of the traffic light, it is determined that no state transition rule has been triggered, then the current state of the traffic light remains unchanged.

6. The traffic light status detection method according to claim 3, characterized in that, The step of determining the actual traffic light state at the current moment based on the initial traffic light color detection result and the current state of the traffic light, using the predefined state transition rules of the finite state machine and the preset smoothing mechanism, includes: If the smoothed traffic light color detection result is off, and the current state of the traffic light is green, then the current state of the traffic light is determined using preset flashing state information to determine whether the current state of the traffic light triggers the state transition rule. The preset flashing state information includes at least one of flashing duration, flashing count, and flashing period.

7. The traffic light status detection method according to claim 1, characterized in that, The traffic light status detection method also includes: Update the status queue based on the actual traffic light status at the current moment; The traffic light state prediction model is updated based on the state data stored in the updated state queue. The updated traffic light state prediction model is used to predict the traffic light state, and the prediction result of the traffic light state is obtained. The prediction result of the traffic light state includes the state of the traffic light at the next moment and the state switching time of the state at the next moment.

8. A traffic light status detection device, characterized in that, The traffic light status detection device includes: The acquisition unit is used to acquire images of the traffic scene at the current moment; A traffic light detection unit is used to perform traffic light detection on the image of the traffic scene at the current moment to obtain the traffic light detection result at the current moment, which includes the initial traffic light color detection result at the current moment. The determining unit is used to determine the actual traffic light state at the current moment based on the initial traffic light color detection result at the current moment, using a predefined finite state machine and a preset smoothing mechanism.

9. An apparatus comprising: processor; And a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the traffic light state detection methods of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the traffic light status detection method according to any one of claims 1 to 7.

Images