Traffic signal lamp identification method and device, electronic equipment and storage medium

Abstract

The invention provides a traffic signal lamp identification method and device, electronic equipment and a storage medium. The method comprises the following steps: determining lane information of an environment where a vehicle is located, road topology information and data information of a traffic signal lamp according to sensing data; according to the lane information, determining a drivable direction after the vehicle enters an intersection corresponding to the traffic signal lamp and a feasible probability of the drivable direction; checking the consistency of the driving direction and the drivable direction of the vehicle in front of the vehicle at the intersection corresponding to the traffic signal lamp to obtain a consistency result; determining a target lane of the vehicle at an intersection corresponding to the traffic signal lamp and a target driving direction of the target lane according to a consistency result; and according to the target driving direction, the feasible probability, the road topology information and the data information, determining a target traffic signal lamp corresponding to the target lane and an identification probability of the target traffic signal lamp. The traffic signal lamp can be identified based on the real-time data, and the identification efficiency and the identification accuracy of the traffic signal lamp are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and more specifically, to a method, device, electronic device, and storage medium for identifying traffic lights. Background Art

[0002] With the rapid development of technology, the frequency of using cars in people's daily lives is also increasing. To ensure the driving safety of cars, identifying traffic lights is a crucial step. Currently, traffic lights are usually identified only based on images or through perception for online inference, but due to the need to combine map information or a large amount of data, the efficiency and accuracy of traffic light identification are reduced. Therefore, how to improve the efficiency and accuracy of traffic light identification has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, embodiments of this application propose a method, device, electronic device, and storage medium for identifying traffic lights to improve the above problems.

[0004] According to the first aspect of the embodiments of this application, a method for identifying traffic lights is provided. The method includes: obtaining sensing data collected by sensors of the vehicle itself, and determining lane information, road topology information, and traffic light data information of the environment where the vehicle itself is located according to the sensing data, where the road topology information includes the entry and exit relationships between each lane and the intersections corresponding to traffic lights; determining the drivable direction and the feasible probability of the drivable direction after the vehicle enters the intersection corresponding to the traffic light according to the lane information; performing a consistency check on the driving direction of the vehicle in front of the vehicle itself at the intersection corresponding to the traffic light and the drivable direction to obtain a consistency result; determining the target lane and the target driving direction of the target lane of the vehicle itself at the intersection corresponding to the traffic light according to the consistency result; and determining the target traffic light corresponding to the target lane and the recognition probability of the target traffic light according to the target driving direction, the feasible probability, the road topology information, and the data information.

[0005] According to a second aspect of the embodiments of the present application, there is provided an identification device for traffic lights. The device includes: an information determination module, configured to obtain sensing data collected by sensors of the vehicle itself, and determine lane information, road topology information, and traffic light data information of the environment where the vehicle itself is located according to the sensing data, wherein the road topology information includes the entry and exit relationships between each lane and the intersections corresponding to the traffic lights; a drivable direction prediction module, configured to determine the drivable direction and the feasible probability of the drivable direction after the vehicle itself enters the intersection corresponding to the traffic lights according to the lane information; a consistency determination module, configured to perform a consistency check on the driving direction of the vehicle in front of the vehicle itself at the intersection corresponding to the traffic lights and the drivable direction, and obtain a consistency result; a target driving direction determination module, configured to determine the target lane and the target driving direction of the target lane of the vehicle itself at the intersection corresponding to the traffic lights according to the consistency result; a target traffic light determination module, configured to determine the target traffic light corresponding to the target lane and the recognition probability of the target traffic light according to the target driving direction, the feasible probability, the road topology information, and the data information.

[0006] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the above-mentioned traffic light identification method is implemented.

[0007] According to a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the above-mentioned traffic light identification method is implemented.

[0008] In the solution of this application, first, lane information, road topology information, and data information of traffic signals in the environment where the vehicle is located are determined based on the acquired sensing data. Furthermore, based on the lane information, the drivable directions after the vehicle enters the intersection corresponding to the traffic signal and the feasible probabilities corresponding to the drivable directions can be determined. Then, a consistency check is performed on the driving direction of the vehicle in front at the intersection corresponding to the traffic signal and the drivable direction of the vehicle, and a consistency result is obtained. Based on this consistency result, the target lane and the target driving direction of the target lane at the intersection corresponding to the traffic signal are determined. Furthermore, based on the target driving direction, the feasible probability, the road topology information, and the data information, the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal can be determined. In the solution of this application, the target traffic signal is recognized through real-time collected data, without the need for map information, avoiding the situation of incorrect recognition or failure to recognize traffic signals caused by the inability to obtain map information, improving the recognition accuracy of traffic signals, and being able to recognize traffic signals based on real-time data, improving the recognition efficiency of traffic signals. At the same time, in this application, a consistency judgment is made based on the driving direction of the vehicle in front, further improving the judgment accuracy of the driving direction of the vehicle, and thus improving the recognition accuracy of traffic signals.

[0009] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application. Obviously, the accompanying drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 is a schematic flowchart of a method for recognizing a traffic signal according to an embodiment of this application.

[0012] Figure 2 is a schematic flowchart of a method for recognizing a traffic signal according to another embodiment of this application.

[0013] Figure 3 is a schematic flowchart of a method for recognizing a traffic signal according to still another embodiment of this application.

[0014] Figure 4 is a schematic flowchart of a method for recognizing a traffic signal according to yet another embodiment of this application.

[0015] Figure 5It is a flowchart showing the specific steps of step 450 according to an embodiment of the present application.

[0016] Figure 6 It is a flowchart showing the specific steps of step 460 according to an embodiment of the present application.

[0017] Figure 7 It is a flowchart showing a method for identifying traffic lights according to another embodiment of the present application.

[0018] Figure 8 It is a block diagram of an apparatus for identifying traffic lights according to an embodiment of the present application.

[0019] Figure 9 It is a hardware structure diagram of an electronic device according to an embodiment of the present application.

[0020] Through the above-mentioned drawings, specific embodiments of the present invention have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the inventive concept in any way, but to illustrate the concept of the present invention to those skilled in the art through specific embodiments. Detailed Embodiments

[0021] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0023] In addition, the described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application may be practiced without one or more of the specific details, or other methods, devices, steps, etc. may be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0024] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices. The flowcharts shown in the drawings are only exemplary illustrations, not necessarily including all the content and operations / steps, nor necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0025] Please refer to Figure 1 , Figure 1 which shows a method for identifying traffic lights provided by an embodiment of the present application. In a specific embodiment, the method for identifying traffic lights can be applied to a traffic light identification device 500 as shown in Figure 8 and an electronic device 600 configured with the traffic light identification device 500 ( Figure 9 ). The specific process of this embodiment will be described below. Of course, it can be understood that this method can be executed by a cloud server with computing and processing capabilities. The following will elaborate in detail on the Figure 1 shown process. The method for identifying traffic lights may specifically include the following steps:

[0026] Step 110, obtain the sensing data collected by the sensors of the vehicle itself, and determine the lane information, road topology information, and traffic light data information of the environment where the vehicle itself is located according to the sensing data. Among them, the road topology information includes the entry and exit relationships between each lane and the intersections corresponding to the traffic lights.

[0027] As a way, there are various sensors on the vehicle, and the environmental data of the vehicle during driving can be collected through the sensors to obtain the sensing data. Among them, the sensing data may include image sensing data, radar sensing data, etc. Optionally, the sensing data may further include the road vector information of the vehicle during driving sensed by the sensors, for example, the lane information and the curb position information of the environment where the vehicle is located, etc.

[0028] As a way, when there are traffic lights in the environment where the vehicle is located, the traffic lights related to the lane where the vehicle is located can be determined, so as to facilitate controlling the movement of the vehicle at the intersection according to the indication information of the traffic lights, so as to avoid the vehicle violating traffic rules and thus causing accidents. Optionally, a neural network can be used to identify the sensing data collected by the vehicle's sensors, so as to determine the lane information, road topology information and traffic light data information of the environment where the vehicle is located. Among them, the collected image sensing data can be used for identification to determine the entry and exit relationship between the roads at the intersection of the environment where the vehicle is located, so as to determine the road topology information of the environment where the vehicle is located.

[0029] Step 120, determine the drivable direction and the feasible probability of the drivable direction after the vehicle enters the intersection corresponding to the traffic light according to the lane information.

[0030] As a way, before the vehicle enters the intersection, it is impossible to accurately know the driving direction of the vehicle when it enters the intersection. Therefore, when the vehicle has not entered the intersection corresponding to the traffic light, the drivable direction of the lane where the vehicle is located when the vehicle enters the intersection corresponding to the traffic light can be predicted through the lane information. And in order to improve the driving accuracy of the vehicle after it enters the intersection corresponding to the traffic light, the feasible probability corresponding to the drivable direction after the predicted vehicle enters the intersection corresponding to the traffic light in the corresponding lane can also be determined.

[0031] Optionally, the drivable direction of the vehicle after it enters the intersection corresponding to the traffic light can be determined by the lane indication sign of the lane where the vehicle is located or the lane position relationship of the lane where the vehicle is located relative to other lanes.

[0032] Optionally, if the lane information is determined by the neural network, when the neural network outputs the lane information, the confidence corresponding to the lane information can be output at the same time. Then, when determining the drivable direction of the vehicle entering the intersection corresponding to the traffic light according to the lane information, the confidence corresponding to the lane information is determined as the feasible probability corresponding to the drivable direction.

[0033] Optionally, it can also be to first determine the number of lanes in the environment where the vehicle is located, obtain the prior probability table corresponding to different numbers of lanes and the corresponding drivable directions, and after determining the number of lanes in the environment where the vehicle is located and the drivable direction of the vehicle entering the intersection corresponding to the traffic light, query the prior probability table to determine the feasible probability corresponding to the drivable direction of the vehicle after it enters the intersection corresponding to the traffic light.

[0034] Step 130, check the consistency between the driving direction of the vehicle in front of the vehicle at the intersection corresponding to the traffic light and the drivable direction, and obtain a consistency result.

[0035] As a way, after determining the drivable direction of the vehicle when entering the intersection corresponding to the traffic signal, the driving direction of the vehicle's leading vehicle after entering the intersection corresponding to the traffic signal can be determined, so as to perform a consistency check based on the driving direction of the leading vehicle and the drivable direction of the vehicle, so as to determine whether the driving direction of the leading vehicle is the same as the drivable direction of the vehicle, so as to obtain a consistency result, thereby improving the accuracy of determining the target driving direction of the vehicle. Optionally, the consistency result may include that the driving direction of the leading vehicle is the same as the drivable direction of the vehicle and that the driving direction of the leading vehicle is different from the drivable direction of the vehicle.

[0036] Step 140, determine the target lane of the vehicle at the intersection corresponding to the traffic signal and the target driving direction of the target lane according to the consistency result.

[0037] As an implementation method, if the consistency result indicates that the driving direction of the leading vehicle is the same as the drivable direction of the vehicle, the target lane of the vehicle in the current environment can be determined based on the drivable direction, and the drivable direction of the vehicle at the intersection corresponding to the traffic signal can be determined as the target driving direction; if the consistency result indicates that the driving direction of the leading vehicle is different from the drivable direction of the vehicle, the target lane can be determined based on the driving direction of the leading vehicle and the drivable direction of the vehicle, and the drivable direction or the driving direction of the leading vehicle can be determined as the target driving direction of the target lane. For example, if the consistent driving result indicates that the driving direction of the leading vehicle is straight and the drivable direction of the vehicle is left turn, the target driving direction can be determined as the straight direction.

[0038] Step 150, determine the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the target driving direction, the feasible probability, the road topology information, and the data information.

[0039] As a way, after determining the target driving direction, the corresponding relationship between different traffic signal groups and the corresponding lanes in the intersection corresponding to the traffic signal can be determined first according to the road topology information and the data information of the traffic signal, so as to determine the control relationship between the traffic signal group and the driving direction of the vehicles driving in each lane, and then the target signal lamp corresponding to the control target driving direction in the traffic signal can be determined, and the recognition probability of the target traffic signal can be determined according to the feasible probability, so that the user can determine whether to control the driving of the vehicle at the intersection corresponding to the traffic signal according to the driving instruction information corresponding to the target traffic signal.

[0040] In an embodiment of the present application, first, lane information, road topology information, and traffic signal data information of the environment where the vehicle is located are determined based on the acquired sensing data. Furthermore, based on the lane information, the drivable direction and the corresponding feasible probability after the vehicle enters the intersection corresponding to the traffic signal can be determined. Then, a consistency check is performed on the driving direction of the vehicle in front at the intersection corresponding to the traffic signal and the drivable direction of the vehicle, and a consistency result is obtained. Based on this consistency result, the target lane and the target driving direction of the target lane at the intersection corresponding to the traffic signal are determined. Furthermore, based on the target driving direction, the feasible probability, the road topology information, and the data information, the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal can be determined. In the solution of the present application, the target traffic signal is identified through real-time collected data without the need for map information, avoiding the situation of incorrect or unrecognizable traffic signal identification when map information cannot be obtained, improving the recognition accuracy of traffic signals, and being able to identify traffic signals based on real-time data, improving the recognition efficiency of traffic signals. At the same time, in the present application, a consistency judgment is made according to the driving direction of the vehicle in front, further improving the judgment accuracy of the driving direction of the vehicle, and thus improving the recognition accuracy of traffic signals.

[0041] Please refer to Figure 2 , Figure 2 which shows a method for identifying traffic signals provided by an embodiment of the present application. The following will elaborate in detail on the Figure 2 process shown. The lane information includes the number of lanes and the relative position relationship of the lanes; the method for identifying traffic signals may specifically include the following steps:

[0042] Step 210, obtain the sensing data collected by the sensors of the vehicle, and determine the lane information, road topology information, and traffic signal data information of the environment where the vehicle is located according to the sensing data, where the road topology information includes the entry and exit relationships between each lane and the intersection corresponding to the traffic signal.

[0043] Step 220, determine whether there are lane signs and road surface arrows in the environment where the vehicle is located.

[0044] As a way, in order to determine the vehicle at the intersection corresponding to the traffic signal, first, identification can be performed according to the sensing data to determine whether there are lane signs and road surface arrows in the lane where the vehicle is located. Then, when the vehicle has not entered the intersection corresponding to the traffic signal, the drivable direction of the vehicle can be accurately determined based on the lane signs and road surface arrows. Optionally, a neural network can be used to identify the collected image sensing data to determine whether there are lane signs and road surface arrows in the environment where the vehicle is located.

[0045] Step 230, if it is determined that there is no such lane sign and road surface arrow, then based on the number of lanes and the relative lane position relationship, determine the drivable direction of the vehicle after entering the intersection corresponding to the traffic signal and the feasible probability corresponding to the drivable direction.

[0046] As a way, when it is determined that there is no lane sign and road surface arrow and it is impossible to directly determine the drivable direction of the vehicle at the intersection corresponding to the traffic signal based on the lane sign or road surface arrow, the drivable direction of the vehicle when entering the intersection corresponding to the traffic signal and the feasible probability corresponding to the drivable direction can be predicted according to the number of lanes and the relative lane position relationship in the lane information. For example, if the number of lanes is 3 and the relative lane position relationship indicates that the lane where the vehicle is located is on the left relative to the other two lanes, it can be determined that the lane where the vehicle is located is the leftmost lane, and its drivable direction can be a left turn or a straight line; if the relative lane position relationship indicates that the lane where the vehicle is located is on the right of one lane and on the left of another lane, it can be determined that the lane where the vehicle is located is the middle lane, and its drivable direction can be a straight line; if the relative lane position relationship indicates that the lane where the vehicle is located is on the right relative to the other two lanes, it can be determined that the lane where the vehicle is located is the rightmost lane, and its drivable direction can be a right turn.

[0047] Optionally, a mapping relationship between different numbers of lanes, different relative lane position relationships and the corresponding probabilities of drivable directions can be preset in advance. After determining the number of lanes and the relative lane position relationship, the feasible probability corresponding to the drivable direction is determined based on this mapping relationship.

[0048] Step 240, if it is determined that there is such a lane sign and road surface arrow, then based on the driving directions indicated by the lane sign and the road surface arrow, determine the drivable direction of the vehicle after entering the intersection corresponding to the traffic signal and the feasible probability corresponding to the drivable direction.

[0049] As a way, after determining that there is a lane sign and a road surface arrow, the image corresponding to the lane sign or the road surface arrow can be recognized to determine the drivable direction of the vehicle corresponding to entering the intersection corresponding to the traffic signal.

[0050] Optionally, when it is determined that there is a lane sign and a road surface arrow, it can be determined that the drivable direction of the vehicle corresponding to entering the intersection corresponding to the traffic signal is determined, that is, the feasible probability corresponding to its drivable direction is 1 or 100%.

[0051] Step 250, perform a consistency check on the driving direction of the vehicle in front of the vehicle at the intersection corresponding to the traffic signal and the drivable direction to obtain a consistency result.

[0052] Step 260: Determine the target lane of the vehicle at the intersection corresponding to the traffic signal and the target driving direction of the target lane according to the consistency result.

[0053] Step 270: Determine the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the target driving direction, the feasible probability, the road topology information, and the data information.

[0054] Specific step descriptions of steps 210 and steps 250 - 270 can refer to steps 110 and steps 130 - 150, which will not be elaborated here.

[0055] In this embodiment, it is possible to determine whether to determine the drivable direction of the vehicle when entering the intersection corresponding to the traffic signal based on the lane signs and road surface arrows by determining whether there are lane signs and road surface arrows in the environment where the vehicle is located. Furthermore, when there are no lane signs and road surface arrows, it is possible to determine the drivable direction of the vehicle when entering the intersection corresponding to the traffic signal according to the number of lanes and the relative position relationship of the lanes, or determine the drivable direction of the vehicle when entering the intersection corresponding to the traffic signal based on the lane signs and road surface arrows, improving the accuracy of the drivable direction and thus improving the recognition accuracy of the target traffic signal.

[0056] Please refer to Figure 3 , Figure 3 which shows a method for recognizing a traffic signal provided in an embodiment of the present application. The following will elaborate in detail on the Figure 3 shown process. The lane information includes the number of lanes and the relative position relationship of the lanes; the method for recognizing a traffic signal may specifically include the following steps:

[0057] Step 310: Obtain the sensing data collected by the sensors of the vehicle, and determine the lane information, road topology information, and data information of the traffic signal of the environment where the vehicle is located according to the sensing data. Among them, the road topology information includes the entry and exit relationships between each lane and the intersection corresponding to the traffic signal.

[0058] Step 320: Determine the drivable direction of the vehicle after entering the intersection corresponding to the traffic signal and the feasible probability of the drivable direction according to the lane information.

[0059] Step 330: Obtain the driving trajectory of the vehicle in front, fit the driving trajectory to obtain a fitting curve, and determine the curvature of the fitting curve.

[0060] As a way, after determining the drivable direction of the vehicle entering the intersection corresponding to the traffic signal and the feasible probability corresponding to the drivable direction, in order to ensure the accuracy of the drivable direction and the feasible probability corresponding to the drivable direction, the driving direction of the preceding vehicle can be determined at the intersection corresponding to the traffic signal to check the consistency between the driving direction of the preceding vehicle and the drivable direction of the vehicle. Among them, in order to determine the driving direction of the preceding vehicle at the intersection of the traffic signal, the driving data of the preceding vehicle can be collected through the sensors of the vehicle to obtain the driving trajectory of the preceding vehicle, and then the driving trajectory can be fitted.

[0061] Optionally, after obtaining the driving trajectory of the preceding vehicle, the driving trajectory of the preceding vehicle can be fitted by the spline interpolation algorithm to obtain a fitting curve, and the curvature of the fitting curve can be obtained by taking the derivative of the function corresponding to the fitting curve.

[0062] Step 340, determine the driving direction of the preceding vehicle after entering the intersection where the traffic signal is located according to the curvature.

[0063] As a way, after determining the curvature of the fitting curve, the curvature can be compared with the curvature threshold to determine the size relationship between the curvature and the curvature threshold to determine the driving direction of the preceding vehicle after entering the intersection where the traffic signal is located. Optionally, if the curvature is less than the curvature threshold, it can be determined that the driving direction of the preceding vehicle after entering the intersection where the traffic signal is located is a left turn; if the curvature is equal to the curvature threshold, it can be determined that the driving direction of the preceding vehicle after entering the intersection where the traffic signal is located is a straight line; if the curvature is greater than the curvature threshold, it can be determined that the driving direction of the preceding vehicle after entering the intersection where the traffic signal is located is a right turn.

[0064] Step 350, check the consistency between the driving direction and the drivable direction to obtain a consistency result.

[0065] As a way, performing the consistency check can be to determine whether the driving direction of the preceding vehicle is the same as the drivable direction of the vehicle, so as to improve the accuracy of the drivable direction and the feasible probability of the drivable direction.

[0066] Optionally, the feasible probability corresponding to the drivable direction can also be adjusted according to the consistency result, so that the accuracy of the drivable probability is higher.

[0067] In some embodiments, after the step 350, it further includes: if the consistency result indicates that the driving direction and the drivable direction are inconsistent, obtain a probability adjustment value and adjust the feasible probability according to the probability adjustment value; or if the consistency result indicates that the driving direction and the drivable direction are consistent, do not adjust the feasible probability.

[0068] As a way, a probability adjustment value can be preset in advance. When the consistency result indicates that the driving direction is inconsistent with the drivable direction, the feasible probability of the drivable direction is adjusted based on the probability adjustment value. Optionally, the number of preceding vehicles can be determined, and the consistency check is performed on all driving directions of the preceding vehicles and the drivable direction of the vehicle itself respectively to obtain multiple consistency results. If the consistency result indicates that there are multiple driving directions of the preceding vehicles different from the drivable direction of the vehicle itself, the number of preceding vehicles with driving directions different from the drivable direction of the vehicle itself can be determined, and the feasible probability is adjusted according to the number and the probability adjustment value, so as to make the accuracy of the feasible probability higher.

[0069] Step 360, determine the target lane of the vehicle at the intersection corresponding to the traffic signal and the target driving direction of the target lane according to the consistency result.

[0070] Step 370, determine the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the target driving direction, the feasible probability, the road topology information and the data information.

[0071] Among them, the specific step descriptions of steps 310 - 320 and steps 340 - 350 can refer to steps 110 - 120 and steps 140 - 150, and will not be elaborated here.

[0072] In this embodiment, after determining the drivable direction of the vehicle itself, the driving trajectory of the obtained preceding vehicle can be fitted and calculated to obtain a fitting curve corresponding to the driving trajectory of the preceding vehicle, and then the curvature corresponding to the fitting curve can be determined, so that the driving direction of the preceding vehicle when entering the intersection corresponding to the traffic signal can be determined according to the curvature, and then the consistency check can be performed on the driving direction of the preceding vehicle and the drivable direction of the vehicle itself to obtain a consistency result. Furthermore, the lane where the vehicle is located and the target driving direction in the target lane can be determined according to the consistency result, which ensures the accuracy of the target driving direction and further improves the recognition accuracy of the target traffic signal.

[0073] Please refer to Figure 4 , Figure 4 which shows the traffic signal recognition method provided by an embodiment of the present application. The following will elaborate in detail on the Figure 4 shown process. The lane information includes the number of lanes and the relative position relationship of the lanes; the traffic signal recognition method may specifically include the following steps:

[0074] Step 410: Obtain the sensing data collected by the sensors of the vehicle, and determine the lane information, road topology information, and traffic signal data information of the environment where the vehicle is located according to the sensing data. Among them, the road topology information includes the entry and exit relationships between each lane and the intersections corresponding to the traffic signals.

[0075] Step 420: Determine the drivable directions after the vehicle enters the intersection corresponding to the traffic signal and the feasible probabilities of the drivable directions according to the lane information.

[0076] Step 430: Check the consistency between the driving direction of the vehicle in front of the vehicle at the intersection corresponding to the traffic signal and the drivable direction, and obtain a consistency result.

[0077] Step 440: Determine the target lane of the vehicle at the intersection corresponding to the traffic signal and the target driving direction of the target lane according to the consistency result.

[0078] Among them, the specific step descriptions of steps 410 - 440 can refer to steps 110 - 140, and will not be elaborated here.

[0079] Step 450: Determine the association relationship and association probability between the traffic signal and each lane according to the lane information, the road topology information, and the data information.

[0080] As a way, since the entry and exit relationships of each lane at the intersection are indicated in the road topology information, the association relationship between the traffic signal and each lane can be determined according to the lane information, the road topology information, and the type and position relative to the vehicle of the traffic signal indicated in the data information. Among them, the association relationship can indicate the driving of the vehicles in the lanes controlled by different types of traffic signal groups.

[0081] Optionally, the association probability can be determined by the confidence level output when the neural network determines the lane information, the road topology information, and the traffic signal data information according to the sensing data, or can be determined by Hungarian matching.

[0082] In some embodiments, the data information includes the position information and plane angle of the traffic signal, where the plane angle is the included angle between the traffic signal and the stop line corresponding to each lane. As Figure 5 shown, step 450 includes:

[0083] Step 451: Perform Hungarian matching according to the road topology information, the lane information, the position information, and the plane angle to determine a bipartite graph.

[0084] As a method, the Hungarian matching is usually used to solve the data association problem in multi-object tracking. In the embodiments of the present application, the association relationship between the traffic lights and each lane can be determined by the Hungarian matching algorithm. Optionally, each lane can be named according to the road topology information and lane information, so as to facilitate the distinction of different lanes, and the entry and exit relationship between the different lanes controlled by different traffic lights can be determined based on the position information and plane angle of the traffic lights, so as to determine a bipartite graph, and the bipartite graph can include the connectivity relationship between the lanes that can be entered after the lanes at different positions pass through the intersection.

[0085] Step 452: Determine the association relationship between the traffic lights and each lane according to the bipartite graph.

[0086] As a method, the bipartite graph can include the connectivity relationship between the lanes that the vehicles in different lanes can enter after entering the intersection. Based on this connectivity relationship and the type of traffic lights, the association relationship between the traffic lights and each lane can be determined. For example, among the traffic lights facing the vehicle, the left-turn traffic light controls the driving of the vehicles on the left-turn lane.

[0087] Step 453: Determine the relative distance between the traffic lights and the stop lines corresponding to each lane according to the lane information and the position information, and determine the matching edge value of the bipartite graph according to the relative distance and the plane angle, and determine the matching edge value as the association probability.

[0088] As a method, the matching edge value of the bipartite graph can be determined by the formula s = w1*(90 - a) + w2*(200 - b), where a is the included angle between the traffic lights and the stop lines corresponding to each lane, b is the relative distance between the traffic lights and the stop lines of each lane, w1 is the weight, and w2 is the metric value. w1 and w2 can be calibrated according to actual needs. Optionally, w1 can be set to 1 and w2 can be set to 0.1.

[0089] Optionally, the coordinate information corresponding to the stop lines of each lane and the coordinate information of the projection of the traffic lights on the ground can be determined according to the lane information, and then the Euclidean distance between the coordinate information corresponding to the stop lines of each lane and the coordinate information of the projection of the traffic lights on the ground can be calculated to determine the relative distance between the traffic lights and the stop lines corresponding to each lane.

[0090] Please continue to refer to Figure 4 , step 460: Determine the target traffic light corresponding to the target lane and the recognition probability of the target traffic light according to the data information, the target driving direction, the feasible probability, the association relationship and the association probability.

[0091] As a way, after determining the association relationship and association probability between the traffic signal and each lane, based on the data information of the traffic signal, the drivable direction of the vehicle when entering the intersection where the traffic signal is located, and the association relationship between the traffic signal and each lane, determine the target traffic signal corresponding to the target lane where the vehicle is located, and determine the recognition probability of the target traffic signal according to the feasible probability and association probability corresponding to the feasible driving direction. In this way, the user can determine whether to control the vehicle to pass through the intersection corresponding to the traffic signal according to the indication information of the target traffic signal, ensuring the driving safety of the vehicle.

[0092] In some embodiments, the data information includes the traffic signal type and the traffic direction indicated by the traffic signal. For example, Figure 6 As shown, step 460 includes:

[0093] Step 461, determine a reference traffic signal group for controlling the driving direction of the vehicle in the target lane according to the association relationship.

[0094] As a way, a red light, a yellow light, and a green light can be used as a reference signal group. Therefore, first determine the reference traffic signal group for controlling the driving direction of the vehicle in the target lane according to the association relationship, so as to facilitate determining the target traffic signal in the traffic signal group.

[0095] Step 462, based on the traffic signal type and the target driving direction, determine the target traffic signal corresponding to the target lane in the reference traffic signal group.

[0096] As a way, after determining the reference traffic signal group corresponding to the lane where the vehicle is located, in order to determine whether the vehicle can currently pass through the intersection where the traffic signal is located, the target traffic signal can be determined in the reference traffic signal group according to the traffic signal type and the target driving direction. For example, if the target driving direction is a left turn, determine the target traffic signal as a green light in the left turn traffic signal group, so that the vehicle can smoothly pass through the intersection where the traffic signal is located when the left turn traffic signal is green.

[0097] Step 463, determine the recognition probability of the target traffic signal according to the feasible probability and the association probability.

[0098] As a way, a first probability weight corresponding to the feasible probability and a second probability weight corresponding to the association probability can be preset in advance, and then weighted calculation is performed according to the feasible probability, the first probability weight, the association probability, and the second probability weight to determine the recognition probability of the target signal light.

[0099] In this embodiment, by determining the association relationship and association probability between the traffic signal and each lane according to the lane information, road topology information, and data information, furthermore, the target traffic signal and the recognition probability of the target traffic signal can be determined based on the association relationship, association probability, feasible probability, association relationship, and association probability between the traffic signal and each lane, thereby improving the recognition accuracy of the target traffic signal.

[0100] Figure 7 This is a traffic signal recognition method shown in an embodiment of the present application. In this embodiment, the road topology information and lane information of the environment where the vehicle is located can be determined first according to the image sensing data obtained by the vehicle or the perceived road vector information, so as to determine the type and position relationship of the traffic signal according to the image sensing data. Then, the association relationship between the traffic signal and the road can be determined according to the roads entering and exiting the intersection, road positions, road orientations, the plane angle of the traffic signal device, and the position relationship of the traffic signal in the road topology information. Then, based on the type and direction of the traffic signal and the association relationship between the traffic signal and the road, the association probability between the traffic signal and the road can be determined. Moreover, the possible driving directions (left turn, straight, or right turn) after the vehicle enters the intersection corresponding to the traffic signal in the lane where the vehicle is located, and the feasible probability corresponding to the possible driving directions can be predicted through the number of lanes, lane relative relationships, road surface arrows, and lane signs in the environment where the vehicle is located. At the same time, the driving direction of the vehicle in front after entering the intersection corresponding to the traffic signal can be determined according to the driving trajectory of the vehicle in front of the vehicle. Then, the driving direction of the vehicle in front and the possible driving direction of the vehicle are used for consistency check to obtain the consistency check result, and the associated traffic lights of the lane where the vehicle is located are determined according to the consistency check result. Thus, the traffic signals with an association probability with this lane are determined according to the associated traffic lights of the lane where the vehicle is located, the association probability between the traffic signal and the road, and the association relationship between the traffic signal and the road, thereby determining the target traffic signal and the recognition probability of the target traffic signal.

[0101] Figure 8 This is a block diagram of a traffic signal recognition device shown in an embodiment of the present application. As Figure 8 shown, the traffic signal recognition device 500 includes: an information determination module 510, a possible driving direction prediction module 520, a consistency determination module 530, a target driving direction determination module 540, and a target traffic signal determination module 550.

[0102] An information determination module 510, configured to obtain sensing data collected by sensors of the vehicle itself, and determine lane information, road topology information, and traffic signal data information of the environment where the vehicle itself is located according to the sensing data, where the road topology information includes the entry and exit relationships between each lane and the intersections corresponding to the traffic signals; A travelable direction prediction module 520, configured to determine the travelable direction and the feasible probability of the travelable direction after the vehicle enters the intersection corresponding to the traffic signal according to the lane information; A consistency determination module 530, configured to perform a consistency check on the travel direction of the vehicle in front of the vehicle at the intersection corresponding to the traffic signal and the travelable direction, and obtain a consistency result; A target travel direction determination module 540, configured to determine the target lane and the target travel direction of the target lane of the vehicle at the intersection corresponding to the traffic signal according to the consistency result; A target traffic signal determination module 550, configured to determine the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the target travel direction, the feasible probability, the road topology information, and the data information.

[0103] In some embodiments, the lane information includes the number of lanes and the relative position relationship of the lanes; the travelable direction prediction module 520 includes: a determination sub-module, configured to determine whether there are lane signs and road surface arrows in the environment where the vehicle is located; A first travelable direction prediction sub-module, configured to, if it is determined that there are no lane signs and road surface arrows, determine the travelable direction and the feasible probability corresponding to the travelable direction after the vehicle enters the intersection corresponding to the traffic signal according to the number of lanes and the relative position relationship of the lanes; A second travelable direction prediction sub-module, configured to, if it is determined that there are lane signs and road surface arrows, determine the travelable direction and the feasible probability corresponding to the travelable direction after the vehicle enters the intersection corresponding to the traffic signal according to the travel direction indicated by the lane signs and road surface arrows.

[0104] In some embodiments, the consistency determination module 530 includes: a curvature determination sub-module, configured to obtain the travel trajectory of the vehicle in front, fit the travel trajectory to obtain a fitted curve, and determine the curvature of the fitted curve; A travel direction determination sub-module, configured to determine the travel direction of the vehicle in front after entering the intersection where the traffic signal is located according to the curvature; A consistency determination sub-module, configured to perform a consistency check on the travel direction and the travelable direction.

[0105] In some embodiments, the recognition device 500 of the traffic signal further includes: a first adjustment module, configured to, if the consistency result indicates that the driving direction is inconsistent with the drivable direction, obtain a probability adjustment value and adjust the feasible probability according to the probability adjustment value; or a second adjustment module, configured to, if the consistency result indicates that the driving direction is consistent with the drivable direction, not adjust the feasible probability.

[0106] In some embodiments, the target traffic signal determination module 550 includes: an association relationship determination sub-module, configured to determine the association relationship and association probability between the traffic signal and each lane according to the lane information, the road topology information, and the data information; a target traffic signal determination sub-module, configured to determine the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the data information, the target driving direction, the feasible probability, the association relationship, and the association probability.

[0107] In some embodiments, the data information includes the position information and planar angle of the traffic signal, where the planar angle is the included angle between the traffic signal and the stop line corresponding to each lane. The association relationship determination sub-module includes: a matching unit, configured to perform Hungarian matching according to the road topology information, the lane information, the position information, and the planar angle to determine a bipartite graph; an association relationship determination unit, configured to determine the association relationship between the traffic signal and each lane according to the bipartite graph; an association probability determination unit, configured to determine the relative distance between the traffic signal and the stop line corresponding to each lane according to the lane information and the position information, and determine the matching edge value of the bipartite graph according to the relative distance and the planar angle, and determine the matching edge value as the association probability.

[0108] In some embodiments, the data information includes the traffic signal type and the traffic direction indicated by the traffic signal. The target traffic signal determination sub-module includes: a reference traffic signal group determination unit, configured to determine a reference traffic signal group for controlling the driving direction of the vehicle in the target lane according to the association relationship; a target traffic signal determination unit, configured to determine the target traffic signal corresponding to the target lane in the reference traffic signal group based on the traffic signal type and the target driving direction; an identification probability determination unit, configured to determine the identification probability of the target traffic signal according to the feasible probability and the association probability.

[0109] According to one aspect of the embodiments of the present application, an electronic device is further provided, such as Figure 9As shown, the vehicle 600 includes a processor 610 and one or more memories 620. The one or more memories 620 are used to store program instructions executed by the processor 610. When the processor 610 executes the program instructions, the above-described traffic signal recognition method is implemented.

[0110] Further, the processor 610 may include one or more processing cores. The processor 610 runs or executes instructions, programs, code sets or instruction sets stored in the memory 620, and calls data stored in the memory 620. Optionally, the processor 610 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 610 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the display content; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor and may be implemented separately through a communication chip.

[0111] According to one aspect of the present application, the present application also provides a computer-readable storage medium. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist separately without being assembled into the electronic device. The above computer-readable storage medium carries computer-readable instructions. When the computer-readable storage instructions are executed by a processor, the method in any of the above embodiments is implemented.

[0112] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0113] The units described in the embodiments of the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not, in some cases, constitute a limitation on the unit itself.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0115] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application.

[0116] It should be understood that the present application is not limited to the exact structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A method for identifying traffic lights, characterized in that, the method includes: Obtain the sensing data collected by the vehicle's sensors, and determine the lane information, road topology information, and traffic light data information of the environment where the vehicle is located according to the sensing data, where the road topology information includes the entry and exit relationships between each lane and the intersections corresponding to the traffic lights; Determine the drivable direction and the feasible probability of the drivable direction after the vehicle enters the intersection corresponding to the traffic light according to the lane information; Perform a consistency check on the driving direction of the vehicle's leading vehicle at the intersection corresponding to the traffic light and the drivable direction to obtain a consistency result; Determine the target lane and the target driving direction of the target lane of the vehicle at the intersection corresponding to the traffic light according to the consistency result; Determine the target traffic light corresponding to the target lane and the recognition probability of the target traffic light according to the target driving direction, the feasible probability, the road topology information, and the data information.

2. The method according to claim 1, characterized in that, the lane information includes the number of lanes and the relative position relationship of the lanes; the step of determining the drivable direction and the feasible probability of the drivable direction after the vehicle enters the intersection corresponding to the traffic light according to the lane information includes: Determine whether there are lane signs and road surface arrows in the environment where the vehicle is located; If it is determined that there are no lane signs and road surface arrows, then determine the drivable direction and the feasible probability corresponding to the drivable direction after the vehicle enters the intersection corresponding to the traffic light according to the number of lanes and the relative position relationship of the lanes; If it is determined that there are lane signs and road surface arrows, then determine the drivable direction and the feasible probability corresponding to the drivable direction after the vehicle enters the intersection corresponding to the traffic light according to the traveling directions indicated by the lane signs and the road surface arrows.

3. The method according to claim 1, characterized in that, the step of performing a consistency check on the driving direction of the vehicle's leading vehicle at the intersection corresponding to the traffic light and the drivable direction to obtain a consistency result includes: Obtain the driving trajectory of the leading vehicle, fit the driving trajectory to obtain a fitting curve, and determine the curvature of the fitting curve; Determine the driving direction of the leading vehicle after entering the intersection where the traffic light is located according to the curvature; Perform a consistency check on the driving direction and the drivable direction.

4. The method according to claim 1, characterized in that, after performing a consistency check on the driving direction of the vehicle's leading vehicle at the intersection corresponding to the traffic light and the drivable direction to obtain a consistency result, it further includes: If the consistency result indicates that the driving direction and the drivable direction are inconsistent, then obtain a probability adjustment value, and adjust the feasible probability according to the probability adjustment value; or If the consistency result indicates that the driving direction and the drivable direction are consistent, then do not adjust the feasible probability.

5. The method according to any one of claims 1-4, characterized in that determining the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the target driving direction, the feasible probability, the road topology information, and the data information includes: determining the association relationship and association probability between the traffic signal and each lane according to the lane information, the road topology information, and the data information; determining the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the data information, the target driving direction, the feasible probability, the association relationship, and the association probability.

6. The method according to claim 5, characterized in that the data information includes the position information and the plane angle of the traffic signal, where the plane angle is the included angle between the traffic signal and the stop line corresponding to each lane, and determining the association relationship and association probability between the traffic signal and each lane according to the lane information, the road topology information, and the data information includes: performing Hungarian matching according to the road topology information, the lane information, the position information, and the plane angle to determine a bipartite graph; determining the association relationship between the traffic signal and each lane according to the bipartite graph; determining the relative distance between the traffic signal and the stop line corresponding to each lane according to the lane information and the position information, determining the matching edge value of the bipartite graph according to the relative distance and the plane angle, and determining the matching edge value as the association probability.

7. The method according to claim 5, characterized in that the data information includes the traffic signal type and the traffic direction indicated by the traffic signal, and determining the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the data information, the target driving direction, the feasible probability, the association relationship, and the association probability includes: determining a reference traffic signal group for controlling the driving direction of the vehicle in the target lane according to the association relationship; determining the target traffic signal corresponding to the target lane in the reference traffic signal group based on the traffic signal type and the target driving direction; determining the recognition probability of the target traffic signal according to the feasible probability and the association probability.

8. An identification device for a traffic signal, characterized in that the device includes: an information determination module, configured to obtain sensing data collected by a sensor of the vehicle itself, and determine lane information, road topology information, and data information of a traffic signal in the environment where the vehicle itself is located, where the road topology information includes the access relationship between each lane and the intersection corresponding to the traffic signal; a drivable direction prediction module, configured to determine the drivable direction and the feasible probability of the drivable direction after the vehicle enters the intersection corresponding to the traffic signal according to the lane information; A consistency determination module, configured to perform a consistency check on the driving direction of the vehicle in front of the host vehicle at the intersection corresponding to the traffic signal and the drivable direction, so as to obtain a consistency result; A target driving direction determination module, configured to determine the target lane of the host vehicle at the intersection corresponding to the traffic signal and the target driving direction of the target lane according to the consistency result; A target traffic signal determination module, configured to determine the target traffic signal corresponding to the target lane and the recognition probability of the target traffic signal according to the target driving direction, the feasible probability, the road topology information, and the data information.

9. An electronic device, characterized in that, the electronic device includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that, program code is stored in the computer-readable storage medium, and the program code can be called by a processor to execute the method according to any one of claims 1 to 7.

Images