Traffic signal determination method and apparatus, vehicle, device, and storage medium

Abstract

The present disclosure provides a traffic signal determination method and device, a vehicle, equipment and a storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of automatic driving and intelligent traffic. The specific implementation scheme is: determining the change information of a signal light group according to real scene map data and high-precision map data; determining the virtual light signal corresponding to the signal light group according to the change information; and determining the target traffic signal indicated by a target signal light in the signal light group according to the virtual light signal and the high-precision map data. The present disclosure combines real scene image data and high-precision map data to determine the change information of the signal light group, effectively avoids the inaccuracy of high-precision map data caused by the change of the signal light group, can timely find the change of the signal light group, improves the accuracy of the virtual light signal formed, thereby improving the accuracy of the target traffic signal and the safety of vehicle driving.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to the fields of artificial intelligence, autonomous driving, and intelligent transportation, and in particular to methods, devices, vehicles, equipment, and storage media for determining traffic signals. Background Technology

[0002] In the field of autonomous driving, when driving at real intersections, cars must strictly abide by traffic regulations and make subsequent driving decisions based on the color information of the traffic lights at the current intersection. Summary of the Invention

[0003] This disclosure provides a method, apparatus, vehicle, device, and storage medium for determining traffic signals, effectively improving the accuracy of traffic signal recognition in autonomous driving.

[0004] According to a first aspect of this disclosure, a method for determining traffic signals is provided, comprising:

[0005] Based on real-scene image data and high-precision map data, determine the change information of the traffic light group;

[0006] Determine the virtual light signal corresponding to the traffic light group based on the change information;

[0007] Based on virtual light signals and high-precision map data, determine the target traffic signal indicated by the target signal light in the signal light group.

[0008] According to a second aspect of this disclosure, a traffic signal determining device is provided, comprising:

[0009] The first determining module is configured to determine the change information of the traffic light group based on real-scene image data and high-precision map data;

[0010] The second determining module is configured to determine the virtual light signal corresponding to the signal light group based on the change information;

[0011] The third determination module is configured to determine the target traffic signal indicated by the target traffic light in the traffic light group based on the virtual light signal and high-precision map data.

[0012] According to a third aspect of this disclosure, a vehicle is provided, comprising:

[0013] An autonomous driving system is configured to perform the method provided in the first aspect, or to include the means provided in the second aspect.

[0014] According to a fourth aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory that is communicatively connected to at least one processor; wherein,

[0017] The memory stores instructions that can be executed by at least one processor, such that the instructions are executed by at least one processor to enable the at least one processor to perform the method provided in the first aspect.

[0018] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the method provided in the first aspect.

[0019] According to a sixth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to the first aspect.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 This is an exemplary system architecture diagram of the traffic signal determination method that can be applied to this disclosure;

[0023] Figure 2 This is a schematic diagram of a first embodiment of the traffic signal determination method according to the present disclosure;

[0024] Figure 3 This is a schematic diagram of a second embodiment of the traffic signal determination method according to the present disclosure;

[0025] Figure 4 This is a schematic diagram of one embodiment of the traffic signal determination device according to the present disclosure;

[0026] Figure 5 This is a block diagram of an electronic device used to implement the traffic signal determination method of the embodiments of this disclosure. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] In current high-level autonomous driving systems, the planning and control module determines the target traffic light that the driver should pay attention to based on a high-precision map. Then, the perception module detects and identifies the status information of the target traffic light and sends it to the planning and control module to assist driving decisions.

[0029] In reality, high-definition maps are not entirely reliable. For example, traffic lights at intersections may malfunction, be added, removed, or have their semantics changed, and the high-definition map may not be updated in a timely manner. If the high-definition map is incorrectly labeled, or if the labeling information on the high-definition map does not match the real-world scene due to construction, traffic light damage, or other factors, the control module may use the wrong target traffic light, potentially leading to red-light running, traffic congestion, and other issues.

[0030] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0031] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the traffic signal determination method or traffic signal determination apparatus of the present disclosure can be applied.

[0032] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102, and a server 103. The network 102 provides a communication link between the terminal device 101 and the server 103, and may include various connection types, such as wired communication links, wireless communication links, or fiber optic cables. For example, the server 103 may be a cloud server or a local server that communicates with the terminal device.

[0033] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send information, etc. Various client applications can be installed on terminal device 101, such as social applications, travel applications, map applications, etc.

[0034] Terminal device 101 can be hardware or software. When terminal device 101 is hardware, it can be various electronic devices, including but not limited to independent electronic devices such as smartphones, tablets, laptops, and desktop computers, as well as vehicle-mounted terminals and other vehicle-attached electronic devices. When terminal device 101 is software, it can be installed in the aforementioned electronic devices. It can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are made here.

[0035] Server 103 can be either hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0036] The traffic signal determination method provided in this embodiment is generally executed by the terminal device 101, and correspondingly, the traffic signal determination device is generally installed in the terminal device 101.

[0037] It should be noted that, Figure 1 The number of terminal devices 101, network 102, and server 103 shown is merely illustrative. Depending on implementation needs, any number of terminal devices 101, network 102, and server 103 can be included.

[0038] Figure 2 A flowchart 200 illustrating an embodiment of the traffic signal determination method according to the present disclosure is shown, with reference to... Figure 2 As shown, the method for determining this traffic signal includes the following steps:

[0039] Step S201: Determine the change information of the traffic light group based on the real-scene image data and high-precision map data.

[0040] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle-mounted terminal or a control device in the vehicle-mounted terminal) determines the change information of the traffic light group based on real-scene image data and high-precision map data.

[0041] The real-scene image data is obtained by the executing entity through onboard sensing devices to capture images of the environment surrounding the vehicle while it is in motion. For example, the executing entity uses an onboard camera to capture images of the actual environment in front of the vehicle at a preset acquisition frequency to obtain real-scene image data.

[0042] For example, real-world image data can be in the form of independent images (e.g., pictures), or in the form of a continuous sequence of images, or in the form of a video composed of a continuous sequence of image frames, without limitation.

[0043] High-precision map data refers to the high-precision map data used for vehicle positioning and trajectory planning during autonomous driving, such as high-precision map data that has been loaded or stored on the vehicle terminal.

[0044] In this embodiment of the disclosure, a traffic light group refers to traffic lights installed on roads (especially at intersections). For example, a traffic light group may include traffic lights for each lane, traffic lights for pedestrian crossings to indicate pedestrian crossing, and traffic lights for timing or counting.

[0045] For example, the traffic light corresponding to each lane may only display color, or it may display both color and numbers, or it may display both color and shape. The colors of the traffic light may include red and green alternating according to a preset pattern, or green, yellow, and red alternating according to a preset time pattern. The shape of the traffic light includes its external shape, such as a circle or rectangle; it may also include the shape of its internal directional arrows, such as left-turn arrows, straight-ahead arrows, U-turn arrows, or right-turn arrows.

[0046] In real-world environments, traffic light systems at intersections may change due to construction, malfunctions, or other reasons, causing discrepancies between the actual traffic signals displayed and the information labeled in the high-precision map data. For example, traffic lights may be deleted, new traffic lights may be added, traffic lights that previously only displayed color may be modified to include directional arrows (i.e., semantic changes), or traffic lights may malfunction and fail to display anything.

[0047] In some alternative implementations, the executing entity can determine the vehicle's current position and orientation based on the vehicle's positioning information, then determine the traffic light group corresponding to the current position and orientation from high-precision map data, and obtain the annotation information of the traffic light group; identify the status information of the traffic light group from real-scene image data, and then compare the annotation information and status information to determine the change information of the traffic light group.

[0048] For example, the status information of the traffic light group identified by the executing entity from the real-scene image data may include at least one of the following: the color of any traffic light in the traffic light group, whether it is obstructed, whether it is flashing, whether it is counting down, whether it includes a pattern (e.g., a pattern of a driving direction arrow, a pattern of a pedestrian or vehicle, a pattern of a counting number, etc.).

[0049] In some optional implementations, the executing entity determines the vehicle's current driving position, road orientation, and corresponding traffic light group from high-precision map data based on the vehicle's positioning information and real-scene image data, and obtains the annotation information of the traffic light group; the executing entity also identifies and analyzes the real-scene image data to obtain the traffic light group corresponding to the current orientation in the real-scene image data, and determines the status information of the traffic light group in the current real-scene image data; then the executing entity compares the annotation information and status information of the traffic light group to determine the change information of the traffic light group.

[0050] The change information for the traffic light group includes whether each traffic light in the group has changed, and if so, the type and result of the change. For example, when a traffic light has changed, the change information may include at least one of the following: the identifier of the changed traffic light, the type of change (e.g., deletion, addition, semantic modification), the semantics of the changed traffic light (e.g., color, shape), and its semantic priority.

[0051] In some optional implementations, the aforementioned execution entity can project the annotation information obtained from the high-precision map data onto the real-scene image data based on the correspondence between the world coordinate system of the high-precision map data and the image coordinate system of the real-scene image data. Then, it can determine the consistency between the annotation information and the status information of the traffic light group from the real-scene image data, thereby determining the change information of the traffic light group.

[0052] In some alternative implementations, the aforementioned execution entity can also convert the state information identified from the real-scene image data into real-scene information in the world coordinate system based on the correspondence between the image coordinate system and the world coordinate system of the real-scene image data. Then, it can compare the real-scene information with the annotation information obtained from the high-precision map data to determine their consistency, thereby determining the change information of the traffic light group.

[0053] In some optional implementations, the aforementioned execution entity can also convert the annotation information into first description information and the status information into second description information according to a preset description template. Then, by comparing the consistency of the first description information and the second description information in the same module of the description template, the change information of the traffic light group can be determined.

[0054] For example, the preset description template can be in any of the following forms: table form, text form, chart form (e.g., scatter plot), etc.

[0055] Taking a table format as an example, the horizontal header of the preset description target can include each signal light in the signal light group, such as the first signal light, the second signal light, the third signal light, and so on; the vertical header can include descriptive information such as the shape, color, and whether the signal light is faulty. The executing entity can determine the consistency of the status information and the annotation information simply by comparing the consistency of the contents of the same cells in the tables corresponding to the first and second description information of the converted route, thereby determining the change information of the signal light group.

[0056] Step S202: Determine the virtual light signal corresponding to the signal light group based on the change information.

[0057] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle terminal or a control device in the vehicle terminal) determines the virtual light signal corresponding to the signal light group based on the change information of the signal light group.

[0058] Because the traffic light groups and their traffic signal indication methods at different intersections are not entirely consistent—for example, some have directional arrows while others do not, some have one traffic light that simultaneously represents both "left turn" and "straight ahead" traffic signals, while in others "left turn" is indicated by a separate traffic light—accurate recognition of traffic signals for all types of traffic light groups requires setting corresponding recognition rules for different types of traffic light groups. This operation is quite complex and prone to errors.

[0059] In theory, a lane at an intersection can have four possible directions of travel: U-turn, left turn, straight ahead, and right turn. This means that the traffic signal indicated by the traffic light for each lane can be at least one of these four directions. Therefore, four virtual lights representing these four directions can be configured for each lane, thus unifying the traffic signal rules for all lanes at all intersections. In other words, for any intersection's traffic light group, based on the traffic signal indicated by the traffic light for each lane in that group, the corresponding virtual light is selected from the four virtual lights for U-turn, left turn, straight ahead, and right turn, and the corresponding virtual light signal is obtained, thus generating the virtual light signal for the intersection's traffic light group.

[0060] In this embodiment of the present disclosure, the execution entity abstracts and fits the traffic light group of each intersection according to the driving direction of its corresponding lane, and abstracts and fits the traffic signal indicated by the traffic light corresponding to each lane according to the driving direction of the lane into at least one of four virtual light signals: U-turn, left turn, straight, and right turn, thereby obtaining the virtual light signal of the lane, and then obtaining the virtual light signal corresponding to the traffic light group of the intersection.

[0061] Based on this virtual traffic light signal, combined with information such as the lane line type and stop line position of each lane at the intersection, the executing entity can precisely control the autonomous driving of vehicles, simplify the traffic signal recognition process, improve the accuracy of traffic signal recognition, and thus enhance the safety of autonomous driving.

[0062] In this embodiment of the disclosure, the execution entity determines whether any traffic lights in the traffic light group have changed based on the change information of the traffic light group. For example, if the change information is empty, it means no traffic light has changed; or, for example, a traffic light has been deleted or its semantics have been changed. Then, the execution entity determines the virtual traffic light corresponding to the traffic light group based on the change information.

[0063] In some optional implementations, if there is no change in the traffic lights, the executing entity can directly determine the virtual light signal based on the annotation information of the traffic light group in the high-precision map data, or it can perform signal tracking on the traffic light group in the real scene image data and determine the virtual light signal corresponding to the traffic light group.

[0064] In some optional implementations, if there is a change in the traffic light, the executing entity can determine whether the target changing light is referable based on the type of change, thereby determining whether to use the signal corresponding to the target changing light when determining the virtual light signal, and then select the virtual light signal corresponding to the traffic light group based on real-scene image data and / or high-precision map data.

[0065] For example, when the executing entity determines that the target changing light is highly referential, such as when the target changing light is a newly added light and has a high semantic priority, it can determine that it is necessary to combine the signal of the newly added light to generate a virtual light signal. This involves determining the tracking information of the newly added light from the real-scene image data and combining it with the annotation information of other unchanged lights in the high-precision map data to generate the virtual light signal corresponding to the signal light group. Alternatively, the executing entity can directly determine the tracking information of the signal light group from the real-scene image data and determine the corresponding virtual light signal accordingly.

[0066] For example, if the executing entity determines that the change type of the target changing light is deletion, that is, determines that the change result has no reference significance for the current autonomous driving, it can directly generate the corresponding virtual light signal based on the annotation information of the signal light group in the high-precision map, or it can directly identify the tracking information of the signal light group from the real scene image data to generate the corresponding virtual light signal.

[0067] Step S203: Based on the virtual light signal and high-precision map data, determine the target traffic signal indicated by the target light in the signal light group.

[0068] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle-mounted terminal or a control device in the vehicle-mounted terminal) determines the target traffic signal indicated by the target traffic light in the traffic light group based on the virtual light signal obtained in step S2020 and in combination with high-precision map data.

[0069] For example, the target signal light in the signal light group can be the target signal light corresponding to the lane where the vehicle is currently located, that is, the target signal light indicating the driving direction and start / stop signal corresponding to the lane where the vehicle is currently located.

[0070] In some alternative implementations, the aforementioned executing entity can annotate the virtual light signal into the high-precision map data, forming part of the signal light group annotation information in the updated high-precision map data; and then determine the target traffic signal indicated by the target signal light in the signal light group based on the updated high-precision map data.

[0071] In some alternative implementations, the aforementioned execution entity, based on the virtual light signal and combined with the lane information (such as lane attributes, stop line position, etc.) in the high-precision map data, determines the target virtual light signal corresponding to the current lane from the virtual light signal and uses it as the traffic signal information indicated by the target traffic light.

[0072] According to the traffic signal determination method provided in this disclosure, the executing entity combines real-scene image data and high-precision map data to determine the change information of the traffic light group, and determines the virtual light signal corresponding to the traffic light group based on the change information, thereby determining the target traffic signal indicated by the target traffic light in the traffic light group. This disclosure uses a combination of real-scene image data and high-precision map data to determine the change information of the traffic light group, effectively avoiding inaccuracies in the high-precision map data caused by changes in the traffic light group. It can promptly detect changes in the traffic light group, improve the accuracy of the formed virtual light signal, thereby improving the accuracy of the target traffic signal and enhancing vehicle driving safety.

[0073] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0074] Figure 3 A schematic diagram of a second implementation process 300 of the traffic signal determination method provided in this disclosure is shown, with reference to... Figure 3 As shown, the method for determining this traffic signal includes the following steps:

[0075] Step S301: Determine the labeling information of the traffic light group based on the real-scene image data and high-precision map data.

[0076] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle-mounted terminal or a control device in the vehicle-mounted terminal) determines the labeling information of the traffic light group based on real-scene image data and high-precision map data.

[0077] The real-scene image data is obtained by the executing entity through onboard sensing devices to capture images of the environment surrounding the vehicle while it is in motion. For example, the executing entity uses an onboard camera to capture images of the actual environment in front of the vehicle at a preset acquisition frequency to obtain real-scene image data.

[0078] For example, real-world image data can be in the form of independent images (e.g., pictures), or in the form of a continuous sequence of images, or in the form of a video composed of a continuous sequence of image frames, without limitation.

[0079] High-precision map data refers to the high-precision map data used for vehicle positioning and trajectory planning during autonomous driving, such as high-precision map data that has been loaded or stored on the vehicle terminal.

[0080] It should be noted that the signal light group mentioned in this plan refers to the signal light group corresponding to the current driving direction of the vehicle, that is, the signal light group directly in front of the vehicle at its current position.

[0081] For example, the aforementioned executing entity can combine real-scene image data and high-precision map data to determine the traffic light group corresponding to the vehicle's current position orientation, and then obtain the annotation information of the traffic light group from the high-precision map data.

[0082] In some optional implementations of the embodiments of this disclosure, determining the labeling information of the traffic light group based on real-scene image data and high-precision map data includes: determining the traffic light group facing the current position based on real-scene image data and high-precision map data; and obtaining the labeling information of the traffic light group from the high-precision map data.

[0083] In this implementation, the aforementioned execution entity can first determine the current intersection location of the vehicle based on high-precision map data and the vehicle's positioning information; combine real-scene image data to help determine the orientation information of the vehicle's current location and the traffic light group corresponding to that orientation information; and then obtain the labeling information of the traffic light group from the high-precision map data.

[0084] In this implementation, the executing entity determines the vehicle's current position based on high-precision map data and uses real-scene image data to help determine the traffic light group corresponding to the orientation of the current position. This can effectively improve the positioning accuracy of the traffic light group, thereby improving the accuracy of the acquired annotation information and avoiding inaccurate annotation information due to the influence of positioning signals or positioning data. This effectively ensures the accuracy of the subsequent determination of changes in the traffic light group.

[0085] Step S302: Determine the change information of the traffic light group based on the annotation information and the status information of the traffic light group in the real scene image data.

[0086] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1The terminal device 101 shown (e.g., a vehicle terminal or a control device in the vehicle terminal) determines the change information of the signal light group based on the labeling information of the signal light group obtained in step S301 and the status information of the signal light group in the real scene image data.

[0087] For example, the aforementioned execution entity identifies the traffic light group in the real-scene image data, determines the status information of the traffic light group in the real scene, and then compares the status information with the annotation information to determine the change information of the traffic light group.

[0088] For example, the status information of the traffic light group identified by the executing entity from the real-scene image data may include at least one of the following: the color of any traffic light in the traffic light group, whether it is obstructed, whether it is flashing, whether it is counting down, whether it includes a pattern (e.g., a pattern of a driving direction arrow, a pattern of a pedestrian or vehicle, a pattern of a counting number, etc.).

[0089] In some alternative implementations, the aforementioned execution entity can project the annotation information onto the real-scene image data based on the correspondence between the world coordinate system of the high-precision map data and the image coordinate system of the real-scene image data, and then determine the consistency between the annotation information and the status information from the real-scene image data, thereby determining the change information of the traffic light group.

[0090] In some alternative implementations, the aforementioned execution entity can also convert the state information identified from the real-scene image data into real-scene information in the world coordinate system based on the correspondence between the image coordinate system and the world coordinate system of the real-scene image data. Then, it can compare the real-scene information with the annotation information to determine their consistency, thereby determining the change information of the traffic light group.

[0091] In some optional implementations, the aforementioned execution entity can also convert the annotation information into first description information and the status information into second description information according to a preset description template (such as any of the following forms of description template: table form, text form, chart form, etc.). Then, by comparing the consistency of the first description information and the second description information in the same module of the description template, the change information of the traffic light group can be determined.

[0092] In this embodiment of the disclosure, the executing entity determines the corresponding traffic light group in the high-precision map data based on high-precision map data and real-scene image data, thereby improving the positioning accuracy of the traffic light group and thus improving the accuracy of the acquired annotation information. Furthermore, the annotation information is combined with the state information determined in the real-scene image data to accurately determine the change information of the traffic light group, thereby improving the accuracy of the obtained change information, and thus improving the accuracy of the autonomous driving control basis and the safety of autonomous driving.

[0093] In some optional implementations of the embodiments of this disclosure, determining the change information of the traffic light group based on the annotation information and the status information of the traffic light group in the real scene image data includes: identifying the status information of the traffic light group in the real scene image data; projecting the annotation information onto the real scene image data to obtain projection information; and comparing the status information and the projection information to determine the change information of the traffic light group.

[0094] In this implementation, the aforementioned execution entity first identifies the status information of the traffic light group in the real-scene image data. For example, it identifies the number of traffic lights in the traffic light group, the shape and position of each traffic light, the color of each traffic light, the meaning of each traffic light (e.g., driving direction, timing, pedestrian or vehicle, etc.), whether each traffic light includes a directional arrow, and the driving direction represented by the included directional arrow, etc.

[0095] For example, after identifying the status information of the traffic light group, the executing entity can mark it in the real-scene image data. For instance, the position and shape of each traffic light (including its outline and meaning shape, such as timing numbers, directional arrows, etc.) can be graphically marked at its respective position using the identified outline. As another example, the color of each traffic light can be indicated by the boundary line color of the outline graphic or the fill color within the graphic.

[0096] Then, the aforementioned executing entity projects the annotation information onto the real-scene image data according to the correspondence between the world coordinate system and the image coordinate system in the real-scene image data, thereby obtaining the corresponding projection information. For example, the projection information can be marked in the same or similar way as the state information, such as using the same principle but with different contour line types or different color relationships.

[0097] The executing entity can compare the status information and annotation information in the real-scene image data based on the marking results to obtain the change information of the traffic light group.

[0098] In this implementation, the executing entity projects the labeling information corresponding to the traffic light group onto the real-scene image data and compares it with the identified status information. By comparing the two types of information in the same data environment, the comparison result can be accurately and intuitively determined, and the change information of the traffic light group can be obtained, ensuring the accuracy of the change information.

[0099] In some optional implementations of the embodiments of this disclosure, the annotation information includes at least one of the following: marker coordinate information, marker shape information, and marker color information; the status information includes at least one of the following: illustration position information, illustration shape information, and illustration color information.

[0100] The marked shape information includes marking information for the outer shape and internal shape of each traffic light in the traffic light group. Correspondingly, the illustrated shape information may include illustrated information for the outer shape and internal shape of each traffic light in the traffic light group.

[0101] For example, the labeling information may also include label quantity information, that is, labeling information for the number of traffic lights in the traffic light group; correspondingly, the status information may also include graphic quantity information, that is, graphic information for the number of traffic lights in the traffic light group. For example, it can be marked by numerical labels or graphic quantities (e.g., 3 traffic lights using 3 solid circles or 3 stars).

[0102] Step S303: Based on the high-precision map data, determine the target traffic light in the traffic light group.

[0103] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle-mounted terminal or a control device in the vehicle-mounted terminal) determines the target traffic light in the traffic light group corresponding to the current location based on high-precision map data.

[0104] In some alternative implementations, the executing entity determines the current lane of the current location based on high-precision map data and the vehicle's positioning information, and then determines the target traffic light corresponding to the current location from the traffic light group based on the current lane.

[0105] In some optional implementations, the executing entity determines the vehicle's current driving direction on the planned trajectory based on high-precision map data, combined with the vehicle's positioning information and planned trajectory. Based on the current location and its driving direction, it determines the target traffic light corresponding to the current location from the traffic light group.

[0106] In some alternative implementations, the executing entity can also determine the location and orientation information of the positioning information in the high-precision map data based on the vehicle's planned trajectory and / or positioning information, and then determine the target traffic light in the traffic light group corresponding to the current location directly based on the binding relationship between the traffic light group and the high-precision map data.

[0107] Step S304: Based on the change information, determine whether the target traffic light has changed.

[0108] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle-mounted terminal or a control device in the vehicle-mounted terminal) determines whether the target traffic light has changed based on the change information of the traffic light group, that is, whether the change information of the traffic light group includes the change information of the target traffic light.

[0109] The executing entity determines whether the target traffic light has changed by checking whether the change information of the traffic light group includes the change information of the target traffic light.

[0110] For example, if the change information is empty, that is, no change has occurred in the traffic light group, it can be directly determined that the target traffic light has not changed.

[0111] For example, if the change information is not empty and there is no change record for the target traffic light, it is determined that the target traffic light has not changed; if the change information is not empty and includes a change record for the target traffic light, such as the target traffic light being deleted, malfunctioning, or having its semantics changed, it is determined that the target traffic light has changed.

[0112] Step S305: In response to a change in the target traffic light, determine the virtual light signal of the traffic light group based on the tracking information of the traffic light group in the real-scene image data.

[0113] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle-mounted terminal or a control device in the vehicle-mounted terminal) determines the virtual light signal corresponding to the signal light group based on the tracking information of the signal light group in the real-scene image data after determining that the target signal light has changed according to the change information.

[0114] When the aforementioned enforcement entity determines that the change information includes changes to the target traffic light, it confirms that the target traffic light has changed. In this case, the enforcement entity confirms that the high-precision map data is incorrect or has not been updated in a timely manner. Therefore, the traffic signal indicated by the current traffic light group needs to be identified based on real-scene image data. Relying solely on high-precision map data may lead to traffic violations and accidents.

[0115] The executing entity tracks the signal changes of the traffic lights in the real-world image data to obtain the tracking information of the traffic lights. Then, it fits the tracking information to determine the virtual light signal corresponding to the traffic lights.

[0116] In some alternative implementations, the execution entity can simultaneously track the signal of each traffic light in the traffic light group, and after denoising and smoothing the tracked signals, fuse them to obtain the virtual light signal corresponding to the traffic light group.

[0117] For example, the executing entity identifies and tracks the status information of traffic light groups in real-world image data of consecutive frames, determines the indication direction, color change sequence, etc. of each traffic light, obtains the signal sequence corresponding to each traffic light, and fuses them to obtain the virtual light signal corresponding to the traffic light group.

[0118] If the executing entity detects noise in the signal sequence corresponding to a certain traffic light, for example, if there are consecutive red signal sequences before and after a certain green signal, then the green signal can be identified as the noise signal and can be filtered out. The consecutive red signals before and after the green signal can be smoothed to obtain the processed signal sequence.

[0119] For example, the driving direction indicated by the virtual light signal may include four types: U-turn, left turn, straight, and right turn.

[0120] In this embodiment of the disclosure, when the executing entity determines that the current traffic light group has changed relative to the annotation information in the high-precision map data, it further determines whether the current target traffic light has changed. If the target traffic light has changed, it generates a virtual light signal of the traffic light group by fusing the tracking information of the traffic light group in the real scene image data.

[0121] In this embodiment of the present disclosure, when the target traffic light on which the current driving basis is based changes, the executing entity tracks the real-scene image data to improve the real-time performance and accuracy of the virtual light signal, avoid signal errors caused by over-reliance on high-precision map data, and ensure driving safety.

[0122] Step S306: In response to the fact that the target traffic light has not changed and the change information is not empty, determine the virtual light signal of the traffic light group according to the change type in the change information.

[0123] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle terminal or a control device in the vehicle terminal) determines the data basis of the virtual light signal of the signal light group based on the change type in the change information when it is determined that the target signal light has not changed and the change information is not empty, and then fuses the virtual light signal of the signal light group accordingly.

[0124] If the executing entity determines that the target traffic light has not changed, but other traffic lights have changed, it determines the type of change of the changed traffic light. Then, based on the type of change, it determines the degree of impact of the changed traffic light on the traffic light group, and accordingly determines whether it is necessary to group the individual traffic lights in the traffic light group, and determines the virtual light signal of the traffic light group based on the grouping result.

[0125] For example, if a traffic light other than the target traffic light is deleted, it is determined that the deleted traffic light has a low impact on the traffic light group and can be ignored. The virtual light signal can be determined only based on the signals of the existing traffic lights.

[0126] For example, if a new traffic light is added in addition to the target signal, the signal of the new traffic light needs to be considered to determine whether it needs to be combined with the previous traffic light group to determine the virtual light signal of the traffic light group.

[0127] In this embodiment, when the execution entity determines that the target traffic light has not changed and the change information is not empty, it determines the virtual light signal for the traffic light group based on the change type in the change information, i.e., the change type of the changed traffic light. The execution entity then determines the degree of influence of the changed traffic lights on the traffic light group based on the change type of the changed traffic lights other than the target traffic light, thereby determining whether to use the signal of the changed traffic light when generating the virtual light signal, thus improving the accuracy and reliability of the virtual light signal.

[0128] In some optional implementations of the embodiments of this disclosure, the change type in the change information includes at least one of the following: adding a traffic light, changing the semantic meaning of a traffic light; and determining the virtual light signal of the traffic light group based on the change type in the change information includes: in response to the change type in the change information being a adding or changing traffic light semantics, determining the semantic priority of the changed traffic light in the change information; in response to the semantic priority of the changed traffic light being higher than the semantic priority of the target traffic light, determining the virtual light signal of the traffic light group based on tracking information; in response to the semantic priority of the changed traffic light being lower than or equal to the semantic priority of the target traffic light, determining the virtual light signal of the traffic light group based on tracking information and high-precision map data.

[0129] The addition of a traffic light results in the creation of at least one new traffic light in the existing traffic light group. Examples include adding a timing signal light or a turn signal light. Semantic changes to traffic lights refer to altering the semantic information of the original traffic lights while keeping their number and location unchanged. For example, changing a traffic light without directional arrows to one with directional arrows; or changing a traffic light without pictorial content to a timing signal light with numerical patterns.

[0130] After determining that the change type in the change information is a new traffic light or a change in the semantics of a traffic light, the aforementioned executing entity obtains the changed traffic light in the change information and determines the semantic priority of the changed traffic light.

[0131] For example, the semantic priority of different traffic lights can be preset. For instance, the semantic priority of a traffic light with a directional arrow is higher than that of a traffic light without a directional arrow.

[0132] The executing entity sorts the traffic lights according to a preset semantic priority, comparing the semantic priority of the changing traffic light with that of the target traffic light. For example, if the target traffic light is an existing traffic light without a directional arrow, and the changing traffic light is a traffic light with a left-turn arrow added next to the target traffic light, then the semantic priority of the changing traffic light is determined to be higher than that of the target traffic light. At this point, the executing entity determines that the signal of the changing traffic light has a significant impact on the virtual light signal of the traffic light group. Therefore, it treats all traffic lights in the traffic light group, including the changing traffic light, as a whole, tracks the signal of each traffic light within this whole from the real-scene image data, obtains the corresponding tracking information, and uses this information to fuse and generate the virtual light signal of the traffic light group.

[0133] If the executing entity determines that the semantic priority of the changed traffic light is lower than or equal to that of the target traffic light—for example, if the changed traffic light is a newly added traffic light with a numerical pattern next to the original traffic light with an directional arrow pattern—the executing entity determines that the signal of the changed traffic light can be ignored, and only the signals of the original traffic lights in the traffic light group are tracked. That is, the executing entity determines to form virtual traffic light signals based on the existing traffic lights. In this case, the executing entity determines the correlation between each traffic light and the target traffic light based on the annotation information in the high-precision map data, groups the traffic lights in the traffic light group accordingly, obtains the corresponding virtual signals based on the grouping results, and then merges them to obtain the virtual traffic light signal for the entire traffic light group.

[0134] In this implementation, when the executing entity determines that a new or semantically changed traffic light has been added to a traffic light other than the target traffic light, it compares the semantic priority of the changed traffic light with that of the target traffic light. Based on the semantic priority, it determines whether to track the signal of the changed traffic light, i.e., whether the virtual traffic light signal includes the signal of the changed traffic light. This solution can minimize the amount of data processing during signal tracking and fusion while ensuring the accuracy and reliability of the generated virtual traffic light signal, thereby improving the efficiency of virtual traffic light signal generation.

[0135] In some optional implementations of the embodiments of this disclosure, the change type in the change information may also include traffic light deletion or traffic light failure, and the virtual light signal of the traffic light group is determined according to the change type in the change information, including: in response to the change type in the change information being traffic light deletion, determining the virtual light signal of the traffic light group based on tracking information and high-precision map data.

[0136] In this implementation, the executing entity determines the change type in the change information to be traffic light deletion or traffic light malfunction. That is, a traffic light in the traffic light group other than the target traffic light has been deleted or has malfunctioned. The executing entity determines that the signal presented by the changed traffic light in the traffic light group is an invalid signal. In other words, even if the signal of the changed traffic light is traced, a valid signal cannot be found.

[0137] At this point, the executing entity determines that the signal of the changed traffic light can be ignored, that is, the change information of the traffic light group can be ignored, and signal tracking will still be performed based on the original traffic lights in the traffic light group. In other words, the executing entity determines to form virtual light signals based on the original traffic lights. In this case, the executing entity determines the correlation between each traffic light and the target traffic light based on the annotation information in the high-precision map data, groups the traffic lights in the traffic light group accordingly, obtains the corresponding virtual signals based on the grouping results, and then merges them to obtain the virtual light signal of the traffic light group.

[0138] In this implementation, when the executing entity determines that a traffic light other than the target traffic light is unable to generate a valid signal due to alteration, no special processing is required for the altered traffic light. Instead, the signal correlation between each traffic light and the target traffic light is determined directly based on the annotation information in the high-precision map data. This allows for the fusion and generation of the corresponding virtual light signal, reducing the amount of data processing while ensuring the accuracy and reliability of the virtual light signal.

[0139] In step S307, in response to the absence of change information, the virtual light signal of the traffic light group is determined based on the tracking information and high-precision map data.

[0140] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 If the terminal device 101 shown (e.g., a vehicle-mounted terminal or a control device in the vehicle-mounted terminal) determines that the change information is empty, that is, no change has occurred in the signal light group, then the virtual light signal of the signal light group is determined according to the annotation information of the signal light group in the high-precision map data.

[0141] Once the aforementioned executing entity confirms that no traffic lights in the traffic light group have changed, it can track the signals of each traffic light in the traffic light group based on real-scene image data, obtain tracking information, and combine it with the annotation information of the traffic light group in high-precision map data to generate virtual light signals for the traffic light group.

[0142] In this implementation, when the executing entity determines that there is no change in the traffic lights in the traffic light group, it combines the tracking information obtained from the real-scene image with the annotation information in the high-precision map data to obtain the virtual light information of the traffic light group, ensuring the real-time performance and accuracy of the virtual light information, thereby ensuring the accuracy of traffic signal recognition.

[0143] In this embodiment of the disclosure, if the executing entity determines, based on the change information, that there are no changes in the traffic lights in the traffic light group, the signals other than the target traffic light have been deleted or malfunctioned, or the semantic priority of the changing traffic light is lower than that of the target traffic light, the executing entity can determine the virtual light signal corresponding to the traffic light group by combining the tracking information and high-precision map data. For example, based on the annotation information in the high-precision map data, the traffic lights in the traffic light group are grouped, and then tracked, denoised, smoothed, and fused separately to obtain at least one set of virtual light signals, which are then merged to obtain the virtual light signal corresponding to the traffic light group.

[0144] In some optional implementations of the embodiments of this disclosure, determining the virtual light signal of the traffic light group based on tracking information and high-precision map data includes: determining the attention group and non-attention group in the traffic light group based on whether the annotation information in the high-precision map data includes the control direction of the target traffic light; determining the first virtual light information corresponding to the attention group based on tracking information and high-precision map data; determining the second virtual light information corresponding to the non-attention group based on tracking information; and determining the virtual light signal corresponding to the traffic light group based on the first virtual light signal and the second virtual light signal.

[0145] For example, the light group of concern includes the target traffic light and the traffic lights that have not changed and are highly correlated with the target traffic light; the light group of non-concern includes traffic lights other than the target traffic light and which are less correlated with the target traffic light.

[0146] In some optional implementations of the embodiments of this disclosure, determining the attention group and non-attention group in the traffic light group based on whether the annotation information in the high-precision map data includes the control direction of the target traffic light includes: in response to the annotation information including the control direction of the target traffic light, determining that the attention group includes the target traffic light and the non-attention group includes the remaining traffic lights other than the target traffic light; in response to the annotation information not including the control direction of the target traffic light, determining that the attention group includes the target traffic light and a first traffic light in the traffic light group whose control direction is different from that of the target traffic light, and the non-attention group includes a second traffic light in the traffic light group whose control direction is the same as that of the target traffic light.

[0147] In this scheme, the implementing entity obtains the annotation information for the traffic light group from high-precision map data. Based on whether the annotation information includes the control direction of the target traffic light, the traffic lights in the traffic light group are divided into attention groups and non-attention groups according to different classification rules.

[0148] For example, the control direction of the target traffic light includes at least one of U-turn, left turn, straight, and right turn.

[0149] If the control direction of the target signal light is determined to be included in the labeling information, the executing entity classifies the target signal light as a separate group of lights of interest, and classifies the remaining signal lights in the group other than the target signal light as a group of lights of non-interest.

[0150] If the executing entity determines that the control direction of the target traffic light is not included in the annotation information, it first determines the control direction of each traffic light in the traffic light group. For example, the executing entity can determine the control direction of each traffic light by combining the lane markings of each lane in high-precision map data with the position of the traffic lights; alternatively, it can identify the control direction corresponding to each traffic light from real-scene image data. Then, based on whether the control direction of each traffic light is the same as the control direction of the target traffic light, it divides the traffic lights into corresponding attention groups and non-attention groups.

[0151] For example, a traffic light group includes four traffic lights. The first is the target traffic light, indicating either a left turn or straight ahead. The second indicates straight ahead, the third indicates a right turn, and the fourth is a counting traffic light that does not indicate a direction. In this case, the second traffic light indicates the same direction as the target traffic light (both can indicate straight ahead), while the third traffic light indicates a different direction, and the fourth traffic light's direction is unrelated to the target traffic light's control direction. The executing entity groups the target traffic light and the third traffic light together as the attention group, and the second and fourth traffic lights as the non-attention group.

[0152] In this embodiment of the disclosure, the executing entity divides the traffic light groups into attention groups and non-attention groups based on the annotation information of traffic light groups in the high-precision map data; then, it generates a first virtual light signal and a second virtual light signal for the attention groups and the non-attention groups respectively, and then merges the first virtual light signal and the second virtual light signal to obtain the virtual light signal corresponding to the traffic light group.

[0153] In some alternative implementations, for the light group of interest, the executing entity can directly obtain the corresponding first virtual light signal by fusing the annotation information in the high-precision map data.

[0154] In some optional implementations, the executing entity can also fuse a first virtual light with a corresponding control direction based on the control direction indicated by the high-precision map data; and determine the signal sequence of the first virtual light based on the tracking information of the real scene image data, thereby generating a first virtual light signal with a signal sequence.

[0155] In some alternative implementations, for non-interested light groups, the executing entity can directly track the information of each traffic light belonging to the non-interested light group in the real-world image data, determine the control direction and signal sequence of each traffic light, perform noise reduction and smoothing on the signal sequence, and then fuse them to obtain a second virtual light signal with the corresponding signal sequence. By tracking the real-world image data to fuse the second virtual light signal, the accuracy and reliability of the second virtual light signal are improved.

[0156] After receiving the first virtual light signal and the second virtual light signal, the executing entity can directly merge the first virtual light signal and the second virtual light signal to obtain the virtual light signal corresponding to the signal light group.

[0157] In this embodiment of the disclosure, the executing entity divides the traffic light group into a group of lights of interest and a group of lights of non-interest based on the annotation information of the traffic light group in the high-precision map data. While ensuring the signal accuracy of the group of lights of interest, a second virtual light signal is generated for the group of lights of non-interest based on the tracking results of the real scene image data. This second virtual light signal is used to assist the first virtual light signal of the group of lights of interest. This improves the accuracy and reliability of the virtual light signal while ensuring the integrity of the virtual light signal of the traffic light group.

[0158] Step S308: Based on the high-precision map data, determine the target virtual light corresponding to the target signal light in the signal light group.

[0159] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle-mounted terminal or a control device in the vehicle-mounted terminal) determines the target virtual light corresponding to the target traffic light in the traffic light group based on high-precision map data.

[0160] The executing entity determines the target traffic light from the high-precision map data based on the current location, and obtains lane information such as lane attributes, lane line positions, and stop line positions corresponding to the current location. Then, based on the lane information and the target traffic light, it determines the target virtual light corresponding to the target traffic light.

[0161] For example, the target virtual light is used to indicate that the current lane's travel direction is at least one of U-turn, left turn, straight ahead, and right turn. For instance, the target virtual light may simultaneously include a U-turn virtual light and a left turn virtual light, or simultaneously include a straight ahead virtual light and a right turn virtual light, or simultaneously include a left turn virtual light and a straight ahead virtual light; it may also individually include any one of a left turn virtual light, a U-turn virtual light, a straight ahead virtual light, and a right turn virtual light.

[0162] Step S309: Determine the traffic signal indicated by the target virtual light from the virtual light signals as the target traffic signal.

[0163] In this embodiment of the disclosure, the entity executing the traffic signal determination method is, for example... Figure 1 The terminal device 101 shown (e.g., a vehicle terminal or a control device in the vehicle terminal) determines the traffic signal indicated by the target virtual light from the virtual light signal based on the target virtual light obtained in step S308, and uses it as the target traffic signal.

[0164] For example, the virtual light signal includes virtual lights indicating each driving direction, and also includes signal sequences corresponding to each virtual light, such as whether a virtual light indicating a certain driving direction is lit, the duration of the light being lit, whether the light being lit indicates a stop or communication, and signal sequences consisting of different states such as lit, off, and flashing.

[0165] After determining the target virtual light corresponding to the lane where the current location is located, the executing entity can directly separate the virtual signal corresponding to the target virtual light from the virtual light signal and use it as the target traffic signal to control the vehicle's autonomous driving.

[0166] In this embodiment of the disclosure, after determining the virtual light signal corresponding to the traffic light group, the executing entity determines the target traffic light and its corresponding target virtual light based on high-precision map data. Then, it extracts the traffic signal indicated by the target virtual light from the virtual light signal as the target traffic signal. By combining the virtual light signal with the high-precision map data, it can effectively assist the autonomous driving behavior of the vehicle and improve the safety of autonomous driving.

[0167] According to the traffic signal determination method provided in this disclosure, by combining real-scene image data with high-precision map data, the change information of traffic light groups is accurately identified. Based on whether the change information includes a target traffic light and its change type, the traffic light groups are divided into different attention groups and non-attention groups, and corresponding first and second virtual light signals are formed. These are then merged to obtain the virtual light signal corresponding to the traffic light group. Finally, the traffic signal indicated by the target virtual light corresponding to the target traffic light is extracted from the high-precision map data to obtain the target traffic signal. This solution can accurately identify the actual change information of traffic light groups, avoiding incorrect traffic signals due to mislabeling or untimely updates of high-precision map data, improving the accuracy and reliability of traffic signals, and thus enhancing the safety of autonomous driving.

[0168] As an implementation of the method for determining the traffic signals shown in the above figures, Figure 4 An embodiment of a traffic signal determination device according to the present disclosure is shown. The traffic signal determination device 400 and... Figure 2 , Figure 3 Corresponding to the method embodiments shown, this device can be applied to various electronic devices.

[0169] Reference Figure 4 As shown, the traffic signal determination device 400 provided in this embodiment includes: a first determination module 401, a second determination module 402, and a third determination module 403. The first determination module 401 is configured to determine the change information of the traffic light group based on real-scene image data and high-precision map data; the second determination module 402 is configured to determine the virtual light signal corresponding to the traffic light group based on the change information; and the third determination module 403 is configured to determine the target traffic signal indicated by the target traffic light in the traffic light group based on the virtual light signal and the high-precision map data.

[0170] The specific processing of the first determining module 401, the second determining module 402, and the third determining module 403 in the traffic signal determining device 400 of this disclosure, and the resulting technical effects, can be found in the following references: Figure 2 The relevant descriptions of steps S201-S203 in the corresponding embodiments will not be repeated here.

[0171] In some optional implementations of the embodiments of this disclosure, the first determining module includes: a first determining unit and a second determining unit, wherein the first determining unit is configured to determine the annotation information of the traffic light group based on real-scene image data and high-precision map data; and the second determining unit is configured to determine the change information of the traffic light group based on the annotation information and the status information of the traffic light group in the real-scene image data.

[0172] In the traffic signal determination device of this disclosure, the specific processing of the first determination unit and the second determination unit, and the resulting technical effects, can be referred to respectively. Figure 3 The relevant descriptions of steps S301-S302 in the corresponding embodiments will not be repeated here.

[0173] In some optional implementations of the embodiments of this disclosure, the first determining unit is configured to: determine the traffic light group facing the current location based on real-scene image data and high-precision map data; and obtain the labeling information of the traffic light group from the high-precision map data.

[0174] In some optional implementations of the embodiments of this disclosure, the second determining unit is configured to: identify the status information of the traffic light group in the real scene image data; project the annotation information onto the real scene image data to obtain projection information; and compare the status information and the projection information to determine the change information of the traffic light group.

[0175] In some optional implementations of the embodiments of this disclosure, the annotation information includes at least one of the following: marker coordinate information, marker shape information, and marker color information; the status information includes at least one of the following: illustration position information, illustration shape information, and illustration color information.

[0176] In some optional implementations of the embodiments of this disclosure, the second determining module includes: a third determining unit, a fourth determining unit, and a fifth determining unit, wherein the third determining unit is configured to determine the target traffic light in the traffic light group based on high-precision map data; the fourth determining unit is configured to determine whether the target traffic light has changed based on change information; and the fifth determining unit is configured to determine the virtual light signal of the traffic light group based on the tracking information of the traffic light group in the real-scene image data in response to the change of the target traffic light.

[0177] In the traffic signal determination device of this disclosure, the specific processing of the third, fourth, and fifth determining units and their resulting technical effects can be found by referring to [reference needed]. Figure 3 The relevant descriptions of steps S303-S305 in the corresponding embodiments will not be repeated here.

[0178] In some optional implementations of the embodiments of this disclosure, the second determining module further includes a sixth determining unit, which is configured to: determine the virtual light signal of the signal light group based on the change type in the change information in response to the target signal light not changing and the change information not being empty.

[0179] In the traffic signal determination device of this disclosure, the specific processing of the sixth determination unit and its resulting technical effects can be found in the following references. Figure 3 The relevant descriptions of step S306 in the corresponding embodiments will not be repeated here.

[0180] In some optional implementations of the embodiments of this disclosure, the change type in the change information includes at least one of the following: adding a traffic light, changing the semantic meaning of a traffic light; and determining the virtual light signal of the traffic light group based on the change type in the change information includes: in response to the change type in the change information being a adding or changing traffic light semantics, determining the semantic priority of the changed traffic light in the change information; in response to the semantic priority of the changed traffic light being higher than the semantic priority of the target traffic light, determining the virtual light signal of the traffic light group based on tracking information; in response to the semantic priority of the target changed traffic light being lower than or equal to the semantic priority of the target traffic light, determining the virtual light signal of the traffic light group based on tracking information and high-precision map data.

[0181] In some optional implementations of the embodiments of this disclosure, the change type in the change information includes traffic light deletion or traffic light failure, and determining the virtual light signal of the traffic light group based on the change type in the change information includes: in response to the change type in the change information being traffic light deletion, determining the virtual light signal of the traffic light group based on tracking information and high-precision map data.

[0182] In some optional implementations of the embodiments of this disclosure, the second determining module further includes a seventh determining unit, which is configured to: determine the virtual light signal of the traffic light group based on the tracking information and high-precision map data in response to the change information being empty.

[0183] In the traffic signal determination device of this disclosure, the specific processing of the seventh determination unit and its resulting technical effects can be found in the following references. Figure 3 The relevant descriptions of step S307 in the corresponding embodiments will not be repeated here.

[0184] In some optional implementations of the embodiments of this disclosure, the seventh determining unit includes: a first determining subunit, a second determining subunit, a third determining subunit, and a fourth determining subunit. The first determining subunit is configured to determine the attention group and non-attention group in the signal light group based on whether the annotation information in the high-precision map data includes the control direction of the target signal light. The second determining subunit is configured to determine the first virtual light signal corresponding to the attention group based on tracking information and the high-precision map data. The third determining subunit is configured to determine the second virtual light signal corresponding to the non-attention group based on tracking information. The fourth determining subunit is configured to determine the virtual light signal corresponding to the signal light group based on the first virtual light signal and the second virtual light signal.

[0185] In some optional implementations of embodiments of this disclosure, the first determining subunit is configured to: in response to the control direction of the target traffic light being included in the annotation information, determine that the light group of interest includes the target traffic light and the light group of non-interest includes the remaining traffic lights other than the target traffic light; in response to the control direction of the target traffic light not being included in the annotation information, determine that the light group of interest includes the target traffic light and a first traffic light in the light group whose control direction is different from that of the target traffic light, and the light group of non-interest includes a second traffic light in the light group whose control direction is the same as that of the target traffic light.

[0186] In some optional implementations of the embodiments of this disclosure, the third determining module is configured to: determine the target virtual light corresponding to the target traffic light in the traffic light group based on high-precision map data; and determine the traffic signal indicated by the target virtual light from the virtual light signals as the target traffic signal.

[0187] In the traffic signal determination device of this disclosure, the specific processing of the third determination module and its resulting technical effects can be found in the following: Figure 3 The relevant descriptions of steps S308-S309 in the corresponding embodiments will not be repeated here.

[0188] According to embodiments of this disclosure, this disclosure also provides a vehicle including an autonomous driving system configured to execute the traffic signal determination method provided above, or including the traffic signal determination device provided above.

[0189] For example, the autonomous driving system can control the vehicle to drive automatically according to the target traffic signal determined by the traffic signal determination method described above, so as to safely pass through the current intersection location.

[0190] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0191] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0192] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0193] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0194] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the traffic signal determination method. For example, in some embodiments, the traffic signal determination method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the traffic signal determination method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the traffic signal determination method by any other suitable means (e.g., by means of firmware).

[0195] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0196] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0197] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0198] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0199] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0200] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0201] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0202] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining traffic signals, comprising: Based on real-scene image data and high-precision map data, determine the change information of the traffic light group. The change type in the change information includes at least one of the following: addition of traffic lights, change of traffic light semantics; Determining the virtual light signal corresponding to the traffic light group based on the change information includes: in response to the target traffic light in the traffic light group not changing and the change information not being empty, and in response to the change type in the change information being a traffic light addition or a traffic light semantic change, determining the semantic priority of the changed traffic light in the change information; in response to the semantic priority of the changed traffic light being higher than the semantic priority of the target traffic light, determining the virtual light signal of the traffic light group based on the tracking information of the traffic light group in the real-scene image data; in response to the semantic priority of the changed traffic light being lower than or equal to the semantic priority of the target traffic light, determining the virtual light signal of the traffic light group based on the tracking information and the high-precision map data. Based on the virtual light signal and the high-precision map data, the target traffic signal indicated by the target light in the signal light group is determined.

2. The method according to claim 1, wherein, The process of determining the change information of the traffic light group based on real-scene image data and high-precision map data includes: The labeling information of the traffic light group is determined based on the real-scene image data and the high-precision map data; Based on the annotation information and the status information of the traffic light group in the real-scene image data, the change information of the traffic light group is determined.

3. The method according to claim 2, wherein, The step of determining the labeling information of the traffic light group based on the real-scene image data and the high-precision map data includes: Based on the real-scene image data and the high-precision map data, determine the traffic light group that the current location is facing; Obtain the labeling information of the traffic light group from the high-precision map data.

4. The method according to claim 2, wherein, The step of determining the change information of the traffic light group based on the annotation information and the status information of the traffic light group in the real-scene image data includes: Identify the status information of the traffic light group in the real-scene image data; The annotation information is projected onto the real-scene image data to obtain projection information; By comparing the status information and the projection information, the change information of the signal light group is determined.

5. The method according to claim 2, wherein, The annotation information includes at least one of the following: marker coordinate information, marker shape information, and marker color information; the status information includes at least one of the following: illustration position information, illustration shape information, and illustration color information.

6. The method according to claim 1, wherein, The step of determining the virtual light signal corresponding to the traffic light group based on the change information includes: Based on the high-precision map data, the target traffic light in the traffic light group is determined; Based on the change information, determine whether the target traffic light has changed; In response to a change in the target traffic light, the virtual light signal of the traffic light group is determined based on the tracking information.

7. The method according to claim 6, wherein, The step of determining the virtual light signal corresponding to the traffic light group based on the change information further includes: In response to the fact that the target traffic light has not changed and the change information is not empty, the virtual light signal of the traffic light group is determined according to the change type in the change information.

8. The method according to claim 7, wherein, The change types in the change information include traffic light deletion or traffic light malfunction, and Determining the virtual light signal of the signal light group based on the change type in the change information includes: In response to the change type in the change information being traffic light deletion, the virtual light signal of the traffic light group is determined based on the tracking information and the high-precision map data.

9. The method according to claim 6, wherein, The step of determining the virtual light signal corresponding to the traffic light group based on the change information further includes: In response to the absence of the change information, the virtual light signal of the traffic light group is determined based on the tracking information and the high-precision map data.

10. The method according to any one of claims 1, wherein, The step of determining the virtual light signal of the traffic light group based on the tracking information and the high-precision map data includes: Based on whether the annotation information in the high-precision map data includes the control direction of the target traffic light, determine the attention group and non-attention group in the traffic light group; Based on the tracking information and the high-precision map data, determine the first virtual light signal corresponding to the light group of interest; The second virtual light signal corresponding to the non-interested light group is determined based on the tracking information; Based on the first virtual light signal and the second virtual light signal, the virtual light signal corresponding to the signal light group is determined.

11. The method according to claim 10, wherein, The step of determining the attention group and non-attention group in the traffic light group based on whether the annotation information in the high-precision map data includes the control direction of the target traffic light includes: In response to the control direction of the target traffic light being included in the annotation information, it is determined that the group of lights of interest includes the target traffic light and the group of lights of non-interest includes the remaining traffic lights other than the target traffic light; In response to the fact that the control direction of the target traffic light is not included in the annotation information, it is determined that the attention light group includes the target traffic light and a first traffic light in the traffic light group whose control direction is different from that of the target traffic light, and the non-attention light group includes a second traffic light in the traffic light group whose control direction is the same as that of the target traffic light.

12. The method according to claim 1, wherein, The step of determining the target traffic signal indicated by the target traffic light in the traffic light group based on the virtual light signal and the high-precision map data includes: Based on the high-precision map data, determine the target virtual light corresponding to the target traffic light in the traffic light group; The traffic signal indicated by the target virtual light is determined from the virtual light signals as the target traffic signal.

13. A traffic signal determining device, comprising: The first determining module is configured to determine the change information of the traffic light group based on real-scene image data and high-precision map data. The change type in the change information includes at least one of the following: addition of traffic lights, semantic change of traffic lights; The second determining module is configured to determine the virtual light signal corresponding to the traffic light group based on the change information, including: in response to the fact that the target traffic light in the traffic light group has not changed and the change information is not empty, and in response to the change type in the change information being a traffic light addition or a traffic light semantic change, determining the semantic priority of the changed traffic light in the change information; in response to the fact that the semantic priority of the changed traffic light is higher than the semantic priority of the target traffic light, determining the virtual light signal of the traffic light group based on the tracking information of the traffic light group in the real-scene image data; in response to the fact that the semantic priority of the changed traffic light is lower than or equal to the semantic priority of the target traffic light, determining the virtual light signal of the traffic light group based on the tracking information and the high-precision map data. The third determining module is configured to determine the target traffic signal indicated by the target traffic light in the traffic light group based on the virtual light signal and the high-precision map data.

14. The apparatus according to claim 13, wherein, The first determining module includes: The first determining unit is configured to determine the labeling information of the traffic light group based on the real-scene image data and the high-precision map data; The second determining unit is configured to determine the change information of the traffic light group based on the annotation information and the status information of the traffic light group in the real scene image data.

15. The apparatus according to claim 14, wherein, The first determining unit is configured as follows: Based on the real-scene image data and the high-precision map data, determine the traffic light group that the current location is facing; Obtain the labeling information of the traffic light group from the high-precision map data.

16. The apparatus according to claim 14, wherein, The second determining unit is configured as follows: Identify the status information of the traffic light group in the real-scene image data; The annotation information is projected onto the real-scene image data to obtain projection information; By comparing the status information and the projection information, the change information of the signal light group is determined.

17. The apparatus according to claim 14, wherein, The annotation information includes at least one of the following: marker coordinate information, marker shape information, and marker color information; the status information includes at least one of the following: illustration position information, illustration shape information, and illustration color information.

18. The apparatus according to claim 13, wherein, The second determining module includes: The third determining unit is configured to determine the target traffic light in the traffic light group based on the high-precision map data; The fourth determining unit is configured to determine whether the target traffic light has changed based on the change information; The fifth determining unit is configured to determine the virtual light signal of the signal light group based on the tracking information in response to a change in the target signal light.

19. The apparatus according to claim 18, wherein, The second determining module further includes: The sixth determining unit is configured as follows: In response to the fact that the target traffic light has not changed and the change information is not empty, the virtual light signal of the traffic light group is determined according to the change type in the change information.

20. The apparatus according to claim 19, wherein, The change types in the change information include traffic light deletion or traffic light malfunction, and Determining the virtual light signal of the traffic light group based on the change type in the change information includes: In response to the change type in the change information being traffic light deletion, the virtual light signal of the traffic light group is determined based on the tracking information and the high-precision map data.

21. The apparatus according to claim 18 or 19, wherein, The second determining module further includes: The seventh determining unit is configured to determine the virtual light signal of the traffic light group based on the tracking information and the high-precision map data in response to the change information being empty.

22. The apparatus according to claim 21, wherein, The seventh determining unit includes: The first determining subunit is configured to determine the attention light group and non-attention light group in the signal light group based on whether the annotation information in the high-precision map data includes the control direction of the target signal light; The second determining subunit is configured to determine the first virtual light signal corresponding to the group of lights of interest based on the tracking information and the high-precision map data; The third determining subunit is configured to determine the second virtual light signal corresponding to the non-concerned light group based on the tracking information; The fourth determining subunit is configured to determine the virtual light signal corresponding to the signal light group based on the first virtual light signal and the second virtual light signal.

23. The apparatus according to claim 22, wherein, The first determining subunit is configured as follows: In response to the control direction of the target traffic light being included in the annotation information, it is determined that the group of lights of interest includes the target traffic light and the group of lights of non-interest includes the remaining traffic lights other than the target traffic light; In response to the fact that the control direction of the target traffic light is not included in the annotation information, it is determined that the attention light group includes the target traffic light and a first traffic light in the traffic light group whose control direction is different from that of the target traffic light, and the non-attention light group includes a second traffic light in the traffic light group whose control direction is the same as that of the target traffic light.

24. The apparatus according to claim 13, wherein, The third determining module is configured as follows: Based on the high-precision map data, determine the target virtual light corresponding to the target traffic light in the traffic light group; The traffic signal indicated by the target virtual light is determined from the virtual light signals as the target traffic signal.

25. A vehicle comprising: An autonomous driving system configured to perform the method of any one of claims 1-12, or to include the means of any one of claims 13-24.

26. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

27. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-12.

28. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-12.

Images