Method and apparatus for generating virtual traffic lights

Abstract

The present disclosure provides a method and device for generating a virtual traffic light, and particularly relates to the technical fields of automatic driving, intelligent traffic and the like. The specific implementation scheme is: based on the obtained images of each traffic light in a target light group, state information of the head of each traffic light is obtained; based on the state information of the head, the effective head shapes in the target light group are fused to obtain at least one fused shape and state information of each fused shape in the at least one fused shape; based on a mapping relationship between each fused shape in the at least one fused shape and an indicating direction, the indicating direction of each fused shape in the at least one fused shape is determined; a virtual traffic light is set, and the state information of the at least one fused shape is assigned to the virtual traffic light based on the indicating direction of the at least one fused shape. The embodiment improves the efficiency of virtual traffic light generation.

Description

Technical Field

[0001] This disclosure relates to the field of computer application technology, specifically to the fields of autonomous driving, intelligent transportation, etc., and in particular to a method and apparatus for generating virtual traffic lights, electronic devices, computer-readable media, and computer program products. Background Technology

[0002] In the field of intelligent driving, vehicles must strictly adhere to traffic rules and make subsequent driving decisions based on the color information of traffic lights at the current intersection. To this end, the vehicle detection module needs to detect and identify the color information of all traffic lights at the current intersection, and use high-precision maps to indicate the traffic light control directions and their binding relationships with lanes, informing the vehicle planning and control module of the permissible lanes so that it can control the vehicle's movement in different lanes according to a given traffic light color. However, in actual driving, the structure of traffic lights at intersections is quite complex; controlling traffic in the same direction often involves multiple traffic lights of different shapes. Simply detecting and identifying these traffic lights individually without processing them is insufficient for direct use. Summary of the Invention

[0003] A method and apparatus for generating virtual traffic lights, an electronic device, a computer-readable storage medium, and a computer program product are provided.

[0004] According to a first aspect, a method for generating a virtual traffic light is provided, the method comprising: obtaining state information of the head of each traffic light based on an acquired image of each traffic light in a target light group; fusing valid head shapes in the target light group based on the head state information to obtain at least one fused shape and state information of each fused shape in the at least one fused shape; determining the indication direction of each fused shape in the at least one fused shape based on the mapping relationship between each fused shape in the at least one fused shape and the indication direction; setting a virtual traffic light, and assigning the state information of the at least one fused shape to the virtual traffic light based on the indication direction of the at least one fused shape.

[0005] According to a second aspect, an apparatus for generating virtual traffic lights is provided, the apparatus comprising: an acquisition unit configured to obtain state information of the lamp heads of each traffic light based on images of each traffic light in a target light group; a fusion unit configured to fuse valid lamp head shapes in the target light group based on the lamp head state information to obtain at least one fused shape and state information of each fused shape; a determination unit configured to determine the indication direction of each fused shape in the at least one fused shape based on a mapping relationship between each fused shape and an indication direction; and an assignment unit configured to set the virtual traffic light by assigning the state information of the at least one fused shape to the virtual traffic light based on the indication direction of the at least one fused shape.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method as described in any implementation of the first aspect.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the method described in any implementation of the first aspect.

[0008] According to a fifth aspect, a computer program product is provided, including a computer program that, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0009] The method and apparatus for generating virtual traffic lights provided in the embodiments of this disclosure first obtain the state information of the lamp heads of each traffic light based on the acquired images of each traffic light in a target light group; second, based on the state information of the lamp heads, fuse the valid lamp head shapes in the target light group to obtain at least one fused shape and the state information of each fused shape; third, based on the mapping relationship between each fused shape and the indicated direction, determine the indicated direction of each fused shape; finally, set up a virtual traffic light, and assign the state information of the at least one fused shape to the virtual traffic light based on the indicated direction of the at least one fused shape. Thus, the fused shape obtained by fusing valid lamp heads can effectively reflect the specific indicated direction of the target light group, providing a reliable basis for vehicles to indicate the current planned path direction, and greatly reducing the generation of redundant information when judging the state information of traffic lights.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a flowchart of an embodiment of the method for generating virtual traffic lights according to the present disclosure;

[0013] Figure 2 This is a schematic diagram of the structure of one embodiment of generating virtual traffic lights according to the present disclosure;

[0014] Figure 3 This is a framework diagram of another embodiment of the method for generating virtual traffic lights according to the present disclosure;

[0015] Figure 4 This is a schematic diagram of one embodiment of the apparatus for generating virtual traffic lights according to the present disclosure;

[0016] Figure 5 This is a block diagram of an electronic device used to implement the method for generating virtual traffic lights according to embodiments of the present disclosure. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] In this embodiment, "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.

[0019] This disclosure provides a method for generating virtual traffic lights. Figure 1 A flow 100 is shown as an embodiment of a method for generating virtual traffic lights according to the present disclosure, the method comprising the following steps:

[0020] Step 101: Based on the images of each traffic light in the target light group, obtain the status information of each traffic light head.

[0021] In this embodiment, the target light group is the traffic lights at the same intersection and in the same direction of travel for the vehicles to be detected. The traffic lights in the target light group can provide guidance for different groups. For example, the traffic lights include pedestrian lights, non-motorized vehicle lights, and motorized vehicle lights.

[0022] In this embodiment, the target light group is determined by the area where the vehicle operates. The location of the target light group and the traffic lights within it can differ depending on the vehicle's operating area. Furthermore, the number of traffic lights included in the target light group can also vary in different areas. After determining the target light group, the entity executing the method for generating virtual traffic lights can acquire images of the target light group in real time using an image acquisition device.

[0023] In this embodiment, the target light group may include at least one traffic light, each traffic light having a frame and at least one lamp head. Therefore, a complete frame can represent a traffic light. The above-mentioned method of obtaining the lamp head status information of each traffic light based on the acquired images of each traffic light in the target light group includes: acquiring images of each traffic light in the target light group, identifying the frame in which each traffic light is located in the image, and establishing a tracking sequence for each lamp head in each frame, using the frame as the unit, and recording the lamp head status information of each traffic light using the frame identifier as the sequence identifier.

[0024] In this embodiment, the status information is used to represent the status of each traffic light head. The status information is obtained through image processing and can reflect the display status of all traffic light heads at the same moment, or it can reflect the display status over a period of time. The display status includes: display shape, display color, and display time, etc. Specifically, the status information may include: the current color of the light head, whether it is flashing, whether it is obstructed, and the remaining countdown timer in the light head.

[0025] Optionally, the above-mentioned method of obtaining the status information of each traffic light head based on the images of each traffic light in the target light group further includes: obtaining the images of each traffic light in the target light group, identifying the head of each traffic light in the images and numbering the head of each traffic light, determining the shape of the head of each traffic light, establishing a tracking sequence for each head of the traffic light, and using each tracking sequence to record the status information of each head of the traffic light.

[0026] like Figure 2 The diagram shows the traffic lights in the target light group L detected at the current intersection at the current time. Figure 2 L i Represents the i-th traffic light at the intersection (in Figure 2 There are three traffic lights (L1, L2, and L3 from left to right). Because the shapes of traffic light heads in actual road conditions are very complex, these traffic lights can be pedestrian lights and non-motorized vehicle lights. The shapes of the light heads include any of the following: disc, U-turn arrow, left turn arrow, straight arrow, right turn arrow, pedestrian light, bicycle light, countdown numbers, etc. Based on the status information of each traffic light head, it can be determined which light head is in an illuminated indicating state, that is, the valid light head at the current moment.

[0027] Step 102: Based on the state information of the lamp head, fuse the effective lamp head shapes in the target lamp group to obtain at least one fused shape and the state information of each fused shape in the at least one fused shape.

[0028] In this embodiment, a valid light head refers to the light head of a traffic signal light whose current indication state is valid, as determined by the status information for each traffic signal light. In this embodiment, the shape of the light head includes the original appearance shape of the light head (such as a round head) and the indication shape represented by the pattern on the light head (such as a light head displaying an arrow). A fused valid light head shape refers to a light head shape with the same semantic meaning.

[0029] The above-mentioned method of fusing effective lamp head shapes in the target light group based on lamp head status information to obtain at least one fused shape includes: determining the current display shape of each traffic light head in the target light group based on lamp head status information; comparing the display shapes of all traffic lights according to shape semantics to determine display shapes with the same shape semantics; assigning a fused shape to each traffic light based on the display shape, so that each traffic light corresponds to a fused shape, thereby obtaining at least one fused shape, wherein the shapes of each fused shape in the at least one fused shape are different.

[0030] It should be noted that the status information includes not only the display shape of the traffic light, but also the display color of the traffic light, as well as whether it is obstructed or flashing. Therefore, after determining the fusion shape, associating the fusion shape with the status information can assign more status information to each fusion shape in at least one fusion shape.

[0031] like Figure 2 As shown, each traffic light head corresponds to a unique display shape X, and the status of light heads with the same shape semantics (such as...) are merged. Figure 2 Following the left arrow in the diagram, at least one fused shape R is obtained, and each fused shape in the at least one fused shape is different.

[0032] Step 103: Based on the mapping relationship between each fusion shape and the indication direction in at least one fusion shape, determine the indication direction of each fusion shape in at least one fusion shape.

[0033] In this embodiment, the fused shape is derived from the display shape of the traffic light head or the original appearance shape of the head, such as... Figure 2 In traffic lights, the merged shape can be round or arrow-shaped. Furthermore, arrows can include straight arrows and left arrows. Based on the direction indicated by the shape of the traffic light display in traffic rules, the direction indicated by each merged shape can be determined. For example, the merged shape of a left arrow indicates a left turn.

[0034] In this embodiment, each fused shape has a corresponding control indication direction. For example, in a conventional shape, the U-turn arrow controls the U-turn direction, the left turn arrow controls the U-turn and straight-going directions, the straight-going arrow controls the straight-going direction, the right turn arrow controls the right turn direction, and the circular light controls the three directions of U-turn, left turn, and straight-going.

[0035] In this embodiment, the directional indication of the fused shape is preset according to inherent traffic rules. For example, during driving, the left turn arrow controls the direction of left turn and U-turn. When the driver needs to make a left turn or U-turn while driving, if there is no U-turn arrow, they only need to pay attention to the left turn arrow.

[0036] Step 104: Set up a virtual traffic light by assigning the status information of at least one fused shape to the virtual traffic light based on the indication direction of at least one fused shape.

[0037] In this embodiment, a virtual traffic light is a virtual signal that can represent shape, current state, and changing state. After setting up the virtual traffic light, it is initialized, and the initialized virtual traffic light can be displayed in gray.

[0038] In this embodiment, the virtual traffic light can be a virtual signal with a preset indication direction. When the virtual traffic light has a preset indication direction, the above-mentioned assignment of the state information of at least one blended shape to the virtual traffic light based on the indication direction of at least one blended shape includes: determining all virtual traffic lights with at least one blended shape indication direction based on the indication direction of at least one blended shape, and lighting up the virtual traffic light with the indication direction, so that the virtual traffic light can indicate the direction according to the indication direction. Furthermore, assigning the state information of at least one blended shape to the virtual traffic light can make the virtual traffic light change its state based on the state information of the blended shape.

[0039] When the virtual traffic light does not have a preset indicating direction, the above-mentioned assignment of status information of at least one blended shape to the virtual traffic light based on the indicating direction of at least one blended shape includes: determining various directions that the virtual traffic light needs to indicate based on the indicating direction of at least one blended shape, and making the virtual traffic light indicate such various directions; assigning at least one blended shape information to the virtual traffic light can enable the virtual traffic light to display status information in real time on the basis of indicating various directions.

[0040] The method for generating virtual traffic lights provided in the embodiments of this disclosure first obtains the state information of the lamp heads of each traffic light based on the acquired images of each traffic light in the target light group; second, based on the state information of the lamp heads, the effective lamp head shapes in the target light group are fused to obtain at least one fused shape and the state information of each fused shape; third, based on the mapping relationship between each fused shape and the indicated direction, the indicated direction of each fused shape is determined; finally, a virtual traffic light is set, and the state information of the at least one fused shape is assigned to the virtual traffic light based on the indicated direction of the at least one fused shape. Thus, the fused shape obtained by fusing effective lamp heads can effectively reflect the specific indicated direction of the target light group, providing a reliable basis for vehicles to indicate the current planned path direction, and greatly reducing the generation of redundant information when judging the state information of traffic lights.

[0041] Currently, regardless of the system, autonomous vehicles or similar ADAS (Advanced Driver Assistance Systems) must detect and identify the traffic lights at intersections to comply with traffic rules. After obtaining the status of each traffic light, they must pass it to the subsequent PNC (Planning and Control module). PNC is a general term for planning and control in autonomous driving, encompassing core modules such as navigation, prediction, decision-making, planning, and control, as well as supporting modules such as environmental perception and map positioning. Directly passing the identification information of each traffic light to the PNC would introduce significant redundancy, and the complexity would increase costs for later debugging and maintenance.

[0042] In some optional implementations of this embodiment, the target traffic light is the traffic light that the autonomous vehicle is about to pass through at the intersection. The method further includes sending the indicated direction of the virtual traffic light and the status information of the virtual traffic light to the planning and control module of the autonomous vehicle, so that when the planning direction is the same as the indicated direction of the virtual traffic light, the planning and control module performs the driving operation corresponding to the status information of the virtual traffic light.

[0043] In this embodiment, the planning and control module is a general term for planning and control in autonomous driving, covering core modules such as navigation, prediction, decision-making, planning, and control, as well as supporting modules such as HMI, environmental perception, and map positioning.

[0044] In this embodiment, the virtual traffic light is a virtual signal that does not actually exist. This virtual signal is also a signal transmitted to the planning and control module. Through the virtual traffic light in this embodiment, the planning and control module can directly provide the planning and control module with the status information of the traffic lights at the current intersection. Thus, regardless of what traffic lights are at the intersection, they are all processed and merged in a unified manner, and finally a virtual traffic light containing multiple directions is obtained. The planning and control module can determine which direction to go by looking at the status of the virtual traffic light in that direction.

[0045] This method is clean and efficient. On the one hand, it can reduce the need for autonomous driving to label traffic lights on high-precision maps and eliminate the dependence on the binding relationship between traffic lights and lanes in high-precision maps. On the other hand, by using a unified and clear virtual representation of virtual traffic lights, it can free the subsequent planning and control parts of the planning and control module from complicated logical judgments.

[0046] In this embodiment, when the autonomous vehicle is about to pass through an intersection, the direction indicated by the virtual traffic light corresponding to the traffic lights at that intersection is obtained. The direction indicated by the virtual traffic light reflects the actual direction indicated by the traffic lights facing the intersection. For example, if the direction indicated by the virtual traffic light is straight ahead, and the status information of the straight ahead direction of the virtual traffic light is green, the straight ahead direction and status information are sent to the planning and control module. When the planning and control module's planning direction is straight ahead, since the planned execution direction is the same as the direction indicated by the virtual traffic light, the planning and control module can execute the straight ahead driving operation corresponding to the green light.

[0047] Optionally, when the virtual traffic light indicates multiple directions, the multiple directions and the corresponding status information of each direction are sent to the planning and control module of the autonomous vehicle, so that the planning and control module can retrieve the direction that is the same as the planned direction from the multiple directions and execute the driving operation corresponding to the status information of the direction that is the same as the planned direction.

[0048] The embodiments of this disclosure provide a method for generating virtual traffic lights. After the state information of the fused shape is assigned to the virtual traffic lights, the virtual traffic lights are displayed according to the state information indicated by the fused shape. The virtual traffic lights directly report the virtual light signals at the intersection based on their indicated direction and state information. Thus, the planning and control module only needs to focus on the current virtual light signal. Furthermore, by combining the virtual light signals, a reliable basis is provided for the planned driving strategy.

[0049] In some optional implementations of this embodiment, the above-mentioned setting of virtual traffic lights, based on the indicating direction of at least one blended shape, and assigning the status information of at least one blended shape to the virtual traffic lights includes: setting virtual traffic lights, the virtual traffic lights including: virtual sub-lights indicating different directions, the directions of the virtual sub-lights including: going straight, turning left, turning right, and making a U-turn; setting priority levels for all blended shapes in the at least one blended shape; controlling the virtual sub-lights corresponding to the indicating direction of the blended shape in the at least one blended shape to turn on sequentially according to the priority order of each blended shape; and assigning the status information corresponding to the blended shape to the virtual sub-light.

[0050] like Figure 2 As shown, the virtual traffic lights include four virtual sub-lights, each representing one of the following directions: straight, left turn, right turn, or U-turn. The virtual lights are merged according to priority. The priority of each merging shape is pre-set during the assignment process. Taking the common shapes (U-turn arrow, left turn arrow, straight arrow, and circular light) as an example, the priority should be assigned in the order of U-turn arrow > left turn arrow > straight arrow > circular light > right turn arrow. The status information associated with each merging shape is then assigned to its corresponding virtual sub-light. This results in virtual sub-lights representing the virtual status information for the four control directions: <U-turn virtual sub-light: Status (U-turn), Left turn virtual sub-light: Status (Left turn), Straight virtual sub-light: Status (Straight), Right turn virtual sub-light: Status (Right turn).

[0051] In this embodiment, at least one fused shape can be one fused shape or multiple fused shapes. When at least one fused shape is one fused shape, the priority of that fused shape is directly set to the highest. When at least one fused shape is multiple fused shapes, a priority level is set for each fused shape. All fused shapes in at least one fused shape are arranged from high to low priority level, or all fused shapes in at least one fused shape are arranged from low to high priority level. The priority order of each fused shape can be obtained by arranging them from high to low or from low to high.

[0052] In this embodiment, the specific order of priority of each fused shape can be determined based on the type of indication direction represented by the fused shape. For example, since the fused shape with a round head indicates three types of directions: straight, left turn, and right turn, while the fused shape with an arrow indicates only one direction, the priority of the fused shape with a round head is greater than that of the fused shape with an arrow.

[0053] The optional implementation provides a method for assigning fusion state information to virtual traffic lights. According to priority, the state information of each fusion shape is assigned to its corresponding virtual sub-light, and the virtual sub-light includes the state information of four control directions, providing a reliable implementation for the generation of virtual traffic lights.

[0054] Optionally, based on the current shape of the traffic light, all merging shapes and their indicating directions are determined, and priority levels are set for all merging shapes. The above-mentioned setting of virtual traffic lights, based on the indicating direction of at least one merging shape, and assigning status information of at least one merging shape to the virtual traffic light, includes: setting virtual traffic lights, which include virtual sub-lights indicating different directions; the directions of the virtual sub-lights include any one of: straight ahead, left turn, right turn, and U-turn; sorting all merging shapes in the at least one merging shape according to the pre-set priority levels; and sequentially controlling the virtual sub-lights corresponding to the indicating directions of the at least one merging shape to turn on according to the sorting result, and assigning the status information corresponding to that merging shape to the virtual sub-light.

[0055] Optionally, determining the indication direction of each fusion shape in at least one fusion shape based on the mapping relationship between each fusion shape and the indication direction includes: sending all fusion shapes in the at least one fusion shape to a semantic translation model to obtain the indication direction of each fusion shape output by the semantic translation model.

[0056] In this optional implementation, the semantic translation model is a model that represents the mapping relationship between merged shapes and indicated directions. The semantic translation model is a model that interfaces merged shapes and virtual traffic lights. It represents the correspondence between at least one merged shape and its indicated direction. The semantic translation model can analyze at least one merged shape, determine the direction represented by all merged shapes within that shape, and translate each merged shape into its indicated direction based on the input order of the merged shapes. This indicated direction is a direction code that can be output to the virtual traffic light. After receiving this direction code, the virtual traffic light indicates the corresponding direction. For example, at least one merged shape includes a straight-ahead shape and a turning shape. The semantic translation model translates the straight-ahead shape and the turning shape, obtaining and outputting a straight-ahead code and a turning code to the virtual traffic light. The straight-ahead code and the turning code are information that the virtual traffic light can recognize. Through this recognition information, the virtual traffic light can sequentially represent the indicated direction corresponding to the recognized information. Simultaneously, when the virtual traffic light represents the indicated direction, it also indicates the corresponding state information, thereby achieving the purpose of representing the state of traffic signals at the same intersection.

[0057] In this embodiment, the semantic translation model is a translation-oriented semantic model, and the training process of the semantic translation model is as follows:

[0058] 1) Collect various fusion shapes as training samples, where the fusion shape can be the shape of the head of a traffic light.

[0059] 2) Label the orientation codes of the training samples to construct a dataset, where the orientation codes are used to represent the different indicative directions of the fused shape.

[0060] 3) Construct a semantic translation model using model structures such as convolutional neural networks. Then, train the semantic translation model using the constructed training samples. During training, the error of the semantic translation model can be determined by the difference between the detection result of the semantic translation model on the direction of the training samples and the direction encoding of the training samples. The parameters of the semantic translation model are iteratively adjusted using error backpropagation to gradually reduce the error. When the error of the semantic translation model converges to a certain range or the number of iterations reaches a preset threshold, parameter adjustment can be stopped, and the trained semantic translation model is obtained.

[0061] In some optional implementations of this embodiment, the above-mentioned setting of a virtual traffic light, based on the indication direction of at least one blended shape, and assigning the state information of at least one blended shape to the virtual traffic light includes: setting a virtual traffic light; in response to the virtual traffic light having the representation of the indication direction of at least one blended shape, assigning the indication direction of at least one blended shape to the virtual traffic light; and when the virtual traffic light represents the indication direction, assigning the state information corresponding to the indication direction to the virtual traffic light.

[0062] In this optional implementation, after setting up the virtual traffic light, the virtual traffic light has different display feedback for control signals indicating multiple directions. In order for the virtual traffic light to represent multiple different directions, when programming the virtual traffic light, it is necessary to first pre-set the indication direction represented by the virtual traffic light control signal. For example, the control signal of the virtual traffic light can represent controlling the display of the straight direction or controlling the display of the turning direction. When the indication direction of each fused shape matches the control signal representing the straight direction in the virtual traffic light, it is determined that the virtual traffic light has the representation of the indication direction.

[0063] The optional implementation provides a method for assigning fusion state information to virtual traffic lights. It obtains the indication direction of the fusion shape based on a semantic translation model, assigns the indication direction to the virtual traffic light, and assigns the state information corresponding to the indication direction to the virtual traffic light when the virtual traffic light represents the indication direction. This provides a reliable implementation method for generating virtual traffic lights.

[0064] In some optional implementations of this disclosure, the above-mentioned method of fusing effective lamp head shapes in the target lamp group based on the lamp head state information to obtain at least one fused shape and state information of each fused shape, including:

[0065] Based on the lamp head state information, the lamp head shapes of traffic lights with the same shape in the target light group are determined, and these lamp head shapes are combined into a fused shape. The state information of each lamp head in the traffic lights with the same shape is detected to be identical. In response to at least one lamp head having different state information, the state information of the lamp heads of the traffic lights with the same shape is selected, and this selected state information is used as the association relationship of the fused shape. In this embodiment, the confidence level of the state information of each lamp head in the traffic lights with the same shape can be calculated, and the state information of the lamp heads of the traffic lights with the same shape is selected based on the magnitude of the confidence level.

[0066] Optionally, the above-mentioned method of fusing effective lamp head shapes in the target lamp group based on the lamp head status information to obtain at least one fused shape and the status information of each fused shape in the at least one fused shape may further include: based on the lamp head status information, in response to the display shapes of lamp heads having at least one traffic light having the same semantics, taking the lamp head shapes of traffic lights with display shapes having the same semantics as a fused shape, and taking the lamp head status information of the fused traffic lights with display shapes having the same semantics as the status information of the fused shape.

[0067] The method for obtaining the fused shape provided by this optional implementation determines the fused shape by comparing the shapes of the lamp heads of traffic lights in the target light group, thus providing an optional implementation for obtaining the fused shape. Furthermore, when the state information of the lamp head of at least one traffic light in a traffic light group with the same shape is different from the state information of the lamp heads of other traffic lights in the same shape, the state information of the lamp heads of the traffic lights with the same shape is selected, thus providing an optional implementation for obtaining the state information of the fused shape.

[0068] In another embodiment of this disclosure, the above-mentioned method of obtaining the status information of each traffic light head based on the images of each traffic light in the target light group includes: obtaining the images of each traffic light in the target light group during a preset time period; determining the time state and state change value of each traffic light head at each moment based on the images, and using the time state and state change value as the status information of each traffic light head.

[0069] In this embodiment, the preset time period can be the collection time period of the execution subject on which the method for generating virtual traffic lights runs. During the collection time period, the execution subject collects the time status multiple times and determines the status change value through the collected time status.

[0070] In this embodiment, the state change value is the state at any given time relative to the state at an adjacent time. For example, if the light head color is red at any given time and the light head color is green at the state at an adjacent time, then the state change value is a change from red to green, and the traffic status of the lane changes from impassable to passable.

[0071] In this embodiment, the time status includes: the color of the lamp head. The status change values ​​include: whether it is blocked, whether it is flashing, and the countdown value.

[0072] In this embodiment, determining the time state and state change value of each traffic light at each moment includes: determining the light head color at each moment; determining whether the light head is flashing based on the light head color within a first set time period; and determining whether the light head is blocked based on the light head color within a second set time period, wherein the second set time period is longer than the first set time period.

[0073] The method for obtaining the state information of traffic lights provided in this embodiment determines the time state and state change value of each traffic light head at each moment based on the image of each traffic light in the target light group within a preset time period, and uses the time state and state change value as state information, providing a reliable way to obtain the state information.

[0074] In one embodiment of this disclosure, such as Figure 3 As shown, for a real-time autonomous vehicle, images of each traffic light in the target light group are acquired (e.g., ...). Figure 3 The "Get Image" option shown, combined with a high-precision map (such as...) Figure 3 The "high-precision map" shown detects the geographical location of the target traffic light group to determine whether it is a traffic light in a preset area. After confirming that the target traffic light group is a traffic light in a preset area, the status information of each traffic light head can be tracked based on the images and high-precision map of each traffic light in the target traffic light group (e.g., ...). Figure 3 The "tracking state" shown refers to the traffic light status information, which may include: the color of the traffic light head, the shape of the traffic light head, and the presence of the traffic light head. Based on the traffic light head status information, the valid traffic light head shapes in the target light group are fused to obtain at least one fused shape and the status information of each of the at least one fused shape (e.g., ...). Figure 3(Semantic fusion as shown). A virtual traffic light is set up, and based on the indicating direction of at least one fused shape, the state information of at least one fused shape is assigned to the virtual traffic light (e.g., ...). Figure 3 (See "Generate Virtual Traffic Lights"). It should be noted that after assigning the state information of the merged shape to the virtual traffic light, the virtual traffic light displays or outputs signals according to the content of the state information. At this point, the shape of the head of each traffic light in the target light group continues to be tracked, and the indicating direction and state information of the virtual traffic light are refreshed.

[0075] During vehicle movement, images captured by cameras are detected, tracked, and identified to obtain the headlights and corresponding shape information of all traffic lights at the current intersection. Then, virtual light fusion is performed sequentially based on the valid shape information. Passing the virtual lights to the PNC (Power Control Center) significantly simplifies the original information complexity.

[0076] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an apparatus for generating virtual traffic lights, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0077] like Figure 4 As shown, the apparatus 400 for generating virtual traffic lights provided in this embodiment includes: an acquisition unit 401, a fusion unit 402, a determination unit 403, and an assignment unit 404. The acquisition unit 401 can be configured to obtain the state information of the lamp heads of each traffic light based on the acquired images of each traffic light in the target light group. The fusion unit 402 can be configured to fuse the valid lamp head shapes in the target light group based on the lamp head state information, obtaining at least one fused shape and state information of each of the at least one fused shape. The determination unit 403 can be configured to determine the indication direction of each of the at least one fused shape based on the mapping relationship between each of the at least one fused shape and the indication direction. The assignment unit 404 can be configured to set up a virtual traffic light and assign the state information of at least one fused shape to the virtual traffic light based on the indication direction of the at least one fused shape.

[0078] In this embodiment, the specific processing of the acquisition unit 401, the fusion unit 402, the determination unit 403, and the assignment unit 404 in the device 400 for generating virtual traffic lights, and the resulting technical effects, can be found in reference to [reference needed]. Figure 1 The relevant descriptions of steps 101, 102, 103, and 104 in the corresponding embodiments will not be repeated here.

[0079] In some optional implementations of this embodiment, the target traffic light is the traffic light that the autonomous vehicle is about to pass through at the intersection, and the device 400 further includes a sending unit (not shown in the figure). The sending unit can be configured to send the indicated direction of the virtual traffic light and the status information of the virtual traffic light to the planning and control module of the autonomous vehicle, so that the planning and control module performs the driving operation corresponding to the status information of the virtual traffic light when the planning direction is the same as the indicated direction of the virtual traffic light.

[0080] In some optional implementations of this embodiment, the above-mentioned assigning unit 404 is further configured to: set virtual traffic lights, the virtual traffic lights including: virtual sub-lights indicating different directions, the directions of the virtual sub-lights including: going straight, turning left, turning right, and making a U-turn; set priority levels for all fusion shapes in at least one fusion shape; according to the priority order of each fusion shape, sequentially control the virtual sub-lights corresponding to the indicated directions of the fusion shapes in at least one fusion shape to turn on; and assign the status information corresponding to the fusion shape to the virtual sub-light.

[0081] In some optional implementations of this disclosure, the assigning unit 404 is further configured to: set a virtual traffic light; in response to the virtual traffic light having at least one blended shape indicating a direction, assign an at least one blended shape indicating a direction to the virtual traffic light; and when the virtual traffic light indicates a direction, assign state information corresponding to the direction to the virtual traffic light.

[0082] In some optional implementations of this disclosure, the fusion unit 402 is further configured to: determine the shape of the head of traffic lights with the same shape in the target light group based on the head status information, and take the head shape of the traffic lights with the same head as a fusion shape; detect whether the status information of each head in the traffic lights with the same head shape is the same; in response to the fact that the status information of at least one head is different, select the status information of the head of the traffic lights with the same shape, and take the selected status information as the status information of the fusion shape.

[0083] In some optional implementations of this disclosure, the acquisition unit 401 is further configured to: acquire images of each traffic light in the target light group during a preset time period; based on the images, determine the time state and state change value of each traffic light head at each moment, and use the time state and state change value as the state information of each traffic light head.

[0084] The apparatus for generating virtual traffic lights provided in the embodiments of this disclosure first obtains the state information of the lamp heads of each traffic light based on the acquired images of each traffic light in a target light group; second, based on the lamp head state information, it fuses the valid lamp head shapes in the target light group to obtain at least one fused shape and the state information of each fused shape; third, based on the mapping relationship between each fused shape and the indicated direction, it determines the indicated direction of each fused shape; finally, it sets up a virtual traffic light and assigns the state information of the at least one fused shape to the virtual traffic light based on the indicated direction of the at least one fused shape. Thus, the fused shape obtained by fusing valid lamp heads can effectively reflect the specific indicated direction of the target light group, providing a reliable basis for vehicles to indicate the current planned path direction, and greatly reducing the generation of redundant information when judging traffic light state information.

[0085] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0086] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0087] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0088] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0089] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0090] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the method for generating virtual traffic lights. For example, in some embodiments, the method for generating virtual traffic lights may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the method for generating virtual traffic lights described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the method for generating virtual traffic lights by any other suitable means (e.g., by means of firmware).

[0091] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable device for generating virtual traffic lights, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0095] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0096] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0097] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0098] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating virtual traffic lights, the method comprising: Based on the images of each traffic light in the target light group, the status information of each traffic light head is obtained; Based on the status information of the lamp head, the lamp head shape of traffic lights with the same shape in the target lamp group is determined, and the lamp head shape of traffic lights with the same lamp head shape is taken as a fusion shape to obtain at least one fusion shape and the status information of each fusion shape in the at least one fusion shape, wherein the fusion shape is derived from the display shape of the traffic light lamp head or the original appearance shape of the lamp head. Based on the mapping relationship between each of the at least one blended shapes and the indication direction, the indication direction of each of the at least one blended shapes is determined; A virtual traffic light is set up, and the status information of the at least one fused shape is assigned to the virtual traffic light based on the indication direction of the at least one fused shape.

2. The method according to claim 1, wherein, The virtual traffic light is the traffic light that the autonomous vehicle is about to face at the intersection, and the method further includes: The virtual traffic light's indicated direction and its status information are sent to the autonomous vehicle's planning and control module, so that when the planning direction matches the virtual traffic light's indicated direction, the planning and control module executes the driving operation corresponding to the virtual traffic light's status information.

3. The method according to claim 1, wherein, The step of setting up a virtual traffic light, based on the indicating direction of the at least one merged shape, and assigning the state information of the at least one merged shape to the virtual traffic light includes: The virtual traffic lights are configured, including virtual sub-lights indicating different directions, wherein the directions of the virtual sub-lights include any one of: going straight, turning left, turning right, and making a U-turn. Set a priority level for all merged shapes in the at least one merged shape; According to the priority order of each fusion shape, the virtual sub-lights corresponding to the indication direction of the fusion shape in the at least one fusion shape are turned on in sequence; The state information corresponding to the fused shape is then assigned to the virtual sub-lamp.

4. The method according to claim 1, wherein, The step of setting up a virtual traffic light, based on the indicating direction of the at least one merged shape, and assigning the state information of the at least one merged shape to the virtual traffic light includes: Set up virtual traffic lights; In response to the representation that the virtual traffic light has an indication direction of at least one blended shape, the indication direction of the at least one blended shape is assigned to the virtual traffic light; When the virtual traffic light represents the indicated direction, the state information corresponding to the indicated direction is assigned to the virtual traffic light.

5. The method according to claim 1, wherein, Also includes: The status information of each lamp head in the traffic lights with the same lamp head shape is detected to be the same. In response to the fact that the status information of at least one light head is different, the status information of the light heads of the traffic lights with the same shape is selected and used as the status information of the fused shape.

6. The method according to any one of claims 1-5, wherein, The process of obtaining the status information of each traffic light head based on the acquired images of each traffic light in the target light group includes: Acquire images of each traffic light in the target light group within a preset time period; Based on the image, the time state and state change value of each traffic light head at each moment are determined, and the time state and the state change value are used as the state information of each traffic light head.

7. An apparatus for generating virtual traffic lights, the apparatus comprising: The acquisition unit is configured to obtain the status information of each traffic light head based on the images of each traffic light in the target light group. The fusion unit is configured to determine the shape of the head of traffic lights with the same shape in the target light group based on the state information of the head, take the head shape of the traffic lights with the same head as a fusion shape, and obtain at least one fusion shape and the state information of each of the at least one fusion shape, wherein the fusion shape is derived from the display shape of the traffic light head or the original appearance shape of the head. The determining unit is configured to determine the indicating direction of each of the at least one blended shapes based on the mapping relationship between each blended shape and the indicating direction. The assigning unit is configured to set a virtual traffic light, assigning the status information of the at least one fused shape to the virtual traffic light based on the indicating direction of the at least one fused shape.

8. The apparatus according to claim 7, wherein, The virtual traffic light is the traffic light that the autonomous vehicle is about to face at the intersection, and the device also includes: The sending unit is configured to send the indicated direction of the virtual traffic light and the status information of the virtual traffic light to the planning and control module of the autonomous vehicle, so that the planning and control module performs the driving operation corresponding to the status information of the virtual traffic light when the planning direction is the same as the indicated direction of the virtual traffic light.

9. The apparatus according to claim 7, wherein, The assigning unit is further configured to: set virtual traffic lights, the virtual traffic lights including: virtual sub-lights indicating different directions, the directions of the virtual sub-lights including: going straight, turning left, turning right, and making a U-turn; Set a priority level for all merged shapes in the at least one merged shape; According to the priority order of each fusion shape, the virtual sub-lights corresponding to the indication direction of the fusion shape in the at least one fusion shape are turned on in sequence; The state information corresponding to the fused shape is then assigned to the virtual sub-lamp.

10. The apparatus according to claim 7, wherein, The assigning unit is further configured to: set a virtual traffic light; and, in response to the virtual traffic light having an indication direction of the at least one blended shape, assign the indication direction of the at least one blended shape to the virtual traffic light; When the virtual traffic light represents the indicated direction, the state information corresponding to the indicated direction is assigned to the virtual traffic light.

11. The apparatus according to claim 7, wherein, The device further includes: The detection unit is configured to detect whether the state information of each lamp head in the traffic lights with the same lamp head shape is the same; in response to at least one lamp head having different state information, the detection unit selects the state information of the lamp heads in the traffic lights with the same shape and uses the selected state information as the state information of the fused shape.

12. The apparatus according to any one of claims 7-11, wherein, The acquisition unit is further configured to: acquire images of each traffic light in the target light group during a preset time period; based on the images, determine the time state and state change value of each traffic light head at each moment, and use the time state and the state change value as the state information of each traffic light head.

13. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images