Data generation method, sensing method, device and storage medium for traffic signal light

Abstract

This application relates to the field of autonomous driving technology, specifically providing a data generation method, perception method, device, and storage medium for traffic lights, aiming to solve the problem of how to quickly and accurately acquire traffic light image annotation data. To this end, the method provided in this application includes tracking and detecting individual lights in each frame of a video clip to obtain the shape and state of the light head; clustering all individual lights in the target image to obtain at least one cluster, and using the cluster as a traffic light intersection; acquiring the lit individual lights within the traffic light intersection, acquiring the traffic direction of the virtual light corresponding to the shape of the lit individual light's light head, and determining the state of the traffic direction based on the state of the lit individual light; generating virtual lights for the traffic light intersection based on the traffic direction and its state; and annotating the target image based on the virtual lights to form image annotation data. Based on the above method, traffic light image annotation data can be acquired quickly and accurately.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to a data generation method, perception method, device, and storage medium for traffic lights. Background Technology

[0002] Traffic lights at intersections primarily use the color of individual lights to indicate the passage status for each traffic direction (such as straight ahead, left turn, right turn, etc.). For example, a green light indicates permission to proceed, while a red light indicates permission to proceed. When controlling autonomous driving at intersections, it's crucial to accurately obtain the passage status of the traffic lights indicating each direction and then determine the vehicle's driving decisions based on that status. For instance, if a vehicle's planned trajectory is to go straight through the intersection, but the passage status for that direction is prohibited, the vehicle must be stopped at the stop line before the intersection and wait for the passage status to change to permitted before proceeding straight through the intersection.

[0003] To accurately obtain the traffic light status for each direction and ensure vehicle safety, a camera on a vehicle can capture images of traffic lights. A perception model then processes these images to identify the traffic lights and determine their status (e.g., color) for each direction. Based on this status, the appropriate traffic flow is determined. For example, if the color of the direction indicator is green, the flow is permitted; if the color is green, the flow is prohibited. The perception model is typically a neural network model, requiring extensive training data to accurately identify the traffic light status from the images. The training data consists of images of traffic lights and their annotations, including the status of the traffic light indicating the desired direction.

[0004] Traffic lights consist of individual lights. When a single light is lit, it typically displays information such as its color and the shape of its head (e.g., a disc or arrow), but does not directly indicate the direction of traffic. Therefore, when acquiring training data, annotators need to identify which traffic directions each light in the image indicates, determine the signal light status (e.g., whether passage is permitted) based on the light's color, and then annotate the image to form annotation information. This is a manual annotation method. However, manual annotation is inefficient and prone to introducing errors, affecting the accuracy of the annotation information. Using manual annotation to acquire training data not only impacts the training efficiency of the perception model but may also affect its performance, thus hindering the accurate perception of the traffic light status in the image.

[0005] Accordingly, a new technical solution is needed in this field to solve the above problems. Summary of the Invention

[0006] In order to overcome the above-mentioned deficiencies, this application is made to solve or at least partially solve the following technical problem: how to quickly and accurately obtain traffic light image annotation data.

[0007] In a first aspect, a method for generating traffic light data is provided, the method comprising:

[0008] Acquire video clips of the vehicle's driving environment captured by a camera on the vehicle, wherein there is at least one traffic light in the environment;

[0009] Tracking and detecting individual lights in each frame of the video segment yields the tracking ID and semantic information of each light in each frame. The semantic information includes the shape of the light head and the state of the individual light. Each frame is an image of each time frame within the video segment.

[0010] Each frame of image is taken as the target image in sequence. Based on the position of the single light in the target image, all the single lights in the target image are clustered to obtain at least one cluster. The cluster is taken as the traffic light intersection appearing in the target image.

[0011] The system acquires the illuminated single light in the traffic light intersection, acquires the traffic direction corresponding to the shape of the illuminated single light head according to a preset correspondence, and determines the state of the traffic direction according to the single light state of the illuminated single light. The preset correspondence is the correspondence between the shape of the light head and the traffic direction of the virtual light.

[0012] Based on the traffic direction and its status, generate virtual lights for the traffic light intersection;

[0013] The target image is labeled based on the virtual light to form image labeling data.

[0014] In one technical solution of the above data generation method, the step of clustering all individual lights in the target image based on their positions includes:

[0015] Based on the position of each individual light in the target image, a first candidate light is obtained from all the individual lights in the target image; wherein, the first candidate light is located in front of the vehicle and the yaw angle between it and the vehicle in the vehicle coordinate system is less than a preset angle threshold.

[0016] Based on the position of the first candidate light, all first candidate lights are clustered.

[0017] In one technical solution of the above data generation method, the method includes: when multiple traffic light intersections appear in the target image, obtaining the distance between each pair of traffic light intersections; if the distance between two traffic light intersections is less than a preset distance threshold, merging the two traffic light intersections into one traffic light intersection.

[0018] In one technical solution of the above data generation method, the method includes: tracking traffic light intersections appearing in the processed image to obtain the tracking IDs of all traffic light intersections, wherein the processed image includes the target image and its previous historical images;

[0019] Obtain the tracking ID of the traffic light intersection appearing in the target image, and update the tracking status of the traffic light intersection based on the tracking ID.

[0020] In one technical solution of the above data generation method, the method includes: obtaining the tracking status of the traffic light intersection based on the tracking ID of the traffic light intersection;

[0021] Obtain the duration during which the tracking status has not been updated;

[0022] If the duration exceeds the third preset duration, the traffic light intersection is discarded.

[0023] In one technical solution of the above data generation method, determining the state of the traffic direction indication based on the state of the single illuminated light includes:

[0024] When there are multiple illuminated single lights with the same lamp head shape at the traffic light intersection, a second candidate light is obtained from all the illuminated single lights with the same lamp head shape; wherein, the second candidate light has the smallest distance between itself and the vehicle in the vehicle coordinate system;

[0025] The state of the traffic direction indication is determined based on the individual light status of the second candidate light.

[0026] In one technical solution of the above data generation method, when the single light being illuminated includes a turn signal and a countdown light, the single light state of the countdown light also includes a countdown digit. Determining the state of the traffic direction indication based on the single light state includes:

[0027] Obtain a turn signal of the same color as the countdown light;

[0028] The Hungarian matching method is used to match the countdown light with each turn signal of the same color based on the distance between the countdown light and each turn signal of the same color, and to obtain a turn signal of the same color that matches the countdown light as the target turn signal;

[0029] Add the countdown numbers of the countdown light to the single-lamp state of the target turn signal;

[0030] The state of the traffic direction indication is determined based on the single-lamp state of the target turn signal.

[0031] In one technical solution of the above data generation method, the method further includes obtaining the target turn signal in the following manner:

[0032] Obtain all target turn signals that match the countdown light in the processed image, track all target turn signals, and obtain the tracking ID of each target turn signal. The processed image includes the target image and its previous historical images.

[0033] A voting process is performed on the tracking IDs of all the target turn signals to obtain the target turn signal indicated by the tracking ID that appears most frequently. This target turn signal is then selected as the final target turn signal in the target image that matches the countdown light.

[0034] In one technical solution of the above data generation method, the method includes smoothing the virtual lights of the traffic light intersection in the target image by means of the following:

[0035] Track the traffic light intersections appearing in the processed images to obtain the tracking IDs of all traffic light intersections. The processed images include the target image and its previous historical images.

[0036] The traffic light intersection appearing in the target image is taken as the target intersection;

[0037] The virtual lights of the target intersection in the target image are obtained, and the virtual lights of the target intersection in the historical image are obtained according to the tracking ID of the target intersection.

[0038] The virtual lights of the target intersection in the target image and the historical image are smoothed to obtain the final virtual lights of the target intersection in the target image.

[0039] In a second aspect, a method for sensing traffic lights is provided, the method comprising: acquiring an image of a vehicle driving environment; inputting the image into a perception model to obtain a virtual light of the image, the virtual light being composed of multiple traffic direction indicators and their states; wherein the perception model is trained using image annotation data, the image annotation data being obtained using any of the traffic light data generation methods provided in the first aspect.

[0040] In a third aspect, an electronic device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program that, when executed by the at least one processor, implements the method described in any of the technical solutions provided in the first aspect.

[0041] In a fourth aspect, a smart device is provided, the smart device comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the technical solutions provided in the second aspect above.

[0042] In a fifth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and executed by a processor to perform the method described in any of the technical solutions provided in the first or second aspect above.

[0043] Solution 1. A method for generating traffic signal light data, characterized in that the method includes:

[0044] Acquire video clips of the vehicle's driving environment captured by a camera on the vehicle, wherein there is at least one traffic light in the environment;

[0045] Tracking and detecting individual lights in each frame of the video segment yields the tracking ID and semantic information of each light in each frame. The semantic information includes the shape of the light head and the state of the individual light. Each frame is an image of each time frame within the video segment.

[0046] Each frame of image is taken as the target image in sequence. Based on the position of the single light in the target image, all the single lights in the target image are clustered to obtain at least one cluster. The cluster is taken as the traffic light intersection appearing in the target image.

[0047] The system acquires the illuminated single light in the traffic light intersection, acquires the traffic direction corresponding to the shape of the illuminated single light head according to a preset correspondence, and determines the state of the traffic direction according to the single light state of the illuminated single light. The preset correspondence is the correspondence between the shape of the light head and the traffic direction of the virtual light.

[0048] Based on the traffic direction and its status, generate virtual lights for the traffic light intersection;

[0049] The target image is labeled based on the virtual light to form image labeling data.

[0050] Solution 2. The method according to Solution 1, characterized in that, the step of clustering all single lights in the target image based on the position of the single light in the target image includes:

[0051] Based on the position of each individual light in the target image, a first candidate light is obtained from all the individual lights in the target image; wherein, the first candidate light is located in front of the vehicle and the yaw angle between it and the vehicle in the vehicle coordinate system is less than a preset angle threshold.

[0052] Based on the position of the first candidate light, all first candidate lights are clustered.

[0053] Solution 3. The method according to Solution 1, characterized in that the method includes: when multiple traffic light intersections appear in the target image, obtaining the distance between each pair of traffic light intersections; if the distance between two traffic light intersections is less than a preset distance threshold, merging the two traffic light intersections into one traffic light intersection.

[0054] Solution 4. The method according to Solution 1, characterized in that the method includes: tracking traffic light intersections appearing in the processed image to obtain the tracking IDs of all traffic light intersections, wherein the processed image includes the target image and its previous historical images;

[0055] Obtain the tracking ID of the traffic light intersection appearing in the target image, and update the tracking status of the traffic light intersection based on the tracking ID.

[0056] Solution 5. The method according to Solution 4, characterized in that the method includes: obtaining the tracking status of the traffic light intersection based on the tracking ID of the traffic light intersection;

[0057] Obtain the duration during which the tracking status has not been updated;

[0058] If the duration exceeds the third preset duration, the traffic light intersection is discarded.

[0059] Solution 6. The method according to Solution 1, characterized in that, determining the state of the traffic direction indication based on the state of the single illuminated lamp includes:

[0060] When there are multiple illuminated single lights with the same lamp head shape at the traffic light intersection, a second candidate light is obtained from all the illuminated single lights with the same lamp head shape; wherein, the second candidate light has the smallest distance between itself and the vehicle in the vehicle coordinate system;

[0061] The state of the traffic direction indication is determined based on the individual light status of the second candidate light.

[0062] Solution 7. The method according to Solution 1, characterized in that, when the single light being illuminated includes a turn signal and a countdown light, the single light state of the countdown light further includes a countdown digit, and determining the state of the traffic direction indication based on the single light state of the illuminated single light includes:

[0063] Obtain a turn signal of the same color as the countdown light;

[0064] The Hungarian matching method is used to match the countdown light with each turn signal of the same color based on the distance between the countdown light and each turn signal of the same color, and to obtain a turn signal of the same color that matches the countdown light as the target turn signal;

[0065] Add the countdown numbers of the countdown light to the single-lamp state of the target turn signal;

[0066] The state of the traffic direction indication is determined based on the single-lamp state of the target turn signal.

[0067] Solution 8. The method according to Solution 7, characterized in that the method further includes acquiring the target turn signal by:

[0068] Obtain all target turn signals that match the countdown light in the processed image, track all target turn signals, and obtain the tracking ID of each target turn signal. The processed image includes the target image and its previous historical images.

[0069] A voting process is performed on the tracking IDs of all the target turn signals to obtain the target turn signal indicated by the tracking ID that appears most frequently. This target turn signal is then selected as the final target turn signal in the target image that matches the countdown light.

[0070] Solution 9. The method according to Solution 1, characterized in that the method includes smoothing the virtual lights of the traffic light intersection in the target image by means of the following:

[0071] Track the traffic light intersections appearing in the processed images to obtain the tracking IDs of all traffic light intersections. The processed images include the target image and its previous historical images.

[0072] The traffic light intersection appearing in the target image is taken as the target intersection;

[0073] The virtual lights of the target intersection in the target image are obtained, and the virtual lights of the target intersection in the historical image are obtained according to the tracking ID of the target intersection.

[0074] The virtual lights of the target intersection in the target image and the historical image are smoothed to obtain the final virtual lights of the target intersection in the target image.

[0075] Option 10. A method for sensing traffic lights, characterized in that the method includes:

[0076] Acquire images of the vehicle's driving environment;

[0077] The image is input into a perception model to obtain a virtual light for the image. The virtual light consists of multiple traffic direction indicators and their states.

[0078] The perception model is trained using image-annotated data, which is obtained using the traffic light data generation method described in any one of schemes 1 to 9.

[0079] Solution 11. An electronic device, characterized in that it comprises:

[0080] At least one processor;

[0081] And, a memory communicatively connected to the at least one processor;

[0082] The memory stores a computer program, which, when executed by the at least one processor, implements the traffic signal light data generation method as described in any one of schemes 1 to 9.

[0083] Option 12. A smart device, characterized in that it comprises:

[0084] At least one processor;

[0085] And, a memory communicatively connected to the at least one processor;

[0086] The memory stores a computer program, which, when executed by the at least one processor, implements the traffic light sensing method of scheme 10.

[0087] Scheme 13. A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the traffic light data generation method of any one of Schemes 1 to 9, or the traffic light sensing method of Scheme 10.

[0088] The above-described technical solutions of this application have at least one or more of the following beneficial effects:

[0089] In one technical solution of the traffic light data generation method provided in this application, a video clip of the vehicle's driving environment captured by a camera on the vehicle is acquired, in which there is at least one traffic light; individual lights in each frame of the video clip are tracked and detected to obtain the tracking ID and semantic information of each individual light in each frame, the semantic information including the shape of the light head and the state of the individual light, and each frame is an image of each time frame in the video clip; each frame is used as a target image in sequence, and all individual lights in the target image are clustered according to the position of the individual lights in the target image to obtain at least one cluster, and the cluster is used as the traffic light intersection appearing in the target image; the lit individual lights in the traffic light intersection are acquired, and the traffic direction corresponding to the shape of the light head of the lit individual light is acquired according to a preset correspondence, and the state of the traffic direction is determined according to the state of the lit individual light, the preset correspondence being the correspondence between the shape of the light head and the traffic direction of the virtual light; virtual lights of the traffic light intersection are generated according to the traffic direction and its state; the target image is labeled according to the virtual lights to form image labeling data.

[0090] Based on the above implementation method, virtual light annotation can be automatically completed for each frame of a video clip, from the first frame to the last frame, eliminating the need for manual annotation, greatly improving annotation efficiency, and effectively ensuring annotation accuracy. Furthermore, the virtual lights consist of multiple traffic direction indicators and their states. Using the annotated image data of the virtual lights to train the perception model allows the model to directly map or obtain the virtual lights (including traffic direction indicators and their states) from the images captured by the camera. This establishes a direct mapping relationship between the input (image) and the output (virtual lights), achieving end-to-end acquisition of traffic direction indicators and their states. Based on this, the perception model can quickly and accurately acquire the traffic direction indicators and their states in the vehicle's environment, ensuring driving safety. Attached Figure Description

[0091] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Wherein:

[0092] Figure 1 This is a schematic flowchart of the main steps of a traffic signal light data generation method according to an embodiment of this application;

[0093] Figure 2 This is a schematic diagram of a physical traffic light and a virtual light according to an embodiment of this application. Figure 1 ;

[0094] Figure 3This is a schematic flowchart illustrating the main steps of clustering all individual lights in a target image according to an embodiment of this application;

[0095] Figure 4 This is a schematic flowchart of the main steps for tracking and processing traffic light intersections according to an embodiment of this application;

[0096] Figure 5 This is a flowchart illustrating the main steps for determining the state of virtual traffic light directions according to an embodiment of this application. Figure 1 ;

[0097] Figure 6 This is a flowchart illustrating the main steps for determining the state of virtual traffic light directions according to an embodiment of this application. Figure 2 ;

[0098] Figure 7 This is a schematic diagram of a physical traffic light and a virtual light according to an embodiment of this application. Figure 2 ;

[0099] Figure 8 This is a schematic flowchart illustrating the main steps of smoothing virtual lights at a traffic light intersection in a target image according to an embodiment of this application.

[0100] Figure 9 This is a schematic diagram of the overall process of a traffic signal light data generation method according to an embodiment of this application;

[0101] Figure 10 This is a schematic diagram illustrating the process of generating and updating traffic light intersections according to an embodiment of this application;

[0102] Figure 11 This is a schematic diagram of the process of obtaining a virtual light according to an embodiment of this application;

[0103] Figure 12 This is a schematic flowchart of the main steps of a traffic signal light sensing method according to an embodiment of this application;

[0104] Figure 13 This is a schematic diagram of the main structure of an electronic device according to an embodiment of this application;

[0105] Figure 14 This is a schematic diagram of the main structure of a smart device according to an embodiment of this application.

[0106] Figure label:

[0107] 11: Memory; 12: Processor; 21: Memory; 22: Processor. Detailed Implementation

[0108] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0109] In the description of this application, "processor" can include hardware, software, or a combination of both. A processor can be a central processing unit, microprocessor, graphics processor, digital signal processor, or any other suitable processor. A processor has data and / or signal processing capabilities. A processor can be implemented in software, in hardware, or a combination of both. Computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B.

[0110] The relevant user personal information that may be involved in the various embodiments of this application is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and includes personal information that users actively provide or that is generated as a result of using the product / service, as well as personal information obtained with user authorization.

[0111] The personal information processed in this application will vary depending on the specific product / service scenario and will be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. This application will treat the user's personal information and its processing with the utmost diligence.

[0112] This application attaches great importance to the security of users' personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect users' information and prevent unauthorized access, disclosure, use, modification, damage or loss of personal information.

[0113] The following describes an embodiment of the traffic signal data generation method provided in this application.

[0114] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a traffic signal light data generation method according to an embodiment of this application. Figure 1 As shown, the data generation method for traffic lights in this embodiment mainly includes the following steps S101 to S106.

[0115] Step S101: Obtain a video clip of the vehicle's driving environment captured by the camera on the vehicle, where there is at least one traffic light.

[0116] The vehicle is equipped with at least one camera. When a camera is installed, it can at least capture images of the driving environment in front of the vehicle. Multiple consecutive frames of images are superimposed to form a video. That is, the video clips in this application are captured by the cameras on the vehicle. In some embodiments, multiple cameras can be installed on the vehicle, distributed in different positions on the vehicle. One camera can at least capture images of the driving environment in front of the vehicle, while the other cameras can capture images of the environment behind or to the sides of the vehicle.

[0117] Step S102: Track and detect individual lights in each frame of the video segment to obtain the tracking ID (Identity Document) and semantic information of each individual light in each frame. The semantic information includes the shape of the light head and the state of the individual light.

[0118] Each frame image is an image from each time frame within a video clip. The time interval between any two adjacent time frames in a video clip is fixed. Each time frame of the video clip is a still image, and each frame image can be understood as a still image presented in the video clip at each time frame.

[0119] A traffic light includes at least one individual light. When the same individual light of a traffic light is tracked and detected in different frame images, it will be assigned the same tracking ID; conversely, when different individual lights of a traffic light are tracked and detected in the same frame image, they will be assigned different tracking IDs. In some embodiments, the 3D position of the individual light in each frame image can be obtained, and the individual light in each frame image can be tracked based on the 3D position to obtain the tracking ID of the individual light in each frame image. The 3D position can be the three-dimensional position of the individual light in the world coordinate system. When obtaining the 3D position, the two-dimensional position (2D position) of the individual light in the image coordinate system can be detected, and then the 2D position can be transformed to the world coordinate system according to the transformation relationship between the image coordinate system and the world coordinate system to obtain the 3D position of the individual light. Furthermore, in this embodiment, conventional tracking methods can be used to track the individual light based on its 3D position; this embodiment does not specifically limit this method.

[0120] In some implementations, a semantic recognition model can be used to detect 2D bounding boxes for a single light in an image, and semantic recognition of the image region containing the 2D bounding box can be performed to obtain the semantic information of the single light. In this implementation, a conventional semantic recognition model can be used to detect 2D bounding boxes and perform semantic recognition of a single light; this implementation does not impose specific limitations on this method.

[0121] In some implementations, the tracking ID and semantic information of the same single light in an image can be associated. Based on this, the semantic information of a single light can be queried using its tracking ID. In some implementations, when associating the tracking ID and semantic information, the 3D position of a single light in the image can be projected onto the image coordinate system to obtain its 2D position. Then, the positional deviation between this 2D position and the aforementioned 2D detection box is obtained. If the positional deviation is less than or equal to a preset deviation threshold, it indicates that the two positions are the same or similar. Therefore, the tracking ID of the single light corresponding to the 2D position is associated with the semantic information of the single light corresponding to the 2D detection box; otherwise, no association is made. When setting the value of the deviation threshold, the maximum error between the 2D position and the 2D detection box of the same single light can be obtained through testing, and the deviation threshold is set based on this maximum error.

[0122] The following explains the single-lamp state and lamp head shape in semantic information.

[0123] 1. Explain the status of a single lamp.

[0124] The status of a single light can include its color, which includes the illuminated color when the light is on and the off color when the light is off. The illuminated color is used for traffic guidance; if a single light displays the illuminated color in multiple consecutive frames of images, then that light can be determined to be on. In some implementations, the illuminated color can include green, red, and yellow, where green indicates permission to proceed, red indicates prohibition to proceed, and yellow indicates a warning. The off color can be black.

[0125] In some implementations, the single-lamp state may also include a countdown number. The countdown number can be understood as the countdown number for the remaining time of the currently lit color by the single lamp. The change of the countdown number reflects the countdown process of the single lamp.

[0126] 2. Describe the shape of the lamp holder.

[0127] Based on the semantic meaning of the lamp head shape, a single lamp can include turn signals and countdown lights. The semantic meaning of the turn signal lamp head shape is the traffic direction indication. The shape of the turn signal lamp head can be a disc, a straight arrow, a left turn arrow, a right turn arrow, and a U-turn arrow, etc. Different lamp head shapes can indicate different traffic directions. For example, a disc indicates traffic directions including straight, left turn, and U-turn; a straight arrow indicates traffic direction of going straight.

[0128] The countdown timer light has a number on its bulb, indicating the remaining time for the current illuminated color. After the remaining time reaches 0, the turn signal will display a different color. For example, if the current illuminated color is green and the countdown timer shows 15, it means there are 15 seconds left in the green light, after which the turn signal will turn red.

[0129] Step S103: Take each frame image as the target image in sequence, and cluster all the single lights in the target image according to the position of the single light in the target image to obtain at least one cluster. Take the cluster as the traffic light intersection that appears in the target image.

[0130] In this embodiment, the images of each time frame within the video clip can be sequentially used as the target images, following the chronological order. Clustering is performed based on the location of individual lights, grouping lights that are close together. In real-world scenarios, lights located at the same intersection are relatively close, while lights at different intersections are relatively far apart. Therefore, a cluster can be considered a traffic light intersection, containing all lights within that cluster. In this embodiment, conventional clustering methods can be used to cluster lights based on their location.

[0131] The position of a single lamp can be its three-dimensional position (3D position) in the world coordinate system. In some implementations, the two-dimensional position (2D position) of a single lamp in the image coordinate system can be detected, and then, based on the transformation relationship between the image coordinate system and the world coordinate system, the 2D position can be transformed to the world coordinate system to obtain the 3D position of the single lamp. In some implementations, the target image has annotation information of the single lamp position, and the 3D position of the single lamp can be directly obtained from the annotation information. This annotation information can be pre-annotated on the video clip by manual annotation, and this annotation information can be obtained synchronously when the image is obtained from the video clip.

[0132] In some implementations, when multiple traffic light intersections appear in the target image, the distance between each pair of traffic light intersections is obtained separately; if the distance between two traffic light intersections is less than a preset distance threshold, the two traffic light intersections are merged into one. Specifically, the average value of the positions of all individual lights in the traffic light intersection can be obtained, and this average value can be used as the position of the traffic light intersection. Then, the distance between two traffic light intersections can be calculated based on this position.

[0133] Step S104: Obtain the illuminated individual lights within the traffic light intersection. Based on a preset correspondence, obtain the traffic direction corresponding to the shape of the illuminated individual light's head. Determine the state of the traffic direction based on the state of the illuminated individual light. The preset correspondence is the relationship between the shape of the light head and the traffic direction of the virtual light. This preset correspondence is a pre-set relationship that is directly invoked when the traffic direction needs to be obtained.

[0134] The virtual traffic light consists of multiple traffic directions and the status of each direction. In some implementations, the multiple traffic directions may include going straight, turning left, turning right, and making a U-turn.

[0135] The state of traffic direction indicators can include color. For example, if the color for going straight is red, it means going straight is prohibited; if the color for going straight is green, it means going straight is permitted. In some implementations, the state of traffic direction indicators may also include flashing status, countdown numbers, etc. When determining the state of traffic direction indicators, the state of a single light can be assigned to the state of the traffic direction indicator.

[0136] In some implementations, the state of a single illuminated lamp can be used as the state of the lamp head shape corresponding to the illuminated lamp. Based on a preset correspondence, the traffic direction corresponding to the lamp head shape can be obtained, and then the state of the single lamp corresponding to the lamp head shape is assigned to the traffic direction corresponding to the lamp head shape. For example, if the illuminated lamp head shape is a disc and the color is red, red is used as the color corresponding to the disc. The traffic directions corresponding to the disc include straight ahead, left turn, and U-turn. Therefore, red is assigned as the color for straight ahead, left turn, and U-turn; that is, the color for straight ahead, left turn, and U-turn is all red.

[0137] In some implementations, since different light head shapes may represent the same traffic direction, if the individual light states corresponding to these two light head shapes are different, it may lead to different states for the same traffic direction, ultimately affecting the accuracy of the virtual light. Therefore, when determining the state of the traffic direction, a preset arrangement order of light head shapes can be followed, and a state value can be assigned to the traffic direction corresponding to each light head shape based on the individual light state of each shape. This ensures that even if different states exist for the same traffic direction, the state will be determined by the last light head shape representing that direction, preventing confusion. In one implementation, the light head shapes include discs, straight arrows, left-turn arrows, right-turn arrows, and U-turn arrows. The arrangement order of these light head shapes is right-turn arrow, disc, straight arrow, left-turn arrow, and U-turn arrow. The traffic directions represented by these light head shapes are shown in Table 1 below. Table 1 Arrangement order lamp head shape Traffic directions 1 Right arrow Turn right 2 round cake Go straight, turn left, make a U-turn 3 Straight arrow Go straight, turn left, make a U-turn 4 Left turn arrow Turn left, make a U-turn 5 Turning arrow U-turn

[0138] Step S105: Generate virtual traffic lights for the intersection based on traffic direction indications and their states. As described in step S104 above, a virtual light consists of multiple traffic direction indications and the states of each direction. Therefore, after determining the traffic direction indications and states of the virtual light, these directions and states can be combined or merged to form the virtual light. Through this step, the physical traffic lights in the vehicle's driving environment can be converted into virtual traffic lights (i.e., virtual lights). By querying the traffic direction indications and states from the virtual lights, the states of the physical traffic lights can be obtained.

[0139] See appendix Figure 2 , Figure 2 This is a schematic diagram of a virtual traffic light generated based on a physical traffic light. (Example:) Figure 2 As shown, the traffic light has four illuminated individual lights. Two of these lights have left-turn arrow shapes and are red, while the other two have disc shapes and are green. Therefore, the light corresponding to the left-turn arrow is red, and the light corresponding to the disc is green. Furthermore, according to Table 1, the traffic direction indicated by the virtual light corresponding to the left-turn arrow includes left turn and U-turn, while the traffic direction indicated by the virtual light corresponding to the disc includes straight ahead, left turn, and U-turn. Since the order of the light shapes is right-turn arrow, disc, straight ahead arrow, left-turn arrow, and U-turn arrow, the green light corresponding to the disc is first assigned to straight ahead, left turn, and U-turn; at this point, straight ahead, left turn, and U-turn are all green. Then, the red light corresponding to the left-turn arrow is assigned to left turn and U-turn; at this point, left turn and U-turn change from green to red, while straight ahead remains green. In this embodiment, the color of the right-turn traffic direction is initialized to indicate that passage is permitted. Finally, it can be determined that the colors of the four traffic direction indicators in the virtual light—U-turn, left turn, straight ahead, and right turn—are red, red, green, and green, respectively. Figure 2 The negative 1 indicates that there is no countdown timer. Since physical traffic lights do not have countdown lights and cannot obtain countdown data, negative 1 is used to represent no countdown timer or an unknown countdown. If there is a countdown timer, the detected countdown timer will be displayed.

[0140] Step S106: Annotate the target image based on the virtual lights to form image annotation data. Specifically, the virtual lights can be used as annotation information to annotate the target image.

[0141] Based on the methods described in steps S101 to S106 above, virtual light annotations can be automatically completed for each frame of a video clip, from the first frame to the last frame, eliminating the need for manual annotation and greatly improving annotation efficiency while effectively ensuring annotation accuracy. Furthermore, training the perception model using the annotated image data allows the model to directly map or obtain virtual lights (including traffic direction indicators and their states) from the images captured by the camera. This establishes a direct mapping relationship between the input (image) and the output (virtual light), achieving end-to-end acquisition of traffic direction indicators and their states. Therefore, the perception model can quickly and accurately acquire traffic direction indicators and their states in the vehicle's environment, ensuring driving safety.

[0142] The following description continues with an embodiment of the traffic light data generation method provided in this application, specifically describing steps S103 to S104 above.

[0143] I. Step S103 in the foregoing embodiments will be explained.

[0144] In some embodiments of step S103 above, it can be achieved by... Figure 3 The following steps S1031 to S1032 are shown to cluster all individual lights in the target image.

[0145] Step S1031: Based on the position of each lamp in the target image, obtain the first candidate lamp from all the lamps in the target image; wherein, the first candidate lamp is located in front of the vehicle and the yaw angle between it and the vehicle in the vehicle coordinate system is less than a preset angle threshold.

[0146] Yaw angle can be understood as the angle between the vector formed by the vehicle pointing towards a single lamp and the Z-axis of the vehicle coordinate system. The smaller the yaw angle, the more likely the single lamp is to be located on the road the vehicle is currently traveling on. Therefore, when the first candidate lamp is in front of the vehicle and the yaw angle is less than a preset angle threshold, it indicates that the first candidate lamp is located on the road in front of the vehicle.

[0147] When setting a preset angle threshold, a large number of individual lights actually located on the road in front of the vehicle can be obtained, and the yaw angle between these individual lights and the vehicle can be obtained. The largest yaw angle can be obtained from these yaw angles, and the angle threshold can be set based on this largest yaw angle.

[0148] Step S1032: Cluster all first candidate lights according to their positions. The clustering method is the same as that in step S103 above, and will not be described again.

[0149] Based on the method described in steps S1031 to S1032 above, single lights located on the road in front of the vehicle can be screened out, while single lights located behind the vehicle or not on the road can be filtered out, reducing interference from these single lights and helping to improve the accuracy of traffic light intersections.

[0150] In some embodiments of step S103 above, it can be achieved by... Figure 4 The following steps S1033 to S1034 are used to track and process traffic light intersections.

[0151] Step S1033: Track the traffic light intersections appearing in the processed images to obtain the tracking IDs of all traffic light intersections. The processed images include the target image and its previous historical images. Specifically, the average position of all individual lights within the traffic light intersection can be obtained, and this average position can be used as the position of the traffic light intersection. The traffic light intersections are then tracked based on their positions. Step S1034: Obtain the tracking IDs of the traffic light intersections appearing in the target image and update the tracking status of the traffic light intersections based on these tracking IDs. Specifically, for each traffic light intersection indicated by a tracking ID, if that intersection appears in the target image, it indicates that the intersection has been tracked, and its tracking status needs to be updated.

[0152] In some implementations, the tracking status of a traffic light intersection can be obtained based on its tracking ID. Then, the duration during which the tracking status has not been updated can be obtained. If the duration is longer than a third preset duration, it indicates that the traffic light intersection has not been tracked for a long time, and the vehicle may have already passed through the traffic light intersection. Therefore, the traffic light intersection can be discarded.

[0153] II. Step S104 in the foregoing embodiments will be explained.

[0154] In some embodiments of step S104 above, it can be achieved by... Figure 5 The following steps S1041 to S1042 are shown to determine the state of the virtual traffic light direction indication.

[0155] Step S1041: When multiple illuminated single lights with the same head shape are present at a traffic light intersection, a second candidate light is selected from all illuminated single lights with the same head shape; the second candidate light has the smallest distance to the vehicle in the vehicle coordinate system. The closer the distance between the single light and the vehicle in the vehicle coordinate system, the more likely the single light is to be located in the vehicle's lane; therefore, the single light with the smallest distance is selected as the second candidate single light. Step S1042: Based on the single light status of the second candidate light, the traffic direction indication status is determined. This implementation method can further filter illuminated single lights located on the road ahead of the vehicle, helping to improve the accuracy of the virtual lights.

[0156] In some embodiments of step S104 above, if illuminating a single lamp includes both a turn signal and a countdown timer, the single lamp status of the countdown timer also includes the countdown digits. In this embodiment, it can be achieved through... Figure 6 The following steps S1043 to S1046 determine the state of traffic direction indication based on the state of the single lamp that is lit.

[0157] Step S1043: Obtain a turn signal of the same color as the countdown light. Specifically, based on the illumination color of the turn signal, obtain a turn signal of the same color as the countdown light. Step S1044: Using the Hungarian Algorithm, match the countdown light with each turn signal of the same color based on the distance between the countdown light and the turn signals of the same color, and obtain a turn signal of the same color that matches the countdown light as the target turn signal. The Hungarian Algorithm is a conventional matching method, and the principle of the Hungarian Algorithm will not be specifically explained in this embodiment. Step S1045: Add the countdown digits of the countdown light to the single-lamp state of the target turn signal. In S1046: Determine the traffic direction state based on the single-lamp state of the target turn signal. Through the above embodiment, the single-lamp state of the countdown light can be accurately assigned to the single-lamp state of the turn signal. Thus, when determining the traffic direction state of the virtual light based on the turn signal, the single-lamp state of the countdown light can also be assigned to the traffic direction state of the virtual light.

[0158] For example, see Appendix Figure 7 As shown, the turn signal of the same color that matches the countdown light is a turn signal with a disc-shaped head. According to Table 1 of step S104 in the aforementioned embodiment, the traffic direction corresponding to the disc includes straight, left turn, and U-turn, and the traffic direction corresponding to the left turn arrow includes left turn and U-turn. Following the order of the head arrangement of "right turn arrow, disc, straight arrow, left turn arrow, and U-turn arrow", the color (green) and countdown number (33) corresponding to the disc are first assigned to straight, left turn, and U-turn. However, since the color corresponding to the left turn arrow is red, the color of left turn and U-turn is again assigned to red. In addition, since the single light of the left turn arrow is not matched with the countdown light, the countdown number in the single light state of this single light is unknown. Therefore, the countdown number of left turn and U-turn is assigned to negative 1, which represents unknown.

[0159] In some embodiments of step S1044 above, a target turn signal matching the countdown light can also be obtained through the following steps 11 to 12.

[0160] Step 11: Obtain all target turn signals that match the countdown light in the processed image. Track all target turn signals to obtain the tracking ID of each target turn signal. The processed image includes the target image and its previous historical images. Step 12: Perform a voting process on the tracking IDs of all target turn signals. Obtain the target turn signal indicated by the tracking ID that appears most frequently. This target turn signal is then selected as the final target turn signal that matches the countdown light in the target image. In the aforementioned implementation method, errors may occur when matching the countdown light and turn signals. This implementation method, even if matching errors occur, can select the accurate target turn signal by performing a voting process on all target turn signals, further improving the accuracy of the target turn signal selection.

[0161] The following describes embodiments of the traffic light data generation method provided in this application. In some embodiments of this application, it is possible to... Figure 8 The following steps S201 to S204 are shown to smooth the virtual lights of the traffic light intersection in the target image to obtain the final virtual lights, and then use the final virtual lights to annotate the target image.

[0162] Step S201: Track the traffic light intersections appearing in the processed images to obtain the tracking IDs of all traffic light intersections. The processed images include the target image and its preceding historical images. Step S202: Use the traffic light intersections appearing in the target image as the target intersections. Step S203: Obtain the virtual lights of the target intersections in the target image, and based on the tracking IDs of the target intersections, obtain the virtual lights of the target intersections in the historical images. Step S204: Smooth the virtual lights of the target intersections in the target image and the historical images to obtain the final virtual lights of the target intersections in the target image. The smoothing process mainly involves smoothing the colors of each traffic direction indicator in the virtual lights. Specifically, the virtual lights at the traffic light intersection in the target image are used as the current virtual lights, and the virtual lights at the traffic light intersection in historical images are used as historical virtual lights. Historical virtual lights from multiple consecutive historical images preceding the target image are obtained. For each traffic direction indicated by the current virtual light, the colors of the traffic directions in the current and historical virtual lights are obtained, and these colors are voted on, with the color appearing most frequently being used as the final color of the traffic direction indicated by the current virtual light. Based on the method described in steps S201 to S204, the accuracy of the virtual lights in the target image can be further improved.

[0163] The following is in conjunction with the appendix Figure 9 To be continued Figure 11 This application describes the data generation method for traffic lights provided in this application. First, please refer to the appendix... Figure 9 , Figure 9 The overall flow of a traffic light data generation method is illustrated exemplarily. For example... Figure 9 As shown, the virtual lights of the traffic light intersection and image annotation data can be generated through the following steps S301 to S303.

[0164] Step S301: Obtain the 3D position, tracking ID, and semantic information of a single lamp.

[0165] Step S302: Traffic light intersection generation and update. See appendix. Figure 10 In step S3021, the method described in steps S1031 to S1032 of the aforementioned method embodiment is used to obtain a first candidate single light and filter out single lights that are not facing the vehicle directly. In step S3022, the method described in step S103 of the aforementioned method embodiment is used to generate a traffic light intersection. In step S3023, the method described in steps S1033 to S1034 of the aforementioned method is used to remove (or discard) the traffic light intersection and the method described in step S103 of the aforementioned method is used to merge traffic light intersections that are close to each other.

[0166] Step S303: Obtain the virtual light. See Appendix Figure 11 In step S3031, the countdown lights and turn signals are matched using steps S1043 to S1046 in the aforementioned method embodiment to obtain the target turn signal; in step S3032, the virtual lights of the traffic light intersection are generated using the method described in steps S104 to S105 in the aforementioned method embodiment; in step S3033, the virtual lights are time-series fused (or smoothed) using the method described in steps S301 to S304 in the aforementioned method embodiment.

[0167] Another aspect of this application provides a method for sensing traffic lights.

[0168] See appendix Figure 12 , Figure 12 This is a schematic flowchart illustrating the main steps of a traffic light sensing method according to an embodiment of this application. Figure 12 As shown, the data generation method for traffic lights in this embodiment mainly includes the following steps S401 to S402.

[0169] Step S401: Acquire an image of the vehicle's driving environment, which may be an image captured by a camera on the vehicle. Step S402: Input the image into a perception model to obtain a virtual light. The virtual light consists of multiple traffic direction indicators and their states. The perception model is trained using image annotation data, which is obtained using the traffic light data generation method described in the aforementioned method embodiment. The virtual light consists of multiple traffic direction indicators and their states. By training the perception model with image annotation data containing the virtual light, the perception model can directly map or obtain the virtual light (including traffic direction indicators and their states) from the image captured by the camera, achieving end-to-end acquisition of traffic direction indicators and their states. Therefore, the perception model can quickly and accurately acquire the traffic direction indicators and their states in the vehicle's environment, ensuring driving safety.

[0170] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of this application, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent to the technical solutions described in this application and therefore will also fall within the protection scope of this application.

[0171] Those skilled in the art will understand that all or part of the processes in the method of the above-described embodiment can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0172] Another aspect of this application provides a computer-readable storage medium.

[0173] In one embodiment of a computer-readable storage medium according to this application, the computer-readable storage medium may be configured to store a program for performing the traffic light data generation method of the above-described method embodiments. This program may be loaded and run by a processor to implement the traffic light data generation method. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium may be a storage device comprising various electronic devices. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0174] Another aspect of this application provides an electronic device. In an embodiment of an electronic device according to this application, the electronic device may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the above-described embodiments of the traffic light data generation method. See Appendix Figure 13 , Figure 13 The image exemplarily illustrates a communication connection between memory 11 and processor 12 via a bus. The electronic device described in this application may be, but is not limited to, tablet computers, desktop computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, etc., and the embodiments of this application do not limit this to any particular type.

[0175] Another aspect of this application provides a smart device. In one embodiment of a smart device according to this application, the smart device may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in the above-described traffic light sensing method embodiment. The smart device may include driving equipment, intelligent vehicles, robots, and other devices. See Appendix Figure 14 , Figure 14 The example shows a memory 21 and a processor 22 connected via a bus communication connection.

[0176] In some embodiments of this application, the smart device may further include at least one sensor for sensing information. The sensor is communicatively connected to any type of processor mentioned in this application. Optionally, the smart device may further include an autonomous driving system for guiding the smart device to drive autonomously or assisting in driving. The processor communicates with the sensor and / or the autonomous driving system to perform the methods described in any of the above embodiments.

[0177] The technical solution of this application has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A method for generating traffic signal light data, characterized in that, The method includes: Acquire video clips of the vehicle's driving environment captured by a camera on the vehicle, wherein there is at least one traffic light in the environment; Tracking and detecting individual lights in each frame of the video segment yields the tracking ID and semantic information of each light in each frame. The semantic information includes the shape of the light head and the state of the individual light. Each frame is an image of each time frame within the video segment. Each frame of image is taken as the target image in sequence. Based on the position of the single light in the target image, all the single lights in the target image are clustered to obtain at least one cluster. The cluster is taken as the traffic light intersection appearing in the target image. The system acquires the illuminated single light in the traffic light intersection, acquires the traffic direction corresponding to the shape of the illuminated single light head according to a preset correspondence, and determines the state of the traffic direction according to the single light state of the illuminated single light. The preset correspondence is the correspondence between the shape of the light head and the traffic direction of the virtual light. Based on the traffic direction and its status, generate virtual lights for the traffic light intersection; The target image is labeled based on the virtual light to form image labeling data.

2. The method according to claim 1, characterized in that, The step of clustering all individual lights in the target image based on their positions includes: Based on the position of each individual light in the target image, a first candidate light is obtained from all the individual lights in the target image; wherein, the first candidate light is located in front of the vehicle and the yaw angle between it and the vehicle in the vehicle coordinate system is less than a preset angle threshold. Based on the position of the first candidate light, all first candidate lights are clustered.

3. The method according to claim 1, characterized in that, The method includes: When multiple traffic light intersections appear in the target image, the distance between each pair of traffic light intersections is obtained; if the distance between two traffic light intersections is less than a preset distance threshold, the two traffic light intersections are merged into one traffic light intersection.

4. The method according to claim 1, characterized in that, The method includes: Track the traffic light intersections appearing in the processed images to obtain the tracking IDs of all traffic light intersections. The processed images include the target image and its previous historical images. Obtain the tracking ID of the traffic light intersection appearing in the target image, and update the tracking status of the traffic light intersection based on the tracking ID.

5. The method according to claim 4, characterized in that, The method includes: Based on the tracking ID of the traffic light intersection, obtain the tracking status of the traffic light intersection; Obtain the duration during which the tracking status has not been updated; If the duration exceeds the third preset duration, the traffic light intersection is discarded.

6. The method according to claim 1, characterized in that, Determining the state of the traffic direction indication based on the state of the single illuminated light includes: When there are multiple illuminated single lights with the same lamp head shape at the traffic light intersection, a second candidate light is obtained from all the illuminated single lights with the same lamp head shape; wherein, the second candidate light has the smallest distance between itself and the vehicle in the vehicle coordinate system; The state of the traffic direction indication is determined based on the individual light status of the second candidate light.

7. The method according to claim 1, characterized in that, When the single light being illuminated includes both a turn signal and a countdown light, the single light status of the countdown light also includes a countdown digit. Determining the traffic direction indication status based on the single light status includes: Obtain a turn signal of the same color as the countdown light; The Hungarian matching method is used to match the countdown light with each turn signal of the same color based on the distance between the countdown light and each turn signal of the same color, and to obtain a turn signal of the same color that matches the countdown light as the target turn signal; Add the countdown numbers of the countdown light to the single-lamp state of the target turn signal; The state of the traffic direction indication is determined based on the single-lamp state of the target turn signal.

8. The method according to claim 7, characterized in that, The method further includes acquiring the target turn signal by means of: Obtain all target turn signals that match the countdown light in the processed image, track all target turn signals, and obtain the tracking ID of each target turn signal. The processed image includes the target image and its previous historical images. A voting process is performed on the tracking IDs of all the target turn signals to obtain the target turn signal indicated by the tracking ID that appears most frequently. This target turn signal is then selected as the final target turn signal in the target image that matches the countdown light.

9. The method according to claim 1, characterized in that, The method includes smoothing the virtual lights at the traffic light intersection in the target image by means of the following: Track the traffic light intersections appearing in the processed images to obtain the tracking IDs of all traffic light intersections. The processed images include the target image and its previous historical images. The traffic light intersection appearing in the target image is taken as the target intersection; The virtual lights of the target intersection in the target image are obtained, and the virtual lights of the target intersection in the historical image are obtained according to the tracking ID of the target intersection. The virtual lights of the target intersection in the target image and the historical image are smoothed to obtain the final virtual lights of the target intersection in the target image.

10. A method for sensing traffic signals, characterized in that, The method includes: Acquire images of the vehicle's driving environment; The image is input into a perception model to obtain a virtual light for the image. The virtual light consists of multiple traffic direction indicators and their states. in, The perception model is trained using image-annotated data, which is obtained using the traffic light data generation method described in any one of claims 1 to 9.

Images