Traffic light image labeling method and device, electronic equipment and storage medium

Abstract

This application discloses a traffic light image annotation method, apparatus, electronic device, and storage medium. The method includes: acquiring a road image of the current road segment and detecting the road image to obtain a traffic light image and vehicle detection results; determining the traffic light display status based on the traffic light image, and determining the vehicle driving status based on the traffic light image and vehicle detection results; determining traffic light annotation data based on the traffic light display status and vehicle driving status; and annotating the traffic light image using the traffic light annotation data to obtain a traffic light image annotation result. This application, based on determining the traffic light display status through image detection, further combines the vehicle driving status determined by image detection to determine the traffic light color, thereby obtaining more accurate traffic light annotation data. This enables self-annotation of traffic light images, significantly reducing manual annotation costs and improving the annotation efficiency of traffic light images.

Description

Technical Field

[0001] This application relates to the field of traffic light recognition technology, and in particular to a traffic light image annotation method and apparatus, electronic device and storage medium. Background Technology

[0002] Currently, some traffic light equipment is equipped with a transmitting device that can send traffic light signals to roadside or vehicle-side equipment. More often, the color of the traffic light can only be obtained through vehicle-side image detection and recognition. However, when the traffic light equipment malfunctions, the traffic light color recognized by the vehicle-side is incorrect. If vehicles drive according to the incorrectly recognized traffic light, it can easily cause traffic accidents and lead to traffic congestion.

[0003] Currently, vehicle-side traffic light recognition relies on deep learning models, which in turn depend on large amounts of labeled data, requiring significant human and material resources. Furthermore, the traffic light images acquired by the vehicle are limited by its own positioning capabilities, potentially leading to recognition errors, or the angle of the captured image may cause incorrect detection. Summary of the Invention

[0004] This application provides a traffic light image annotation method and apparatus, electronic device and storage medium to realize self-annotation of traffic light image data and improve annotation efficiency and accuracy.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a method for annotating traffic light images, wherein the method includes:

[0007] Acquire a road image of the current road segment and perform detection on the road image to obtain traffic light images and vehicle detection results;

[0008] The traffic light display status is determined based on the traffic light image, and the vehicle driving status is determined based on the traffic light image and the vehicle detection results;

[0009] Based on the traffic light display and vehicle movement, determine the traffic light labeling data;

[0010] The traffic light image is annotated using the traffic light annotation data to obtain the traffic light image annotation result.

[0011] Optionally, acquiring the road image of the current road segment and performing detection on the road image to obtain traffic light images and vehicle detection results includes:

[0012] Obtain the high-precision map corresponding to the current road segment, and the correspondence between the high-precision map and the camera image;

[0013] Based on the high-precision map corresponding to the current road segment and the correspondence between the high-precision map and the camera image, the traffic light image is segmented from the road image.

[0014] Optionally, the traffic light image includes multiple frames of traffic light images, and determining the traffic light display status based on the traffic light image includes:

[0015] Perform pixel difference summation on any two adjacent traffic light images in a multi-frame traffic light image dataset;

[0016] Based on the pixel difference and processing results of two adjacent traffic light images, the color change situation and the display duration of the color are determined as the traffic light display situation.

[0017] Optionally, determining the vehicle driving status based on the traffic light image and the vehicle detection result includes:

[0018] Obtain a high-precision map corresponding to the current road segment, wherein the high-precision map contains the correspondence between lanes and traffic lights;

[0019] The lane corresponding to the traffic light in the traffic light image is determined based on the correspondence between the lane and the traffic light;

[0020] The position of the target vehicle corresponding to the lane is determined based on the lane corresponding to the traffic light in the traffic light image and the vehicle detection result.

[0021] The vehicle's driving status is determined based on the target vehicle's location and the stop line position of the current road segment.

[0022] Optionally, determining the traffic light label data based on the traffic light display and vehicle movement includes:

[0023] Based on the traffic light display and the vehicle driving situation, determine whether the traffic light image meets the preset traffic light labeling conditions;

[0024] If the traffic light image meets the preset traffic light labeling conditions, the traffic light labeling data is determined based on the traffic light display and the vehicle driving conditions.

[0025] Optionally, the traffic light image includes a first traffic light image and a second traffic light image, wherein the second traffic light image is obtained before the first traffic light image. Determining whether the traffic light image meets preset traffic light labeling conditions based on the traffic light display and vehicle traffic conditions includes:

[0026] Based on the traffic light display and vehicle driving conditions corresponding to the first traffic light image, determine whether the first traffic light image meets the first preset traffic light labeling conditions;

[0027] If the first traffic light image meets the first preset traffic light labeling conditions, determine the light color detection result corresponding to the first traffic light image;

[0028] Based on the light color detection result corresponding to the first traffic light image and the traffic light display status corresponding to the second traffic light image, the light color detection result corresponding to the second traffic light image is determined;

[0029] Based on the light color detection results corresponding to the first traffic light image and the second traffic light image, determine whether the first traffic light image and the second traffic light image meet the second preset traffic light labeling conditions.

[0030] Optionally, determining whether the first traffic light image meets the first preset traffic light labeling conditions based on the traffic light display and vehicle driving conditions corresponding to the first traffic light image includes:

[0031] Based on the traffic light display situation corresponding to the first traffic light image, determine whether the color of the traffic light is the same for multiple consecutive frames, and based on the vehicle driving situation, determine whether the position of the target vehicle in the corresponding multiple frames is within the stop line position of the current road segment.

[0032] If so, after determining that the traffic light color has changed according to the traffic light display situation corresponding to the first traffic light image, determine whether the position of the target vehicle in the corresponding multi-frame is outside the stop line position according to the vehicle driving situation;

[0033] If so, then the first traffic light image is determined to meet the first preset traffic light labeling conditions.

[0034] Optionally, determining whether the first traffic light image and the second traffic light image meet the second preset traffic light labeling conditions based on the light color detection results corresponding to the first traffic light image and the second traffic light image includes:

[0035] The display duration of the traffic light colors is determined based on the light color detection results corresponding to the first traffic light image and the second traffic light image. The display duration of the traffic light colors includes the display duration of red, green and yellow lights.

[0036] If the display duration of the red light and the green light are both longer than the display duration of the yellow light, then the first traffic light image and the second traffic light image are determined to meet the second preset traffic light labeling conditions.

[0037] Secondly, embodiments of this application also provide a traffic light image annotation device, wherein the device includes:

[0038] The acquisition unit is used to acquire road images of the current road segment and detect the road images to obtain traffic light images and vehicle detection results;

[0039] The first determining unit is used to determine the traffic light display status based on the traffic light image, and to determine the vehicle driving status based on the traffic light image and the vehicle detection result;

[0040] The second determining unit is used to determine traffic light labeling data based on the traffic light display and the vehicle driving situation;

[0041] The annotation unit is used to annotate the traffic light image using the traffic light annotation data to obtain the traffic light image annotation result.

[0042] Thirdly, embodiments of this application also provide an electronic device, including:

[0043] Processor; and

[0044] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0045] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.

[0046] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: The traffic light image annotation method of this application embodiment first acquires a road image of the current road segment and detects the road image to obtain a traffic light image and vehicle detection results; then, it determines the traffic light display status based on the traffic light image and the vehicle driving status based on the traffic light image and vehicle detection results; subsequently, it determines traffic light annotation data based on the traffic light display status and vehicle driving status; finally, it annotates the traffic light image using the traffic light annotation data to obtain the traffic light image annotation result. The traffic light image annotation method of this application embodiment, based on determining the traffic light display status through image detection, further combines the detected vehicle driving status in the image to determine the traffic light color, thereby obtaining more accurate traffic light annotation data and achieving self-annotation of traffic light images, reducing a significant amount of manual annotation costs and improving the annotation efficiency of traffic light images. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0048] Figure 1 This is a flowchart illustrating a traffic light image annotation method according to an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of the structure of a traffic light image annotation device according to an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0053] This application provides a method for annotating traffic light images, such as... Figure 1 The diagram shows a flowchart of a traffic light image annotation method according to an embodiment of this application. The method includes at least the following steps S110 to S140:

[0054] Step S110: Obtain the road image of the current road segment and perform detection on the road image to obtain traffic light images and vehicle detection results.

[0055] The traffic light image annotation method of this application embodiment can be performed by the roadside. The roadside can detect traffic lights and send the results to the vehicle side, which is more accurate because the position and shape of traffic lights in the image are relatively fixed for the roadside camera, and training data can be added to improve the detection accuracy. However, the detection and recognition of traffic lights by the roadside also depends on a large amount of annotation data. Therefore, the traffic light image annotation method of this application embodiment is used to realize the self-annotation of traffic light image data.

[0056] When annotating traffic light image data, it is necessary to first acquire road images of the current road segment captured by roadside cameras, and then use a preset image detection algorithm to detect the road image. The detection here mainly includes the detection of traffic light areas and the detection of vehicle targets contained in the image. Traffic light area detection refers to segmenting the local image of the area where the traffic lights are located from the current road image. Vehicle detection can be achieved using existing target detection networks such as YOLO V5, thereby detecting the location of vehicle targets from the image.

[0057] Step S120: Determine the traffic light display status based on the traffic light image, and determine the vehicle driving status based on the traffic light image and the vehicle detection results.

[0058] Based on continuously detected traffic light images, the traffic light display status can be determined further by analyzing changes in image pixels, such as the color transitions and display duration. Furthermore, since different traffic light colors and their transitions typically have a significant impact on vehicle movement, the traffic light images and corresponding vehicle detection results can be used to further determine vehicle movement, thus assisting in traffic light color recognition.

[0059] Step S130: Determine traffic light labeling data based on the traffic light display and vehicle driving conditions.

[0060] Traffic light displays can reflect the color changes of traffic lights over a period of time and the duration of the same color display. If the traffic lights are displaying normally, the red and green lights can be roughly distinguished from the yellow lights based on the traffic light change rules. However, when the traffic lights malfunction, such as the red light malfunctioning and flashing, the detection results reflected by the traffic light displays will also be incorrect, and the color cannot be accurately distinguished.

[0061] Based on this, the embodiments of this application further incorporate vehicle driving conditions to assist in detecting the color and changes of traffic lights. Since most human-driven vehicles can accurately judge and distinguish traffic light malfunctions, this avoids the problem that relying solely on traffic light images is unsuitable for detecting scenarios such as flashing traffic lights. Of course, even in scenarios where traffic lights are displaying normally, combining vehicle driving conditions can verify the detection results of the traffic light images, thereby further improving the accuracy of traffic light color recognition.

[0062] Step S140: Use the traffic light annotation data to annotate the traffic light image to obtain the traffic light image annotation result.

[0063] The traffic light annotation data mentioned above can be regarded as the accurate annotation data of the currently detected traffic light image. Specifically, it can include information such as the location of the traffic light and the corresponding light color. Based on the annotation data, the traffic light image can be self-annotated, so as to serve as a sample for subsequent training of the traffic light color recognition model at the roadside or vehicle side.

[0064] The traffic light image annotation method of this application, based on determining the traffic light display through image detection, further combines the vehicle driving situation detected in the image to determine the color of the traffic light, thereby obtaining more accurate traffic light annotation data, and thus realizing the self-annotation of traffic light images, reducing a lot of manual annotation costs and improving the annotation efficiency of traffic light images.

[0065] In some embodiments of this application, the step of acquiring a road image of the current road segment and detecting the road image to obtain a traffic light image and vehicle detection results includes: acquiring a high-precision map corresponding to the current road segment, and the correspondence between the high-precision map and the camera image; and segmenting the traffic light image from the road image based on the high-precision map corresponding to the current road segment and the correspondence between the high-precision map and the camera image.

[0066] Since the traffic light image annotation method in this application embodiment is performed by the roadside, and the position of the roadside camera is usually relatively fixed, it will not change much unless manually adjusted. Therefore, the position of the traffic light in the road image captured by the roadside camera is also basically fixed.

[0067] Based on this, the embodiments of this application can first perform joint calibration of the high-precision map and the roadside camera to obtain the correspondence between the camera image and the high-precision map. Based on the correspondence between the camera image and the high-precision map, the specific area where the traffic light is located can be cropped from the corresponding position of the camera image according to the traffic light position of the current road segment provided in the high-precision map, and used as the traffic light image.

[0068] In some embodiments of this application, the traffic light image includes multiple frames of traffic light images, and determining the traffic light display status based on the traffic light image includes: performing pixel difference sum processing on any two adjacent frames of traffic light images in the multiple frames of traffic light images; and determining the light color change situation and the light color display duration based on the pixel difference sum processing result of the two adjacent frames of traffic light images, as the traffic light display status.

[0069] The detection of traffic light images at the roadside is a real-time and continuous process. Based on the detected multiple frames of traffic light images, pixel difference sum processing can be performed on any two adjacent frames of traffic light images. That is, each pixel value in the current frame of traffic light image is subtracted from each pixel value in the previous frame of traffic light image, and finally the differences of all the obtained pixel values ​​are summed as the pixel difference sum processing result of the two adjacent frames of traffic light images.

[0070] The purpose of pixel difference sum processing is to determine the traffic light transition, that is, to detect whether the traffic light is changing color in the current frame or is in a normal display of a certain color. If the current frame is experiencing a color change such as yellow to red, green to yellow, or red to green, the color change will inevitably lead to a large difference in pixel values. Therefore, the pixel difference sum of the current frame relative to the previous frame should be large. An adjustable threshold can be set to determine this. Conversely, if the current frame and the previous frame are continuously displaying yellow, green, or red lights, the pixel values ​​between adjacent frames remain essentially unchanged. Therefore, the pixel difference sum between adjacent frames should be very small, close to or equal to 0.

[0071] Therefore, by using the above pixel difference summation processing method, the traffic light transition can be determined during the continuous detection of traffic light images, that is, in which frame the transition occurs. At the same time, the duration of continuous display of a single light color can also be determined based on the time interval between two adjacent transitions.

[0072] In some embodiments of this application, the traffic light display status includes the duration of the light color display. After determining the traffic light display status based on the traffic light image, the method further includes: comparing the duration of the light color display with a preset duration threshold; if the duration of the light color display exceeds the preset duration threshold, then determining that the traffic light has malfunctioned and sending a reminder message to the fault platform.

[0073] Based on the foregoing embodiments, the continuous display duration of a single light color can be determined. Under normal circumstances, the continuous display duration of the same light color should be within a reasonable range. However, if the traffic light malfunctions, the display may be abnormal. Therefore, this embodiment can compare the light color display duration with a pre-set preset duration threshold. If the threshold is exceeded, the traffic light display is considered to be malfunctioning, and then before and after video evidence can be sent to the fault platform for timely fault handling. The size of the aforementioned preset duration threshold can be flexibly set according to actual needs and is not specifically limited here.

[0074] In some embodiments of this application, determining the vehicle driving status based on the traffic light image and the vehicle detection result includes: acquiring a high-precision map corresponding to the current road segment, the high-precision map containing the correspondence between lanes and traffic lights; determining the lane corresponding to the traffic light in the traffic light image based on the correspondence between lanes and traffic lights; determining the position of the target vehicle corresponding to the lane based on the lane corresponding to the traffic light in the traffic light image and the vehicle detection result; and determining the vehicle driving status based on the position of the target vehicle and the stop line position of the current road segment.

[0075] In real-world road scenarios, not every intersection has only one traffic light; many intersections have different traffic lights corresponding to different lanes. High-definition maps often provide the correspondence between traffic lights and lanes at each intersection. Therefore, this embodiment of the application can determine the specific lane corresponding to each traffic light in the traffic light image based on the correspondence between lanes and traffic lights in the high-definition map, as well as the correspondence between lanes and roadside camera images in the high-definition map. In other words, it can determine which area in the image corresponds to which lane, and which traffic light corresponds to that lane. Therefore, the traffic light detection in this embodiment of the application is lane-level detection, and the lane information obtained here is also the basis for subsequently determining the vehicle traffic situation in the lane corresponding to the traffic light.

[0076] In some embodiments of this application, determining traffic light labeling data based on the traffic light display and vehicle driving conditions includes: determining whether the traffic light image meets preset traffic light labeling conditions based on the traffic light display and vehicle driving conditions; and determining the traffic light labeling data based on the traffic light display and vehicle driving conditions if the traffic light image meets the preset traffic light labeling conditions.

[0077] To ensure the accuracy of traffic light image annotation data, this application embodiment needs to meet certain preset traffic light annotation conditions when determining the annotation data based on the traffic light display and vehicle traffic conditions. This is because in some special cases, it is still impossible to accurately determine the color of the traffic light based solely on the traffic light display and vehicle traffic conditions. For example, when the traffic light display changes from green to yellow or from yellow to red, some vehicles may still be passing through the intersection. Therefore, it is difficult to accurately determine the specific color of the traffic light before and after the change based on the vehicle traffic conditions.

[0078] Based on this, the embodiments of this application can first determine whether the current traffic light image meets the preset traffic light labeling conditions. If it does, the traffic light labeling data can be determined based on the traffic light display and vehicle driving conditions. For image data that does not meet the preset traffic light labeling conditions, it can be obtained by reverse reasoning based on the currently labeled data and the traffic light change pattern.

[0079] In some embodiments of this application, the traffic light image includes a first traffic light image and a second traffic light image, wherein the second traffic light image is obtained before the first traffic light image. Determining whether the traffic light image meets preset traffic light labeling conditions based on the traffic light display and vehicle driving conditions includes: determining whether the first traffic light image meets a first preset traffic light labeling condition based on the traffic light display and vehicle driving conditions corresponding to the first traffic light image; if the first traffic light image meets the first preset traffic light labeling condition, determining the light color detection result corresponding to the first traffic light image; determining the light color detection result corresponding to the second traffic light image based on the light color detection result corresponding to the first traffic light image and the traffic light display corresponding to the second traffic light image; and determining whether the first traffic light image and the second traffic light image meet a second preset traffic light labeling condition based on the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image.

[0080] Since the detection of traffic light images at the roadside is performed in real time, for the latest detected traffic light images, namely the first traffic light detection images mentioned above, it can be first determined whether the first traffic light detection images meet the first preset traffic light labeling conditions. The first preset traffic light labeling conditions can be understood as the situation that can accurately distinguish the specific light color of a part of the traffic light images based on the vehicle driving situation and the traffic light display situation. For example, when the traffic light display situation changes from red to green, the vehicle driving situation at the intersection often changes significantly, so this situation can be accurately identified based on the corresponding vehicle driving trend.

[0081] Therefore, if the first traffic light image meets the first preset traffic light labeling conditions, the light color detection result corresponding to the first traffic light image can be determined. For example, the light color corresponding to the traffic light image between the current light color change and the previous light color change should be red, and the light color corresponding to the traffic light image after the current light color change should be green. Since the changes in traffic light colors follow a certain pattern, such as usually changing sequentially in the order of red-green-yellow-red..., after determining the light color of the first traffic light image, the light color detection result corresponding to the second traffic light image can be deduced backward based on the light color detection result corresponding to the first traffic light image and the pattern of traffic light color changes. The second traffic light image can refer to the image that has not yet been labeled since the last traffic light image that met the first preset traffic light labeling conditions.

[0082] After obtaining the light color detection results corresponding to the first and second traffic light images, it is further possible to determine whether the first and second traffic light images meet the second preset traffic light labeling conditions. The second preset traffic light labeling conditions are mainly used to determine whether the display duration of different traffic light colors conforms to the normal display pattern. When both of the above conditions are met, it can be considered that the light color detection results corresponding to the first and second traffic light images obtained this time are accurate and can be used as traffic light labeling data.

[0083] In some embodiments of this application, determining whether the first traffic light image meets the first preset traffic light labeling condition based on the traffic light display and vehicle driving conditions corresponding to the first traffic light image includes: determining whether the color of the traffic light is the same for multiple consecutive frames based on the traffic light display corresponding to the first traffic light image, and determining whether the corresponding multi-frame target vehicle positions are within the stop line position of the current road segment based on the vehicle driving conditions; if so, after determining that the color of the traffic light has changed based on the traffic light display corresponding to the first traffic light image, determining whether the corresponding multi-frame target vehicle positions are outside the stop line position based on the vehicle driving conditions; if so, determining that the first traffic light image meets the first preset traffic light labeling condition.

[0084] The first traffic light image in this embodiment can be divided into a first traffic light image before the change and a first traffic light image after the change based on the color transition. The color of the first traffic light image before the change remains the same for multiple frames. At this time, it is determined whether the vehicle position in the corresponding lane is within the stop line and has not moved out. If so, after one color change, it is determined whether the vehicle position in the first traffic light image after the change has moved out of the stop line. If so, it means that the color of the traffic light image corresponding to the first traffic light image before the change should be red, and the color of the traffic light image corresponding to the first traffic light image after the change should be green. Continuing to trace back, the color of the light before the change to red should be yellow. If the color change conflicts with the vehicle's driving situation, it is considered that a traffic light malfunction may have occurred, and the preceding and following video evidence is sent to the fault platform for repair as soon as possible.

[0085] The principle of the above embodiment is that during the process of the red light changing to the green light, the driving direction of the vehicles in the corresponding lane will change significantly, that is, the behavior of stopping before the stop line at the intersection will change to the behavior of moving out of the stop line. Therefore, by observing the change in the driving situation of the vehicles, the color of the traffic light before and after this change can be accurately distinguished. Although it cannot be ruled out that some vehicles violate traffic rules and run red lights, based on the accumulation of a large amount of data, the driving situation of most vehicles meets the above requirements.

[0086] In some embodiments of this application, determining whether the first traffic light image and the second traffic light image meet the second preset traffic light labeling conditions based on the light color detection results corresponding to the first traffic light image and the second traffic light image includes: determining the light color display duration of the traffic light based on the light color detection results corresponding to the first traffic light image and the second traffic light image, wherein the light color display duration of the traffic light includes the light color display duration of red, green and yellow lights; if the light color display duration of red light and green light is greater than the light color display duration of yellow light, then it is determined that the first traffic light image and the second traffic light image meet the second preset traffic light labeling conditions.

[0087] To further improve the accuracy of traffic light detection, this embodiment can further determine whether the color detection results of the first and second traffic light images obtained in the above embodiments meet the second preset traffic light labeling conditions. Under normal circumstances, the display duration of a yellow light is generally only a few seconds, while the display duration of red and green lights is longer than that of a yellow light. Therefore, based on the color detection results of the first and second traffic light images, the continuous display duration of each light color can be statistically analyzed. If the statistically obtained display durations of red and green lights are both longer than the display duration of yellow lights, then the color detection results can be considered accurate; otherwise, it is considered that the traffic light may be malfunctioning, and before-and-after video evidence is sent to the fault platform for prompt repair.

[0088] This application also provides a traffic light image annotation device 200, such as... Figure 2 The diagram shows a schematic representation of a traffic light image annotation device according to an embodiment of this application. The device 200 includes: an acquisition unit 210, a first determination unit 220, a second determination unit 230, and an annotation unit 240, wherein:

[0089] The acquisition unit 210 is used to acquire a road image of the current road segment and detect the road image to obtain a traffic light image and vehicle detection results;

[0090] The first determining unit 220 is used to determine the traffic light display status based on the traffic light image, and to determine the vehicle driving status based on the traffic light image and the vehicle detection result;

[0091] The second determining unit 230 is used to determine traffic light labeling data based on the traffic light display and the vehicle driving situation;

[0092] The annotation unit 240 is used to annotate the traffic light image using the traffic light annotation data to obtain the traffic light image annotation result.

[0093] In some embodiments of this application, the acquisition unit 210 is specifically used to: acquire a high-precision map corresponding to the current road segment, and the correspondence between the high-precision map and the camera image; and segment the traffic light image from the road image based on the high-precision map corresponding to the current road segment and the correspondence between the high-precision map and the camera image.

[0094] In some embodiments of this application, the traffic light image includes multiple frames of traffic light images, and the first determining unit 220 is specifically used to: perform pixel difference sum processing on any two adjacent frames of traffic light images in the multiple frames of traffic light images; and determine the light color change situation and the light color display duration based on the pixel difference sum processing result of the two adjacent frames of traffic light images, as the traffic light display situation.

[0095] In some embodiments of this application, the first determining unit 220 is specifically used to: obtain a high-precision map corresponding to the current road segment, the high-precision map containing the correspondence between lanes and traffic lights; determine the lane corresponding to the traffic light in the traffic light image according to the correspondence between lanes and traffic lights; determine the position of the target vehicle corresponding to the lane according to the lane corresponding to the traffic light in the traffic light image and the vehicle detection result; and determine the vehicle driving situation according to the position of the target vehicle and the stop line position of the current road segment.

[0096] In some embodiments of this application, the second determining unit 230 is specifically used to: determine whether the traffic light image meets the preset traffic light labeling conditions based on the traffic light display and the vehicle driving conditions; and determine the traffic light labeling data based on the traffic light display and the vehicle driving conditions if the traffic light image meets the preset traffic light labeling conditions.

[0097] In some embodiments of this application, the traffic light image includes a first traffic light image and a second traffic light image, wherein the second traffic light image is obtained before the first traffic light image. The second determining unit 230 is specifically configured to: determine whether the first traffic light image meets a first preset traffic light labeling condition based on the traffic light display and vehicle driving conditions corresponding to the first traffic light image; determine the light color detection result corresponding to the first traffic light image if the first traffic light image meets the first preset traffic light labeling condition; determine the light color detection result corresponding to the second traffic light image based on the light color detection result corresponding to the first traffic light image and the traffic light display corresponding to the second traffic light image; and determine whether the first traffic light image and the second traffic light image meet a second preset traffic light labeling condition based on the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image.

[0098] In some embodiments of this application, the second determining unit 230 is specifically used to: determine whether the color of the traffic light is the same for multiple consecutive frames based on the traffic light display situation corresponding to the first traffic light image, and determine whether the position of the target vehicle in the corresponding multiple frames is within the stop line position of the current road segment based on the vehicle driving situation; if so, after determining that the color of the traffic light has changed based on the traffic light display situation corresponding to the first traffic light image, determine whether the position of the target vehicle in the corresponding multiple frames is outside the stop line position based on the vehicle driving situation; if so, determine that the first traffic light image satisfies the first preset traffic light marking condition.

[0099] In some embodiments of this application, the second determining unit 230 is specifically used to: determine the display duration of the traffic light color based on the light color detection result corresponding to the first traffic light image and the light color detection result corresponding to the second traffic light image, wherein the display duration of the traffic light color includes the display duration of the red light, green light and yellow light; if the display duration of the red light and the display duration of the green light are both greater than the display duration of the yellow light, then it is determined that the first traffic light image and the second traffic light image meet the second preset traffic light labeling condition.

[0100] It is understood that the traffic light image annotation device described above can realize all the steps of the traffic light image annotation method provided in the foregoing embodiments. The relevant explanations of the traffic light image annotation method are applicable to the traffic light image annotation device, and will not be repeated here.

[0101] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0102] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0103] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0104] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a traffic light image annotation device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0105] Acquire a road image of the current road segment and perform detection on the road image to obtain traffic light images and vehicle detection results;

[0106] The traffic light display status is determined based on the traffic light image, and the vehicle driving status is determined based on the traffic light image and the vehicle detection results;

[0107] Based on the traffic light display and vehicle movement, determine the traffic light labeling data;

[0108] The traffic light image is annotated using the traffic light annotation data to obtain the traffic light image annotation result.

[0109] The above is as stated in this application. Figure 1 The method executed by the traffic light image annotation device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0110] The electronic device can also perform Figure 1 The method for implementing a traffic light image annotation device, and realizing the traffic light image annotation device in... Figure 1 The functions of the embodiments shown are not described in detail here.

[0111] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the traffic light image annotation device in the illustrated embodiment is specifically used to perform:

[0112] Acquire a road image of the current road segment and perform detection on the road image to obtain traffic light images and vehicle detection results;

[0113] The traffic light display status is determined based on the traffic light image, and the vehicle driving status is determined based on the traffic light image and the vehicle detection results;

[0114] Based on the traffic light display and vehicle movement, determine the traffic light labeling data;

[0115] The traffic light image is annotated using the traffic light annotation data to obtain the traffic light image annotation result.

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0121] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0122] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0123] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0124] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A traffic light image labeling method, wherein, The method includes: Acquire a road image of the current road segment and perform detection on the road image to obtain traffic light images and vehicle detection results; The traffic light display status is determined based on the traffic light image, and the vehicle driving status is determined based on the traffic light image and the vehicle detection results; Based on the traffic light display and vehicle movement, determine the traffic light labeling data; The traffic light image is annotated using the traffic light annotation data to obtain the traffic light image annotation result; The step of determining the traffic light label data based on the traffic light display and vehicle movement includes: Based on the traffic light display and the vehicle driving situation, determine whether the traffic light image meets the preset traffic light labeling conditions; If the traffic light image meets the preset traffic light labeling conditions, the traffic light labeling data is determined based on the traffic light display and the vehicle driving conditions. The traffic light image includes a first traffic light image and a second traffic light image, wherein the second traffic light image is obtained before the first traffic light image. Determining whether the traffic light image meets preset traffic light labeling conditions based on the traffic light display and vehicle traffic conditions includes: Based on the traffic light display and vehicle driving conditions corresponding to the first traffic light image, determine whether the first traffic light image meets the first preset traffic light labeling conditions; If the first traffic light image meets the first preset traffic light labeling conditions, determine the light color detection result corresponding to the first traffic light image; Based on the light color detection result corresponding to the first traffic light image and the traffic light display status corresponding to the second traffic light image, the light color detection result corresponding to the second traffic light image is determined; Based on the light color detection results corresponding to the first traffic light image and the second traffic light image, determine whether the first traffic light image and the second traffic light image meet the second preset traffic light labeling conditions.

2. The method of claim 1, wherein, The process of acquiring a road image of the current road segment and detecting the road image to obtain traffic light images and vehicle detection results includes: Obtain the high-precision map corresponding to the current road segment, and the correspondence between the high-precision map and the camera image; Based on the high-precision map corresponding to the current road segment and the correspondence between the high-precision map and the camera image, the traffic light image is segmented from the road image.

3. The method as described in claim 1, wherein, The traffic light image includes multiple frames of traffic light images, and determining the traffic light display status based on the traffic light image includes: Perform pixel difference summation on any two adjacent traffic light images in a multi-frame traffic light image dataset; Based on the pixel difference and processing results of two adjacent traffic light images, the color change situation and the display duration of the color are determined as the traffic light display situation.

4. The method as described in claim 1, wherein, Determining vehicle driving status based on the traffic light image and the vehicle detection result includes: Obtain a high-precision map corresponding to the current road segment, wherein the high-precision map contains the correspondence between lanes and traffic lights; The lane corresponding to the traffic light in the traffic light image is determined based on the correspondence between the lane and the traffic light; The position of the target vehicle corresponding to the lane is determined based on the lane corresponding to the traffic light in the traffic light image and the vehicle detection result. The vehicle's driving status is determined based on the target vehicle's location and the stop line position of the current road segment.

5. The method as described in claim 1, wherein, The step of determining whether the first traffic light image meets the first preset traffic light labeling conditions based on the traffic light display and vehicle driving conditions corresponding to the first traffic light image includes: Based on the traffic light display situation corresponding to the first traffic light image, determine whether the color of the traffic light is the same for multiple consecutive frames, and based on the vehicle driving situation, determine whether the position of the target vehicle in the corresponding multiple frames is within the stop line position of the current road segment. If so, after determining that the traffic light color has changed according to the traffic light display situation corresponding to the first traffic light image, determine whether the position of the target vehicle in the corresponding multi-frame is outside the stop line position according to the vehicle driving situation; If so, then the first traffic light image is determined to meet the first preset traffic light labeling conditions.

6. The method of claim 1, wherein, The step of determining whether the first traffic light image and the second traffic light image meet the second preset traffic light labeling conditions based on the light color detection results corresponding to the first traffic light image and the second traffic light image includes: The display duration of the traffic light colors is determined based on the light color detection results corresponding to the first traffic light image and the second traffic light image. The display duration of the traffic light colors includes the display duration of red, green and yellow lights. If the display duration of the red light and the green light are both longer than the display duration of the yellow light, then the first traffic light image and the second traffic light image are determined to meet the second preset traffic light labeling conditions.

7. A traffic light image annotation device, wherein, The device includes: The acquisition unit is used to acquire road images of the current road segment and detect the road images to obtain traffic light images and vehicle detection results; The first determining unit is used to determine the traffic light display status based on the traffic light image, and to determine the vehicle driving status based on the traffic light image and the vehicle detection result; The second determining unit is used to determine traffic light labeling data based on the traffic light display and the vehicle driving situation; The annotation unit is used to annotate the traffic light image using the traffic light annotation data to obtain the traffic light image annotation result; The second determining unit is specifically used for: Based on the traffic light display and the vehicle driving situation, determine whether the traffic light image meets the preset traffic light labeling conditions; If the traffic light image meets the preset traffic light labeling conditions, the traffic light labeling data is determined based on the traffic light display and the vehicle driving conditions. The traffic light image includes a first traffic light image and a second traffic light image, wherein the second traffic light image is obtained before the first traffic light image, and the second determining unit is specifically used for: Based on the traffic light display and vehicle driving conditions corresponding to the first traffic light image, determine whether the first traffic light image meets the first preset traffic light labeling conditions; If the first traffic light image meets the first preset traffic light labeling conditions, determine the light color detection result corresponding to the first traffic light image; Based on the light color detection result corresponding to the first traffic light image and the traffic light display status corresponding to the second traffic light image, the light color detection result corresponding to the second traffic light image is determined; Based on the light color detection results corresponding to the first traffic light image and the second traffic light image, determine whether the first traffic light image and the second traffic light image meet the second preset traffic light labeling conditions.

8. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 6.

Images