Traffic light marking method, device, electronic device and roadside computing device

Abstract

The present disclosure provides a method, device, electronic device and roadside computing device for annotating a traffic light, which relates to the field of artificial intelligence, and in particular to the field of intelligent transportation. The specific implementation scheme is: obtaining a first annotation result obtained by annotating an image to be annotated, wherein the image to be annotated at least includes a traffic light area, the traffic light area is composed of multiple sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of multiple lamp heads, and constructing a data structure diagram according to the first annotation result, adjusting the first annotation result based on the data structure diagram to obtain a second annotation result, annotating a target area in the traffic light area according to the annotation result of the historical annotation image and the second annotation result to obtain a target annotation result, wherein the target area is an area where lamp heads with display brightness lower than a preset brightness are located among multiple lamp heads. The present disclosure at least solves the technical problem of low efficiency of traffic light annotation in the prior art.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to the field of intelligent transportation. Specifically, it relates to a method, device, electronic device and roadside computing device for marking traffic lights. Background Art

[0002] When a vehicle is driving intelligently, accurately identifying traffic lights plays an important role in the safe driving of the vehicle.

[0003] In the prior art, when performing traffic light color recognition, it is first necessary to perform target detection on the environment to obtain the image to be labeled, then detect the traffic light area in the image to be labeled, and then obtain the image of the traffic light according to the position of the detection frame and perform light color recognition.

[0004] However, when using nighttime data for neural network model training, annotators are required to pre-process the labeled data. If black lights are missing in nighttime traffic light images, the neural network model cannot detect the unlabeled black lights on its own, and manual review is required to check whether there are any missing label errors in the annotation files. Summary of the invention

[0005] The present disclosure provides a traffic light labeling method, device, electronic device and roadside computing device to at least solve the technical problem of low efficiency of traffic light labeling in the prior art.

[0006] According to one aspect of the present disclosure, a method for annotating a traffic light is provided, comprising: obtaining a first annotation result obtained by annotating an image to be annotated, wherein the image to be annotated at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads, and constructing a data structure diagram according to the first annotation result, thereby adjusting the first annotation result based on the data structure diagram to obtain a second annotation result, and then annotating a target area in the traffic light area according to the annotation result of the historical annotation image and the second annotation result to obtain a target annotation result, wherein the target area is an area where lamp heads whose display brightness is lower than a preset brightness are located among the plurality of lamp heads.

[0007] Furthermore, the traffic light labeling method also includes: obtaining multiple labeling boxes for labeling the image to be labeled, and filtering the multiple labeling boxes to obtain multiple first labeling boxes, thereby determining position information and identification information of the multiple first labeling boxes to obtain a first labeling result.

[0008] Furthermore, the traffic light annotation method further includes: determining the distance information between the plurality of first annotation boxes according to the position information in the first annotation result, and determining the edge in the data structure graph according to the distance information. Then, obtaining the identification information of the plurality of first annotation boxes from the first annotation result, and obtaining the vertex in the data structure graph, wherein the data structure graph is composed of the edge and the vertex.

[0009] Furthermore, the traffic light labeling method also includes: determining at least one connected domain corresponding to multiple first labeling boxes according to the value of the edge in the data structure diagram, and determining at least one second labeling box from the multiple first labeling boxes according to the at least one connected domain, wherein at least one second labeling box belongs to the same connected domain, thereby determining a target labeling box corresponding to at least one connected domain according to the at least one second labeling box, and then adjusting the first labeling result based on the target labeling box to obtain a second labeling result.

[0010] Furthermore, the traffic light labeling method also includes: when the value of the edge is greater than a preset threshold, determining that the two first labeling boxes corresponding to the edge are located in different connected domains, and when the value of the edge is less than or equal to the preset threshold, determining that the two first labeling boxes corresponding to the edge are located in the same connected domain.

[0011] Furthermore, the traffic light labeling method also includes: obtaining the center position and side length information of at least one second labeling box, and calculating the average value of the center position of at least one second labeling box to obtain the target center point, thereby calculating the average value of the side length information of at least one second labeling box to obtain the target side length, and then determining the target labeling box based on the target center point and the target side length.

[0012] Furthermore, the traffic light labeling method also includes: determining the number of lamp heads included in the traffic light area based on the second labeling result, and when the number of lamp heads is less than a preset number, comparing the labeling results of the historical labeling image and the second labeling result to determine the target area; labeling the target area to obtain the target labeling result.

[0013] Furthermore, the traffic light labeling method further includes: when the number of lamp heads is greater than or equal to a preset number, determining the second labeling result as a target labeling result.

[0014] Furthermore, the traffic light labeling method also includes: labeling a target area in the traffic light area according to the labeling results of the historical labeling image and the second labeling results, and after obtaining the target labeling results, segmenting the traffic light area according to the target labeling results to obtain multiple sub-light areas, and obtaining the brightness value of each sub-light area, so as to determine the color type corresponding to each sub-light area according to the brightness value.

[0015] Furthermore, the traffic light marking method further includes: after determining the color type corresponding to each sub-light area according to the brightness value, controlling the vehicle to move forward or stop according to the color type corresponding to each sub-light area.

[0016] According to another aspect of the present disclosure, a traffic light labeling device is also provided, including: an acquisition module, used to acquire a first labeling result obtained by labeling an image to be labeled, wherein the image to be labeled at least includes a traffic light area, the traffic light area is composed of multiple sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of multiple lamp heads; a construction module, used to construct a data structure diagram according to the first labeling result; an adjustment module, used to adjust the first labeling result based on the data structure diagram to obtain a second labeling result; a labeling module, used to label a target area in the traffic light area according to the labeling result of the historical labeling image and the second labeling result to obtain a target labeling result, wherein the target area is an area where lamp heads among multiple lamp heads whose display brightness is lower than a preset brightness are located.

[0017] According to another aspect of the present disclosure, an electronic device is also provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned traffic light marking method.

[0018] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable a computer to execute the above-mentioned traffic light labeling method.

[0019] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned traffic light marking method is implemented.

[0020] According to another aspect of the present disclosure, an autonomous driving vehicle is also provided, including: an acquisition unit, for acquiring a traffic light image, wherein the traffic light image at least includes a traffic light area, the traffic light area is composed of multiple sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of multiple lamp heads; a processing unit, for annotating the traffic light image to obtain a first annotation result, constructing a data structure diagram according to the first annotation result, and adjusting the first annotation result based on the data structure diagram to obtain a second annotation result, and then annotating a target area in the traffic light area according to the annotation result of the historical annotation image and the second annotation result to obtain a target annotation result, wherein the target area is an area where lamp heads among the multiple lamp heads whose display brightness is lower than a preset brightness are located; a control unit, for controlling the autonomous driving vehicle to move forward or stop according to the target annotation result.

[0021] According to another aspect of the present disclosure, a roadside computing device is also provided, comprising the above-mentioned electronic device and a communication unit, wherein the communication unit is used to send the target labeling result to a cloud control platform.

[0022] In the present disclosure, a method of annotating a target area in a traffic light area according to the annotation results of a historical annotation image and a second annotation result is adopted, by obtaining a first annotation result obtained by annotating the image to be annotated, wherein the image to be annotated at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads, and a data structure diagram is constructed according to the first annotation result, thereby adjusting the first annotation result based on the data structure diagram to obtain a second annotation result, and then annotating the target area in the traffic light area according to the annotation results of the historical annotation image and the second annotation result to obtain a target annotation result, wherein the target area is an area where lamp heads among the plurality of lamp heads whose display brightness is lower than a preset brightness are located.

[0023] In the above process, since the data structure diagram is constructed based on the first annotation result after annotating the image to be annotated, the relative position of each traffic light area can be presented in the data structure diagram, so that the neural network model can obtain the number and position information of the traffic light area through the data structure diagram, and achieve the effect of counting all the traffic light areas in the current environment. Moreover, since the present disclosure can automatically mark the area where the lamp heads with display brightness lower than the preset brightness among multiple lamp heads are located (i.e., the black light area) by comparing the annotation results of the historical annotation images and the second annotation results, it avoids the problem of manually marking the black light area, and achieves the effect of improving the efficiency of traffic light marking.

[0024] It can be seen that the solution provided by the present disclosure achieves the purpose of avoiding the low labeling efficiency caused by manual labeling of traffic lights, thereby achieving the technical effect of improving the efficiency of traffic light labeling, and further solving the problem of low traffic light labeling efficiency in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0026] Figure 1 It is a schematic diagram of a traffic light marking method according to the prior art;

[0027] Figure 2 It is a schematic diagram of a traffic light marking method according to the prior art;

[0028] Figure 3 is a flow chart of a method for marking a traffic light according to Embodiment 1 of the present disclosure;

[0029] Figure 4 is a schematic diagram of a data structure according to Embodiment 1 of the present disclosure;

[0030] Figure 5 is a schematic diagram of a traffic light marking device according to Embodiment 2 of the present disclosure;

[0031] Figure 6 It is a block diagram of an electronic device used to implement the traffic light labeling method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0034] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0035] Example 1

[0036] According to an embodiment of the present disclosure, an embodiment of a method for marking a traffic light is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Figure 3 is a flow chart of a method for marking a traffic light according to an embodiment of the present disclosure, such as Figure 3As shown, the method comprises the following steps:

[0038] Step S302: Obtain a first annotation result obtained by annotating the image to be annotated.

[0039] In an optional embodiment, in a road environment, sensors or cameras for acquiring images to be annotated are deployed on both sides of the road and are fixed. Therefore, in the acquired images to be labeled, the position of the traffic lights is also fixed. The images to be labeled contain at least a traffic light area, which is composed of a plurality of sub-light areas, each of which corresponds to a lamp head in a traffic light, and a traffic light is composed of a plurality of lamp heads. For example, Figure 1 It is a schematic diagram of a traffic light marking method according to the prior art, such as Figure 1 As shown, in the clear night traffic light image, the traffic light area (the large frame outside each traffic light) and the sub-light area (the small frame inside each traffic light) can be seen, where: Figure 1 A total of 4 traffic light areas are shown in FIG. Each traffic light area may have 1 sub-light area or 3 sub-light areas. The embodiment of the present disclosure does not limit the sub-light areas in the traffic light. In addition to the traffic light area, the image to be annotated may also include other image information, such as passing vehicles, billboards, buildings, and trees.

[0040] Optionally, the neural network model can traverse the image to be annotated and annotate it to obtain a first annotation result, wherein the neural network model can only select the annotation result of the traffic light area with three lamp heads, without selecting the annotation results of other traffic light areas such as the traffic light area with two lamp heads and the traffic light area with a halo, wherein the annotation result can be displayed in the form of an annotation box. Furthermore, after obtaining the annotation box of the traffic light area in the image to be annotated, the neural network model can express the content of each traffic light area in the following coordinate form: (x_left_top, y_left_top,; x_right_down, y_right_down,). Among them, x_left_top, y_left_top are the coordinates of the upper left vertex of the annotation box, and x_right_down, y_right_down are the coordinates of the lower right vertex of the annotation box.

[0041] In the above process, by annotating the image to be annotated, the required traffic light area is identified from the image to be annotated, thereby avoiding interference from image information of other traffic light areas in the image to be annotated.

[0042] Step S304: construct a data structure diagram according to the first annotation result.

[0043] In an optional embodiment, the neural network model can convert all traffic light areas in the image to be labeled into a data structure diagram according to the first labeling result. Figure 4 As shown, the data structure graph is a collection of vertices, which are connected by a series of edges. The vertices can be marked by circles, and the edges are the lines connecting these circles. In this step, the neural network model can take each traffic light as a vertex in the data structure graph, and the distance between the center points of each two traffic light areas as the edge of the graph.

[0044] In the above process, by converting all the traffic light areas in the image to be labeled into a data structure graph, the effect of establishing the positional relationship between the traffic light areas is achieved.

[0045] Step S306: adjust the first annotation result based on the data structure diagram to obtain a second annotation result.

[0046] In an optional embodiment, after converting all traffic light areas in the image to be annotated into a data structure diagram, the neural network model can further adjust the first annotation result. First, a preset threshold is set to determine whether the traffic light areas in the data structure diagram can be considered connected. By calculating the length of the edge, if the length of the edge is greater than the preset threshold, the neural network model believes that the two traffic light areas corresponding to the edge are not connected, and the two traffic light areas are not in a connected domain; if the length of the edge is less than or equal to the preset threshold, the neural network model believes that the two traffic light areas corresponding to the edge are connected, and the two traffic light areas are in a connected domain. The neural network model can also use a depth-first search method to obtain the number of connected domains in the data structure diagram. The traffic light areas in each connected domain can be considered as the annotation results of the traffic light areas in the same actual environment, that is, the second annotation results.

[0047] Optionally, for all traffic light areas belonging to the same connected domain, the neural network model can calculate the center points of all traffic light areas and find the mean. For example, if there are 5 traffic light areas in a connected domain, the abscissas of the center points of the 5 traffic light areas are summed up and divided by 5 to get the mean of the abscissas; the ordinates of the center points of the 5 traffic light areas are summed up and divided by 5 to get the mean of the ordinates; at the same time, the neural network model can also take out the length and width of all traffic light areas and find the corresponding mean.

[0048] Furthermore, for each connected domain, the average frame of the connected domain is calculated based on the calculated center point average, length average, and width average. For example, for an image to be labeled captured by a camera, if the neural network model detects that there are 5 connected domains in total and calculates the average frame of these 5 connected domains, it is considered that there are 5 traffic lights in total at the intersection under real conditions, and the location information of each traffic light (i.e., the average frame) is determined.

[0049] In the above process, by constructing a data structure diagram, the relative position of each traffic light area can be presented in the data structure diagram, so that the number and location information of the traffic light areas can be obtained through the data structure diagram, thereby achieving the effect of counting all the traffic light areas in the current environment.

[0050] Step S308 , annotating the target area in the traffic light area according to the annotation result of the historical annotation image and the second annotation result to obtain a target annotation result.

[0051] In an optional embodiment, the annotation result of the historical annotation image may be the annotation result obtained by the neural network model before, for example, the annotation result in the last frame of the annotation image obtained. The target area is the area where the lamp heads whose display brightness is lower than the preset brightness among the multiple lamp heads are located, for example, the black lamp area.

[0052] Optionally, when there are 5 connected domains in the second annotation result, the neural network model traverses the annotation results in all historical annotation images. If the number of traffic light areas in a certain historical annotation image is greater than or equal to 5, no processing is performed on the historical annotation image. If it is found that the number of traffic light areas is less than 5, the neural network model can determine which average box the missing traffic light area corresponds to based on the previously calculated average box, and determine the corresponding traffic light area by the position of the average box. For the determined traffic light area, the neural network model can divide it into three parts (according to the neural network model in step S302 when obtaining the first annotation result, only the annotation results of the traffic light area with three lamp heads are selected). If it is a vertical traffic light area, the neural network model cuts it twice in the horizontal direction to divide the traffic light area into 3 parts; if it is a horizontal traffic light area, the neural network model cuts it twice in the vertical direction to divide the traffic light area into 3 parts. At the same time, the neural network model will calculate the average brightness of the three parts respectively. If each part is greater than a larger brightness threshold, or each part is less than a smaller brightness threshold, the traffic light area will be taken out according to the average frame and the color will be marked as unknown. Otherwise, it will not be marked as unknown. This is mainly because there may be lights that are not black lights in the image to be labeled that are missed, and these traffic light areas should not be processed as unknown light colors.

[0053] It should be noted that if Figure 2As shown in the figure, in the night traffic light image, sometimes due to the flickering of the traffic light, the sensor or camera just happens to capture the traffic light in the off state, that is, Figure 2 In addition, since the neural network model can only learn the labeled content, the neural network model cannot detect the above Figure 2 For the unlabeled black lights in the annotation file, the only way to check whether there are any errors in the black lights being missed is through manual review, which leads to the problem of low efficiency in traffic light annotation in the neural network model training.

[0054] In the embodiment of the present disclosure, by comparing the annotation results of the historical annotation image and the second annotation results, the neural network model can automatically annotate the areas where lamp heads with display brightness lower than a preset brightness among multiple lamp heads are located (i.e., the target areas where black lights are located), thereby avoiding the problem of manually annotating the target areas and achieving the effect of improving the efficiency of traffic light annotation.

[0055] Based on the scheme defined in the above steps S302 to S308, it can be known that in the embodiment of the present disclosure, a method of annotating a target area in a traffic light area according to the annotation results of a historical annotation image and a second annotation result is adopted, by obtaining a first annotation result obtained by annotating the image to be annotated, wherein the image to be annotated at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads, and a data structure diagram is constructed according to the first annotation result, thereby adjusting the first annotation result based on the data structure diagram to obtain a second annotation result, and then annotating the target area in the traffic light area according to the annotation results of the historical annotation image and the second annotation result to obtain a target annotation result, wherein the target area is an area where lamp heads whose display brightness is lower than a preset brightness are located among the plurality of lamp heads.

[0056] It is easy to notice that in the above process, since the data structure diagram is constructed based on the first annotation result after annotating the image to be annotated, the relative position of each traffic light area can be presented in the data structure diagram, so that the neural network model can obtain the number and position information of the traffic light area through the data structure diagram, and achieve the effect of counting all the traffic light areas in the current environment. Moreover, since the present disclosure can automatically mark the area where the lamp heads with display brightness lower than the preset brightness among multiple lamp heads are located (i.e., the black light area) by comparing the annotation results of the historical annotation images and the second annotation results, it avoids the problem of manually marking the black light area, and achieves the effect of improving the efficiency of traffic light annotation.

[0057] It can be seen that the solution provided by the present disclosure achieves the purpose of avoiding the low labeling efficiency caused by manual labeling of traffic lights, thereby achieving the technical effect of improving the efficiency of traffic light labeling, and further solving the problem of low traffic light labeling efficiency in the prior art.

[0058] In an optional embodiment, the neural network model obtains multiple annotation boxes for annotating the image to be annotated, and filters the multiple annotation boxes to obtain multiple first annotation boxes, thereby determining position information and identification information of the multiple first annotation boxes to obtain a first annotation result.

[0059] Optional, such as Figure 2 As shown, there may be multiple types of traffic light areas in the image to be annotated, and each type of traffic light area is composed of a different number of sub-light areas. For example, the traffic light area can be a three-lamp traffic light area, or a light halo traffic light area of ​​a single countdown light, or a two-lamp traffic light area. The neural network model can annotate all of these traffic light areas, thereby obtaining multiple annotation boxes. However, when training the neural network model, in order to improve the training accuracy, it is necessary to distinguish different types of traffic light areas. Therefore, in the embodiment of the present disclosure, the neural network model can filter multiple annotation boxes to select a type of traffic light area for training, for example, select the annotation box of the three-lamp traffic light area, and filter the annotation boxes of other traffic light areas such as the two-lamp traffic light area and the light halo traffic light area.

[0060] Furthermore, after selecting multiple first annotation boxes, the neural network model will also determine the position information and identification information of the multiple first annotation boxes to obtain the first annotation results. For example, the position information of each first annotation box is determined in the form of (x_left_top, y_left_top; x_right_down, y_right_down), and each first annotation box is marked with a serial number for timely positioning and searching, where x_left_top, y_left_top are the coordinates of the upper left vertex of the annotation box, and x_right_down, y_right_down are the coordinates of the lower right vertex of the annotation box.

[0061] In the above process, by filtering multiple annotation boxes to obtain multiple first annotation boxes, one type of traffic light area can be individually annotated and trained, avoiding interference from other different types of traffic light areas, which is conducive to improving the accuracy of traffic annotation.

[0062] In an optional embodiment, the neural network model determines the distance information between multiple first annotation boxes based on the position information in the first annotation result, and determines the edges in the data structure graph based on the distance information, thereby obtaining the identification information of the multiple first annotation boxes from the first annotation result to obtain the vertices in the data structure graph, wherein the data structure graph is composed of edges and vertices.

[0063] Optionally, the neural network model can convert the first annotation box into a data structure diagram according to the first annotation result. The neural network model can determine the distance information between multiple first annotation boxes according to the position information in the first annotation result. For example, the center points in the first annotation boxes corresponding to the two traffic light areas are connected by a line segment. The length of the straight line is the distance between the two first annotation boxes. At the same time, the line segment can be an edge in the data structure diagram. In addition, each first annotation box can be used as a vertex in the data structure diagram.

[0064] It should be noted that, by converting all the first annotation boxes in the image to be annotated into a data structure graph, the effect of determining the positional relationship of the first annotation boxes is achieved.

[0065] In an optional embodiment, the neural network model determines at least one connected domain corresponding to the plurality of first annotation boxes according to the edge value in the data structure graph, and determines at least one second annotation box from the plurality of first annotation boxes according to the at least one connected domain, wherein the at least one second annotation box belongs to the same connected domain, thereby determining a target annotation box corresponding to the at least one connected domain according to the at least one second annotation box, and then adjusting the first annotation result based on the target annotation box to obtain a second annotation result.

[0066] Optionally, the connected domain can be a single connected domain or a multiply connected domain. The neural network model can use each first annotation box as a vertex in the data structure diagram, and the distance between the center points of each two first annotation boxes as the edge of the data structure diagram. When the value of the edge is greater than a preset threshold, it is considered that the two first annotation boxes do not belong to the same connected domain. When it is less than or equal to the preset threshold, it is considered that the two first annotation boxes belong to the same connected domain. At the same time, the neural network model can regard the first annotation boxes belonging to the same connected domain as second annotation boxes. There can be multiple second annotation boxes in a connected domain.

[0067] Furthermore, based on at least one second annotation frame, the neural network model can calculate the center points of all second annotation frames and find the average. At the same time, the neural network model also calculates the average of the length and width of all second annotation frames. Therefore, for each connected domain, the neural network model calculates the average frame (i.e., the target annotation frame) of the connected domain based on the calculated center point average, length average, and width average, i.e., the target annotation frame. On this basis, for all images captured by a certain camera, if the neural network model detects that there are a total of 5 connected domains and calculates the average frame of these 5 connected domains, it is considered that there are a total of 5 traffic lights at the intersection under real conditions, and the location information of each traffic light (i.e., the target annotation frame) is determined.

[0068] Through the above process, the neural network model can determine whether multiple first annotation boxes are in a connected domain through the edges in the data structure graph, and integrate the annotation boxes in a connected domain into a target annotation box, so as to adjust the first annotation result and obtain the second annotation result. This process achieves the effect of reducing the annotation workload and improving the annotation efficiency.

[0069] In an optional embodiment, the neural network model determines that the two first annotation boxes corresponding to the edge are located in different connected domains when the value of the edge is greater than a preset threshold, and determines that the two first annotation boxes corresponding to the edge are located in the same connected domain when the value of the edge is less than or equal to the preset threshold.

[0070] Optionally, the neural network model can be used to determine whether the two first annotation boxes corresponding to the edge are in different connected domains by setting a preset threshold. For example, if the length of the edge is greater than the preset threshold, the neural network model believes that the two first annotation boxes corresponding to the edge are not connected, and the two first annotation boxes are not in a connected domain; if the length of the edge is less than or equal to the preset threshold, the neural network model believes that the two first annotation boxes corresponding to the edge are connected, and the two first annotation boxes are in the same connected domain.

[0071] In the above process, by setting a preset threshold, the first annotation frames with similar distances are summarized into the same connected domain, which is beneficial to reducing the amount of calculation and improving the annotation efficiency while ensuring the relative accuracy of the position.

[0072] In an optional embodiment, the center position and side length information of at least one second annotation box are obtained, and the average value of the center position of at least one second annotation box is calculated to obtain the target center point, and the average value of the side length information of at least one second annotation box is calculated to obtain the target side length, thereby determining the target annotation box based on the target center point and the target side length.

[0073] Optionally, the neural network model can obtain the center position and side length information of at least one second annotation box, and calculate the corresponding average value to obtain the target center point and target side length. For example, if there are 5 second annotation boxes in a connected domain, the horizontal coordinates of the center positions of the 5 second annotation boxes are summed up and divided by 5 to obtain the average value of the horizontal coordinate of the target center point; the vertical coordinates of the center positions of the 5 second annotation boxes are summed up and divided by 5 to obtain the average value of the vertical coordinate of the target center point; at the same time, the neural network model can also take out the length and width of all the second annotation boxes and calculate the corresponding average value, so as to obtain the target center point and target side length of the target annotation box.

[0074] In the above process, by determining the target annotation box according to the target center point and the target side length, multiple second annotation boxes in the same connected domain can be aggregated into a target annotation box to represent the location information and identification information of the traffic light area in this connected domain, which is conducive to improving the annotation efficiency.

[0075] In an optional embodiment, the neural network model can determine the number of lamp heads included in the traffic light area based on the second labeling result. When the number of lamp heads is less than a preset number, the labeling result of the historical labeling image and the second labeling result are compared to determine the target area, label the target area, and obtain the target labeling result; when the number of lamp heads is greater than or equal to the preset number, the second labeling result is determined as the target labeling result.

[0076] Optionally, when there are 5 connected domains in the second annotation result, the neural network model traverses the annotation results in all historical annotation images, and if the number of traffic light areas in a historical annotation image is greater than or equal to 5, no processing is performed on the historical annotation image. If it is found that the number of traffic light areas is less than 5, the neural network model can determine which average box the missing traffic light area corresponds to based on the previously calculated average box, and determine the corresponding traffic light area by the position of the average box.

[0077] It should be noted that, by comparing the annotation results of the historical annotation images with the second annotation results, the neural network model can automatically annotate the areas where lamp heads with display brightness lower than the preset brightness among multiple lamp heads are located (i.e., the target areas where black lights are located), thereby avoiding the problem of manually annotating the target areas and achieving the effect of improving the efficiency of traffic light annotation.

[0078] In an optional embodiment, the neural network model can segment the traffic light area according to the target labeling result to obtain multiple sub-light areas, and obtain the brightness value of each sub-light area, so as to determine the color type corresponding to each sub-light area according to the brightness value.

[0079] Optionally, the neural network model can segment the traffic light area according to the target labeling result to obtain multiple sub-light areas. For example, for a traffic light area with three lamp heads, the neural network model can divide it into three parts. If it is a vertical traffic light area, the neural network model will cut it twice in the horizontal direction to divide the traffic light area into three parts; if it is a horizontal traffic light area, the neural network model will cut it twice in the vertical direction to divide the traffic light area into three parts. At the same time, the neural network model will calculate the average brightness of the three parts respectively. If each part is greater than a larger brightness threshold, or each part is less than a smaller brightness threshold, the traffic light area is taken out according to the average frame and the color is marked as unknown, otherwise it is not marked as unknown. This is mainly because there may be lights that are not black lights in the image to be labeled that are missed, and these traffic light areas should not be processed as unknown light colors.

[0080] In addition, the neural network model can also determine the color type corresponding to each sub-light area based on the brightness value, such as red, yellow, and green in a traffic light.

[0081] In the above process, the neural network model can cut the traffic light area in the image to be labeled, thereby identifying the color corresponding to the sub-light area, thereby achieving the effect of accurately judging the current state of the traffic light.

[0082] In an optional embodiment, the electronic equipment in the vehicle can control the vehicle to move forward or stop according to the color type corresponding to each sub-light area.

[0083] Among them, after the neural network model is trained, it can be deployed on the electronic equipment in the vehicle, so that the electronic equipment can control the driving of the vehicle according to the corresponding color type of each sub-light area, where the color type represents whether the current traffic light is passable, for example, red represents stop driving and green represents forward driving.

[0084] Through the above process, the effect of improving vehicle driving safety is achieved and the risk of traffic accidents is reduced.

[0085] Since the data structure diagram is constructed based on the first annotation result after annotating the image to be annotated, the relative position of each traffic light area can be presented in the data structure diagram, so that the neural network model can obtain the number and position information of the traffic light area through the data structure diagram, and achieve the effect of counting all the traffic light areas in the current environment. Moreover, since the present disclosure can automatically mark the area where the lamp heads with display brightness lower than the preset brightness among multiple lamp heads are located (i.e., the black light area) by comparing the annotation results of the historical annotation images and the second annotation results, it avoids the problem of manually marking the black light area, and achieves the effect of improving the efficiency of traffic light marking.

[0086] Example 2

[0087] According to an embodiment of the present disclosure, there is also provided an embodiment of a traffic light labeling device, wherein: Figure 5 According to a schematic diagram of a traffic light labeling device according to embodiment 2 of the present disclosure, the device includes: an acquisition module 501, used to acquire a first labeling result obtained by labeling an image to be labeled, wherein the image to be labeled at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads; a construction module 503, used to construct a data structure diagram according to the first labeling result; an adjustment module 505, used to adjust the first labeling result based on the data structure diagram to obtain a second labeling result; a labeling module 507, used to label a target area in the traffic light area according to the labeling result of the historical labeling image and the second labeling result to obtain a target labeling result, wherein the target area is an area where lamp heads whose display brightness is lower than a preset brightness are located among the plurality of lamp heads.

[0088] It should be noted that the above-mentioned acquisition module 501, construction module 503, adjustment module 505 and annotation module 507 correspond to steps S302 to S308 in the above-mentioned embodiment, and the examples and application scenarios implemented by the four modules are the same as the corresponding steps, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0089] Optionally, the acquisition module further includes: a first acquisition module, a filtering module and a determination module. The first acquisition module is used to acquire multiple annotation boxes for annotating the image to be annotated; the filtering module is used to filter the multiple annotation boxes to obtain multiple first annotation boxes; the determination module is used to determine the position information and identification information of the multiple first annotation boxes to obtain the first annotation result.

[0090] Optionally, the construction module further includes: a first determination module, a second determination module, and a second acquisition module. The first determination module is used to determine the distance information between the multiple first annotation boxes according to the position information in the first annotation result; the second determination module is used to determine the edges in the data structure graph according to the distance information; the second acquisition module is used to obtain the identification information of the multiple first annotation boxes from the first annotation result to obtain the vertices in the data structure graph, wherein the data structure graph is composed of edges and vertices.

[0091] Optionally, the adjustment module further includes: a third determination module, a fourth determination module, a fifth determination module and a first adjustment module. The third determination module is used to determine at least one connected domain corresponding to multiple first annotation boxes according to the edge value in the data structure diagram; the fourth determination module is used to determine at least one second annotation box from multiple first annotation boxes according to at least one connected domain, wherein at least one second annotation box belongs to the same connected domain; the fifth determination module is used to determine a target annotation box corresponding to at least one connected domain according to at least one second annotation box; and the first adjustment module is used to adjust the first annotation result based on the target annotation box to obtain a second annotation result.

[0092] Optionally, the traffic light labeling device further includes: a sixth determination module and a seventh determination module. The sixth determination module is used to determine that the two first labeling boxes corresponding to the edge are located in different connected domains when the value of the edge is greater than a preset threshold; the seventh determination module is used to determine that the two first labeling boxes corresponding to the edge are located in the same connected domain when the value of the edge is less than or equal to the preset threshold.

[0093] Optionally, the traffic light marking device further includes: a second acquisition module, a first calculation module, a second calculation module and an eighth determination module. The second acquisition module is used to obtain the center position and side length information of at least one second marking frame; the first calculation module is used to calculate the average value of the center position of at least one second marking frame to obtain the target center point; the second calculation module is used to calculate the average value of the side length information of at least one second marking frame to obtain the target side length; the eighth determination module is used to determine the target marking frame according to the target center point and the target side length.

[0094] Optionally, the traffic light labeling device further includes: a ninth determination module, a tenth determination module and a first labeling module. The ninth determination module is used to determine the number of lamp heads included in the traffic light area according to the second labeling result; the tenth determination module is used to compare the labeling result of the historical labeling image and the second labeling result to determine the target area when the number of lamp heads is less than a preset number; the first labeling module is used to label the target area to obtain the target labeling result.

[0095] Optionally, the traffic light labeling device further includes: an eleventh determination module, configured to determine that the second labeling result is a target labeling result when the number of lamp heads is greater than or equal to a preset number.

[0096] Optionally, the above-mentioned labeling module further includes: a segmentation module, a third acquisition module and a twelfth determination module. The segmentation module is used to segment the traffic light area according to the target labeling result to obtain multiple sub-light areas; the third acquisition module is used to obtain the brightness value of each sub-light area; and the twelfth determination module is used to determine the color type corresponding to each sub-light area according to the brightness value.

[0097] Optionally, the traffic light marking device further includes: a control module, used to control the vehicle to move forward or stop according to the color type corresponding to each sub-light area.

[0098] Example 3

[0099] According to another aspect of the embodiments of the present disclosure, an autonomous driving vehicle is further provided, including: a collection unit, configured to collect a traffic light image, wherein the traffic light image at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads;

[0100] The processing unit is used to annotate the traffic light image to obtain a first annotation result, construct a data structure diagram according to the first annotation result, and adjust the first annotation result based on the data structure diagram to obtain a second annotation result, and then annotate the target area in the traffic light area according to the annotation result of the historical annotation image and the second annotation result to obtain a target annotation result, wherein the target area is an area where the lamp heads whose display brightness is lower than the preset brightness among the multiple lamp heads are located. The processing unit can execute the traffic light annotation method in the above-mentioned embodiment 1.

[0101] A control unit is used to control the autonomous driving vehicle to move forward or stop according to the target labeling result.

[0102] In an embodiment of the present disclosure, a method of annotating a target area in a traffic light area according to an annotation result of a historically annotated image and a second annotation result is adopted, by obtaining a first annotation result obtained by annotating the image to be annotated, wherein the image to be annotated at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads, and a data structure diagram is constructed according to the first annotation result, thereby adjusting the first annotation result based on the data structure diagram to obtain a second annotation result, and then annotating the target area in the traffic light area according to the annotation result of the historically annotated image and the second annotation result to obtain a target annotation result, wherein the target area is an area where lamp heads among the plurality of lamp heads whose display brightness is lower than a preset brightness are located.

[0103] It is easy to notice that, after annotating the image to be annotated to obtain the first annotation result, a data structure diagram is constructed according to the first annotation result, and the first annotation result is adjusted based on the data structure diagram to obtain the second annotation result. Therefore, the relative position of each traffic light area can be presented in the data structure diagram, so that the neural network model can obtain the number and position information of the traffic light area, and achieve the effect of counting all the traffic light areas in the current environment. Moreover, since the present disclosure can automatically mark the area (i.e., the target area) where the lamp heads displaying brightness lower than the preset brightness among multiple lamp heads by comparing the annotation results of the historical annotation images and the second annotation results, it avoids the problem of manually marking the target area, and achieves the effect of improving the efficiency of traffic light marking.

[0104] It can be seen that the solution provided by the present disclosure achieves the purpose of avoiding manual labeling of traffic lights, thereby achieving the technical effect of improving the efficiency of traffic light labeling, and further solving the problem of low efficiency of traffic light labeling in the prior art.

[0105] Example 4

[0106] According to another aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is further provided, wherein the computer instructions are used to enable a computer to execute the traffic light labeling method in the above-mentioned embodiment 1.

[0107] Example 5

[0108] According to another aspect of the embodiments of the present disclosure, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the method for marking a traffic light in the above-mentioned embodiment 1 is implemented.

[0109] Example 6

[0110] According to another aspect of an embodiment of the present disclosure, an electronic device is also provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the traffic light marking method in the above-mentioned embodiment 1.

[0111] Figure 6A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0112] like Figure 6 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0113] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0114] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the traffic light labeling method. For example, in some embodiments, the traffic light labeling method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or the communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the traffic light labeling method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the traffic light labeling method in any other appropriate manner (e.g., by means of firmware).

[0115] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. The program code for implementing the method disclosed herein may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0116] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0119] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0120] Example 7

[0121] According to another aspect of an embodiment of the present disclosure, a roadside computing device is further provided, comprising the electronic device in the above-mentioned embodiment 6, and a communication unit, wherein the communication unit is used to send the target labeling result to a cloud control platform.

[0122] Optionally, the electronic device can obtain data from a perception device (such as a roadside camera), such as pictures and videos, to perform image and video processing and data calculation. Optionally, the electronic device itself can also have perception data acquisition and communication functions. In this case, the electronic device is a roadside computing device, such as an AI camera, and the electronic device can directly perform image and video processing and data calculation based on the acquired perception data.

[0123] Optionally, the cloud control platform can also be called a vehicle-road collaborative management platform, an edge computing platform, a cloud computing platform, a central system, a cloud server, etc.

[0124] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0125] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for marking a traffic light, comprising: Acquire a first annotation result obtained by annotating the image to be annotated, wherein the image to be annotated at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads; Constructing a data structure graph according to the first annotation result, wherein the data structure graph is composed of edges and vertices, the edges are determined by distance information between a plurality of first annotation boxes in the first annotation result, the vertices are determined by identification information of the plurality of first annotation boxes, and the first annotation boxes are used to annotate traffic light areas of the same type in the image to be annotated; Determine at least one connected domain corresponding to the plurality of first annotation boxes according to the edge value in the data structure graph; determine at least one second annotation box from the plurality of first annotation boxes according to the at least one connected domain, wherein the at least one second annotation box belongs to the same connected domain; determine a target annotation box corresponding to the at least one connected domain according to the at least one second annotation box; and adjust the first annotation result based on the target annotation box to obtain a second annotation result; A target area in the traffic light area is annotated according to the annotation results of the historical annotated image and the second annotation result to obtain a target annotation result, wherein the target area is an area where lamp heads whose display brightness is lower than a preset brightness among the multiple lamp heads are located.

2. The method according to claim 1, wherein: Obtaining a first annotation result obtained by annotating the image to be annotated, including: Acquire multiple annotation boxes for annotating the image to be annotated; Filtering the multiple annotation boxes to obtain multiple first annotation boxes; The position information and identification information of the multiple first annotation boxes are determined to obtain the first annotation result.

3. The method according to claim 2, wherein: Constructing a data structure diagram according to the first annotation result, including: The distance information between the plurality of first annotation boxes is determined according to the position information in the first annotation result.

4. The method according to claim 3, further comprising: When the value of the edge is greater than a preset threshold, determining that the two first annotation boxes corresponding to the edge are located in different connected domains; When the value of the edge is less than or equal to the preset threshold, it is determined that the two first annotation boxes corresponding to the edge are located in the same connected domain.

5. The method according to claim 3, further comprising: Obtaining the center position and side length information of the at least one second annotation frame; Calculating the average value of the center position of the at least one second annotation frame to obtain a target center point; Calculating an average value of the side length information of the at least one second annotation box to obtain a target side length; The target annotation box is determined according to the target center point and the target side length.

6. The method according to claim 5, wherein: The target area in the traffic light area is labeled according to the labeling result of the historical labeling image and the second labeling result to obtain the target labeling result, including: Determining the number of lamp heads included in the traffic light area according to the second labeling result; When the number of lamp heads is less than a preset number, comparing the annotation result of the historical annotation image with the second annotation result to determine the target area; The target area is labeled to obtain the target labeling result.

7. The method according to claim 6, further comprising: When the number of lamp heads is greater than or equal to the preset number, the second marking result is determined to be the target marking result.

8. The method according to claim 1, after annotating the target area in the traffic light area according to the annotation result of the historical annotation image and the second annotation result to obtain the target annotation result, the method further comprises: Segmenting the traffic light area according to the target labeling result to obtain the plurality of sub-light areas; Get the brightness value of each sub-light area; The color type corresponding to each sub-lamp area is determined according to the brightness value.

9. The method according to claim 8, after determining the color type corresponding to each sub-lamp area according to the brightness value, the method further comprises: The vehicle is controlled to move forward or stop according to the color type corresponding to each sub-light area.

10. A traffic light marking device, comprising: an acquisition module, configured to acquire a first annotation result obtained by annotating the image to be annotated, wherein the image to be annotated at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads; a construction module, configured to construct a data structure graph according to the first annotation result, wherein the data structure graph is composed of edges and vertices, the edges are determined by distance information between a plurality of first annotation boxes in the first annotation result, the vertices are determined by identification information of the plurality of first annotation boxes, and the first annotation boxes are used to annotate traffic light areas of the same type in the image to be annotated; an adjustment module, configured to determine at least one connected domain corresponding to the plurality of first annotation boxes according to the edge value in the data structure graph; determine at least one second annotation box from the plurality of first annotation boxes according to the at least one connected domain, wherein the at least one second annotation box belongs to the same connected domain; determine a target annotation box corresponding to the at least one connected domain according to the at least one second annotation box; and adjust the first annotation result based on the target annotation box to obtain a second annotation result; A labeling module is used to label a target area in the traffic light area according to the labeling results of the historical labeling image and the second labeling result to obtain a target labeling result, wherein the target area is an area where lamp heads whose display brightness is lower than a preset brightness among the multiple lamp heads are located.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the traffic light marking method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the traffic light marking method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for marking a traffic light according to any one of claims 1 to 9 is implemented.

14. An autonomous driving vehicle comprising: A collection unit, used for collecting a traffic light image, wherein the traffic light image at least includes a traffic light area, the traffic light area is composed of a plurality of sub-light areas, each sub-light area corresponds to a lamp head in a traffic light, and the traffic light is composed of a plurality of lamp heads; a processing unit, configured to annotate the traffic light image to obtain a first annotation result, and construct a data structure graph according to the first annotation result, wherein the data structure graph is composed of edges and vertices, the edges are determined by distance information between a plurality of first annotation boxes in the first annotation result, and the vertices are determined by identification information of the plurality of first annotation boxes, the first annotation boxes are used to annotate traffic light areas of the same type in the image to be annotated, and at least one connected domain corresponding to the plurality of first annotation boxes is determined according to the edge values ​​in the data structure graph; at least one second annotation box is determined from the plurality of first annotation boxes according to the at least one connected domain, wherein the at least one second annotation box belongs to the same connected domain; a target annotation box corresponding to the at least one connected domain is determined according to the at least one second annotation box; the first annotation result is adjusted based on the target annotation box to obtain a second annotation result, and a target area in the traffic light area is annotated according to the annotation result of the historical annotation image and the second annotation result to obtain a target annotation result, wherein the target area is an area where lamp heads whose display brightness is lower than a preset brightness are located among the plurality of lamp heads; A control unit is used to control the autonomous driving vehicle to move forward or stop according to the target labeling result.

15. A roadside computing device, comprising the electronic device according to claim 11, and a communication unit, wherein the communication unit is used to send the target annotation result to a cloud control platform.

Images