A method and device for marking traffic lights on a three-dimensional map

Abstract

The present application relates to the field of autonomous driving technology, and discloses a method and device for annotating traffic lights on a three-dimensional map, wherein the method comprises obtaining the position of the outer frame of the traffic light to be annotated; based on the position of the outer frame, obtaining a data packet containing different perspectives of the traffic light to be annotated; calculating the view cone of all cameras according to the data packet; based on the view cone, obtaining pictures containing the traffic light to be annotated at different perspectives located in a preset area; inputting the obtained pictures into a preset picture model respectively to obtain the attribute information of the traffic light to be annotated; fusing and judging the attribute information of the traffic light to be annotated to obtain the backfill information of the traffic light to be annotated; projecting the outer frame of the traffic light to be annotated onto pictures containing the traffic light to be annotated at different perspectives, and annotating the backfill information on the traffic light to be annotated. The present application can improve the problem that it is difficult to obtain attribute information or the obtained attribute information is inaccurate when annotating map traffic lights.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more particularly to a method and device for marking traffic lights on a three-dimensional map. Background Art

[0002] At present, when building a map, it is often necessary to annotate traffic lights in a 3D map, including the outer frame, light arrangement, number of bulbs, bulb color, bulb shape and other attribute information. In a point cloud high-precision map, only the outer frame information of the traffic light can be obtained, and other information is difficult to obtain, or the obtained attribute information is inaccurate.

[0003] With respect to the above-mentioned related technologies, the inventors have discovered that the existing method of marking traffic lights on maps has the problem of difficulty in obtaining attribute information or the obtained attribute information is inaccurate. Summary of the invention

[0004] In order to improve the problem that it is difficult to obtain attribute information or the obtained attribute information is inaccurate when annotating map traffic lights, the present application provides a method and device for annotating three-dimensional map traffic lights.

[0005] In a first aspect, the present application provides a method for marking traffic lights on a three-dimensional map.

[0006] This application is achieved through the following technical solutions:

[0007] A method for marking traffic lights on a three-dimensional map comprises the following steps:

[0008] Get the position of the outer frame of the traffic light to be marked;

[0009] Based on the position of the outer frame, obtaining data packets containing different viewing angles of the traffic light to be marked;

[0010] Calculate the viewing frustum of all cameras according to the data packet;

[0011] Based on the viewing cone, pictures containing the traffic light to be labeled at different viewing angles within a preset area are obtained;

[0012] Input the obtained pictures into a preset picture model respectively to obtain the attribute information of the traffic light to be marked;

[0013] Fusion determines the attribute information of the traffic light to be labeled, and obtains backfill information of the traffic light to be labeled;

[0014] The outer frame of the traffic light to be marked is projected onto pictures containing the traffic light to be marked at different viewing angles, and the backfill information is marked on the traffic light to be marked.

[0015] In a preferred example, the present application may be further configured as follows: the step of obtaining pictures containing the traffic lights to be labeled at different viewing angles in a preset area based on the viewing cone includes:

[0016] Searching for all images and corresponding parameters within an area with a position of the outer frame of the traffic light to be marked as the center and a preset distance as the radius;

[0017] For each image searched, calculate the corresponding camera's view frustum in the world coordinate system according to the corresponding parameters;

[0018] Based on the viewing cone, pictures of different perspectives of the outer frame of the traffic light to be marked at the position within the viewing cone are filtered.

[0019] In a preferred example, the present application may be further configured as follows: the step of fusing and determining the attribute information of the traffic light to be labeled to obtain the backfill information of the traffic light to be labeled includes:

[0020] By voting, the number of identical attribute information of the traffic lights to be marked is counted respectively;

[0021] The attribute information of the traffic lights to be marked with the largest number is used as the backfill information of the traffic lights to be marked.

[0022] In a preferred example, the present application can be further configured as follows: the training steps of the preset image model include:

[0023] Based on the same traffic light, collect pictures from different perspectives, different shooting positions and different time points;

[0024] Mark the outer frame, lit light frame, light shape, light color, number of lights and light arrangement of the traffic light in the picture;

[0025] The outer frame of the traffic light, the lit light frame, the light shape, the light color, the number of lights and the light arrangement information are input into the artificial intelligence model and trained to obtain a picture model.

[0026] In a preferred example, the present application may be further configured as follows: the step of projecting the outer frame of the traffic light to be labeled onto the pictures containing the traffic light to be labeled at different viewing angles includes:

[0027] Detecting pictures containing the traffic lights to be labeled from different perspectives, and obtaining reference frames of at least two traffic lights to be labeled on each picture;

[0028] Calculate the degree of overlap based on the outer frame of the traffic light to be marked and the reference frame, and determine whether the degree of overlap is less than or equal to a preset threshold;

[0029] If the degree of overlap is less than or equal to a preset threshold, the reference frame is retained as the outer frame of the traffic light to be marked.

[0030] In a preferred example, the present application can be further configured as follows: it also includes the following steps:

[0031] If the degree of overlap is greater than a preset threshold, the reference frame is discarded.

[0032] In a preferred example, the present application can be further configured as follows: the attribute information includes shape information, color information, quantity information and sorting information.

[0033] In a second aspect, the present application provides a device for marking traffic lights on a three-dimensional map.

[0034] This application is achieved through the following technical solutions:

[0035] A device for marking traffic lights on a three-dimensional map, comprising:

[0036] A reference module is used to obtain the position of the outer frame of the traffic light to be marked;

[0037] A data packet module, used for acquiring data packets containing different viewing angles of the traffic light to be marked based on the position of the outer frame;

[0038] A frustum module, used for calculating the frustums of all cameras according to the data packet;

[0039] A picture module, used for obtaining pictures containing the traffic light to be marked at different viewing angles in a preset area based on the viewing cone;

[0040] An attribute module, used for inputting the obtained pictures into a preset picture model respectively to obtain attribute information of the traffic light to be marked;

[0041] A fusion judgment module, used for fusion judgment of the attribute information of the traffic light to be labeled, and obtaining backfill information of the traffic light to be labeled;

[0042] The marking module is used to project the outer frame of the traffic light to be marked onto pictures containing the traffic light to be marked at different viewing angles, and mark the backfill information on the traffic light to be marked.

[0043] In a third aspect, the present application provides a computer device.

[0044] This application is achieved through the following technical solutions:

[0045] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned methods for marking traffic lights on a three-dimensional map are implemented.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium.

[0047] This application is achieved through the following technical solutions:

[0048] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned methods for marking traffic lights on a three-dimensional map.

[0049] In summary, compared with the prior art, the technical solution provided by this application has at least the following beneficial effects:

[0050] The position of the outer frame of the traffic light to be marked is obtained to obtain more data information; based on the position of the outer frame, a data packet containing different viewing angles of the traffic light to be marked is obtained; according to the data packet, the view cones of all cameras are calculated; based on the view cones, pictures containing the traffic light to be marked at different viewing angles within a preset area are obtained to obtain more pictures of traffic lights within the preset area; based on the inspection results of multiple pictures, pictures with obvious detection errors or pictures with detection errors or pictures with interference caused by factors such as light are eliminated, which is conducive to improving the fault tolerance rate of subsequent models and making the attribute information of traffic lights more accurate; and then The obtained pictures are respectively input into preset picture models to obtain the attribute information of the traffic light to be labeled, so as to facilitate the acquisition of the attribute information of the traffic light for subsequent filling of more traffic light information; the attribute information of the traffic light to be labeled is integrated and judged to obtain the backfill information of the traffic light to be labeled, so as to further improve the accuracy of the attribute information of the traffic light; the outer frame of the traffic light to be labeled is projected onto pictures containing the traffic light to be labeled from different perspectives, and the backfill information is marked on the traffic light to be labeled to complete the labeling of the traffic light; the problem of difficulty in obtaining attribute information or inaccurate attribute information when labeling map traffic lights is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of a method for marking traffic lights on a three-dimensional map is provided as an exemplary embodiment of the present application.

[0052] Figure 2 A rendering of a method for marking traffic lights on a three-dimensional map provided as yet another exemplary embodiment of the present application.

[0053] Figure 3A structural block diagram of a device for marking traffic lights on a three-dimensional map provided by another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0054] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0057] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0058] Reference Figure 1 , an embodiment of the present application provides a method for marking traffic lights on a three-dimensional map, and the main steps of the method are described as follows.

[0059] S1: Get the position of the outer frame of the traffic light to be marked;

[0060] S2: Based on the position of the outer frame, obtaining data packets containing different perspectives of the traffic light to be marked;

[0061] S3: Calculate the viewing cones of all cameras according to the data packet;

[0062] S4: Based on the viewing cone, obtaining pictures containing the traffic light to be labeled at different viewing angles within a preset area;

[0063] S5: inputting the obtained pictures into a preset picture model respectively to obtain the attribute information of the traffic light to be marked;

[0064] S6: Fusion determines the attribute information of the traffic light to be labeled, and obtains backfill information of the traffic light to be labeled;

[0065] S7: Projecting the outer frame of the traffic light to be labeled onto pictures containing the traffic light to be labeled at different viewing angles, and marking the backfill information on the traffic light to be labeled.

[0066] When building a map, it is necessary to annotate the traffic lights in the map, such as red and green lights. Traffic lights include full-screen lights and arrow lights. When annotating traffic lights, attribute information such as outer borders, shape information (such as straight lights, left turn lights, right turn lights, ETC lights, circles, left arrows, right arrows, etc.), color information (such as red, yellow, green, etc.), quantity information (such as 1, 3, etc.) and sorting information (such as horizontal arrangement, vertical arrangement, etc.) is required. In point cloud high-precision maps, only the outer borders of traffic lights can be obtained, and other information is difficult to obtain and judge. Therefore, this application introduces a method for automatically annotating traffic lights using high-precision maps combined with collected data packets.

[0067] Specifically, the position of the outer frame of the traffic light to be marked on the high-precision map is obtained. Based on the position of the outer frame, a data packet containing different perspectives of the traffic light to be marked is obtained, wherein the data packet contains the current position information of the vehicle and pictures taken in various directions. According to the obtained data packet, the view cone of all cameras is calculated; based on the view cone, pictures containing the traffic light to be marked at different perspectives located in the preset area are obtained, so as to find different pictures containing the traffic light to be marked through the view cone. The obtained pictures are respectively input into the preset picture model to obtain the attribute information of the traffic light to be marked, so as to obtain the detailed information of the traffic light in the picture with the help of the picture model detection. The attribute information of the traffic light to be marked obtained by fusion judgment is used to make accurate traffic light information judgment and obtain the backfill information of the traffic light to be marked. The outer frame of the traffic light to be labeled is projected onto pictures containing the traffic light to be labeled from different perspectives, so as to accurately match the outer frame of the traffic light on the high-precision map with the traffic light detected in the picture, and the backfill information is marked on the traffic light to be labeled, so as to fill the traffic light information back into the outer frame of the traffic light on the high-precision map; the traffic light information is obtained by using two-dimensional image data, and then filled into the outer frame of the traffic light on the high-precision map, so as to backfill the traffic light information into the high-precision map and complete the labeling of the traffic light on the three-dimensional map; on the basis of obtaining the three-dimensional outer frame of the traffic light, more traffic light information can be filled in, including the number of bulbs, bulb color, bulb shape, bulb arrangement, etc.

[0068] Furthermore, the step of obtaining pictures containing the traffic lights to be labeled at different viewing angles in a preset area based on the viewing cone includes:

[0069] Searching for all images and corresponding parameters within an area with a position of the outer frame of the traffic light to be marked as the center and a preset distance as the radius;

[0070] For each image searched, calculate the corresponding camera's view frustum in the world coordinate system according to the corresponding parameters;

[0071] Based on the viewing cone, pictures of different perspectives of the outer frame of the traffic light to be marked at the position within the viewing cone are filtered.

[0072] Specifically, the viewing cone is the visible space of the camera, which is surrounded by 6 clipping planes connected by 8 vertices. Based on the position of the outer frame of the traffic light to be marked, all pictures taken within a preset distance from the outer frame and the corresponding parameters are queried. Among them, the corresponding parameters include camera intrinsic parameters, extrinsic parameters, longitudinal field of view angle yfov, lateral field of view angle xfov, camera position camear_pos and camera rotation quaternion camera_qua. In this embodiment, the preset distance can be 5 meters.

[0073] For each acquired picture, calculate the corresponding camera's view frustum in the world coordinate system based on the corresponding picture parameters, including:

[0074] Get the offset between the upper and lower sides of the frustum and its xz plane float yf = Mathf.Tan(yfov / 2*Mathf.Deg2Rad);

[0075] Get the offset between the left and right sides of the frustum and its yz plane float xf = Mathf.Tan(xfov / 2*Mathf.Deg2Rad);

[0076] Default Vector3 forward=(1,0,0), Vector3 right=(0,-1,0), Vector3 up=(0,0,1);

[0077] Then the direction vectors of the four sides of the view frustum, that is, the sides not on the near plane or the origin plane, are:

[0078] Vector3 f0=forward-right*xf-up*yf;

[0079] Vector3 f1=forward-right*xf+up*yf;

[0080] Vector3 f2=forward+right*xf-up*yf;

[0081] Vector3 f3=forward+right*xf+up*yf;

[0082] Normalize f0, f1, f2 and f3;

[0083] The near plane distance is taken as near = 1, and the far plane distance is taken as far = 1000;

[0084] Therefore, the 8 vertices of the camera's view cone in the world coordinate system are:

[0085] Vector3 farLeftBottom=camera_pos+far*f0*camera_qua;

[0086] Vector3 farLeftTop=camera_pos+far*f1*camera_qua;

[0087] Vector3 farRightBotoom=camera_pos+far*f2*camera_qua;

[0088] Vector3 farRightTop=camera_pos+far*f3*camera_qua;

[0089] Vector3 nearLeftBottom=camera_pos+near*f0*camera_qua;

[0090] Vector3 nearLeftTop=camera_pos+near*f1*camera_qua;

[0091] Vector3 nearRightBotoom=camera_pos+near*f2*camera_qua;

[0092] Vector3 nearRightTop=camera_pos+near*f3*camera_qua.

[0093] Based on the visual cone, it is determined whether the outer frame of the traffic light to be marked is within the visual cone, and the pictures of different perspectives containing the outer frame of the traffic light to be marked that are located within the visual cone are filtered out. Through the visual cone, it is possible to calculate whether the traffic light in the three-dimensional space is captured on the picture corresponding to the visual cone. If the traffic light is within the visible space, it will be photographed. According to the visible space of all cameras, all the traffic light pictures at the specified location are obtained to obtain pictures of different perspectives containing the outer frame of the traffic light to be marked within the preset area. With the help of the pictures obtained by the visual cone, the shooting position, angle, and time point (weather, lighting, surrounding environment, etc.) are all different. These different pictures can obtain more traffic light information, and by superimposing the inspection results of multiple pictures, more pictures with obvious detection errors or pictures that may be detected incorrectly or pictures that are interfered by factors such as light are eliminated, thereby improving the fault tolerance rate.

[0094] Furthermore, the step of integrating and judging the attribute information of the traffic light to be labeled to obtain the backfill information of the traffic light to be labeled includes:

[0095] By voting, the number of identical attribute information of the traffic lights to be marked is counted respectively;

[0096] The attribute information of the traffic lights to be marked with the largest number is used as the backfill information of the traffic lights to be marked.

[0097] Specifically, the image model identifies the attribute information such as the color and shape of the traffic light based on different perspective images of the outer frame of the same traffic light. Then, a voting method is used to count the number of attribute information of the same traffic lights to be labeled, such as a traffic light with 1 bulb horizontally, 1 bulb vertically, 2 bulbs horizontally, 2 bulbs vertically, etc. The arrangement A with the largest number of traffic lights and the number of bulbs N are calculated, and the recognition results of traffic lights whose arrangement is not A or whose number is not N are filtered out. The attributes such as shape information, color information, quantity information and sorting information that appear the most times are filled into the outer frame as the backfill information of the traffic light frame to be labeled, so as to accurately obtain the attribute information of the traffic light through multi-dimensional images, and further improve the recognition accuracy of the image detection model.

[0098] In one embodiment, the training step of the preset image model includes:

[0099] Based on the same traffic light, collect pictures from different perspectives, different shooting positions and different time points;

[0100] Mark the outer frame, lit light frame, light shape, light color, number of lights and light arrangement of the traffic light in the picture;

[0101] The outer frame of the traffic light, the lit light frame, the light shape, the light color, the number of lights and the light arrangement information are input into the artificial intelligence model and trained to obtain a picture model.

[0102] Specifically, the same traffic light is photographed by a camera at different time points, different shooting positions and different viewing angles to collect pictures of the same traffic light at different viewing angles, different shooting positions and different time points. Then, the outer frame of the traffic light, the lit red and green light frame, the shape of the light, the color of the light, the number of lights and the arrangement of the lights in the collected pictures are manually annotated. Finally, the annotated pictures are input into the artificial intelligence model for training to obtain the picture model. By using the trained picture model, the corresponding attribute information can be obtained by inputting the traffic light picture, so as to facilitate the acquisition of detailed attribute information of the traffic light. In this embodiment, the artificial intelligence model can use YOLOV3.

[0103] Furthermore, the step of projecting the outer frame of the traffic light to be labeled onto the pictures containing the traffic light to be labeled at different viewing angles includes:

[0104] Detecting pictures containing the traffic lights to be labeled from different perspectives, and obtaining reference frames of at least two traffic lights to be labeled on each picture;

[0105] Calculate the degree of overlap based on the outer frame of the traffic light to be marked and the reference frame, and determine whether the degree of overlap is less than or equal to a preset threshold;

[0106] If the degree of overlap is less than or equal to a preset threshold, the reference frame is retained as the outer frame of the traffic light to be marked;

[0107] If the degree of overlap is greater than a preset threshold, the reference frame is discarded.

[0108] The outer frame of the traffic light to be labeled is projected onto all images, and the two-dimensional traffic light matched by the traffic light to be labeled on the image is obtained through the overlap IOU. Since the image has no depth information, at least two reference frames of the traffic light to be labeled are matched on each image. Therefore, the outer frame of the traffic light to be labeled is determined by calculating the overlap IOU of the reference frames of any two traffic lights to be labeled and judging whether the overlap is less than or equal to the preset threshold.

[0109] For example, suppose there are 6 rectangular boxes on a certain picture, and they are sorted according to the classification probability of the classifier category. The probabilities of belonging to the outer frame of the traffic light to be labeled from small to large are A, B, C, D, E, and F. Start with the rectangular box F with the highest probability, and judge whether the overlap IOU between AE and F is greater than the preset threshold; assuming that the overlap between B, D and F exceeds the threshold, then discard B and D, and mark F as the first rectangular box; continue to select E with the highest probability from the remaining rectangular boxes A, C, and E, and then judge the overlap between E and A and C. If the overlap is greater than the threshold, then discard A and C, and mark E as the second rectangular box; repeat the judgment, and finally the only rectangular box that will be retained is the outer frame of the traffic light to be labeled.

[0110] Reference Figure 2 , by obtaining the outer frame of the traffic light on the high-precision map, projecting it onto the pictures obtained from different perspectives containing the traffic lights to be annotated. For example, the gray boxes on the two pictures are the projection results. At the same time, the attribute information of the left-turn green vertical light bulb obtained by the picture model detection is backfilled into the outer frame of the traffic light on the high-precision map.

[0111] In summary, a method for annotating traffic lights on a three-dimensional map obtains the position of the outer frame of the traffic light to be annotated to obtain more data information; based on the position of the outer frame, a data packet containing different perspectives of the traffic light to be annotated is obtained, and according to the data packet, the view cone of all cameras is calculated, and based on the view cone, pictures containing the traffic light to be annotated at different perspectives within a preset area are obtained to obtain more traffic light pictures within the preset area. According to the inspection results of multiple pictures, pictures with obvious detection errors or pictures with detection errors or pictures with interference caused by factors such as light are eliminated, which is conducive to improving the fault tolerance rate of subsequent models, so that the traffic lights are The attribute information is more accurate; the obtained pictures are respectively input into the preset picture model to obtain the attribute information of the traffic light to be labeled, so as to facilitate the acquisition of the attribute information of the traffic light for subsequent filling of more traffic light information; the attribute information of the traffic light to be labeled is integrated and judged to obtain the backfill information of the traffic light to be labeled, so as to further improve the accuracy of the attribute information of the traffic light; the outer frame of the traffic light to be labeled is projected onto the pictures containing the traffic light to be labeled at different perspectives, and the backfill information is marked on the traffic light to be labeled to complete the labeling of the traffic light; the problem of difficulty in obtaining attribute information or inaccurate attribute information when labeling map traffic lights is greatly improved.

[0112] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0113] Reference Figure 3 The embodiment of the present application also provides a device for marking traffic lights on a three-dimensional map, and the device for marking traffic lights on a three-dimensional map corresponds one-to-one to the method for marking traffic lights on a three-dimensional map in the above embodiment. The device for marking traffic lights on a three-dimensional map includes:

[0114] A reference module is used to obtain the position of the outer frame of the traffic light to be marked;

[0115] A data packet module, used for acquiring data packets containing different viewing angles of the traffic light to be marked based on the position of the outer frame;

[0116] A frustum module, used for calculating the frustums of all cameras according to the data packet;

[0117] A picture module, used for obtaining pictures containing the traffic light to be marked at different viewing angles in a preset area based on the viewing cone;

[0118] An attribute module, used for inputting the obtained pictures into a preset picture model respectively to obtain attribute information of the traffic light to be marked;

[0119] A fusion judgment module, used for fusion judgment of the attribute information of the traffic light to be labeled, and obtaining backfill information of the traffic light to be labeled;

[0120] The marking module is used to project the outer frame of the traffic light to be marked onto pictures containing the traffic light to be marked at different viewing angles, and mark the backfill information on the traffic light to be marked.

[0121] The specific definition of a device for marking a three-dimensional map traffic light can be found in the above definition of a method for marking a three-dimensional map traffic light, which will not be repeated here. Each module in the above-mentioned device for marking a three-dimensional map traffic light can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0122] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, any of the above-mentioned methods for marking traffic lights on a three-dimensional map is implemented.

[0123] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0124] S1: Get the position of the outer frame of the traffic light to be marked;

[0125] S2: Based on the position of the outer frame, obtaining data packets containing different perspectives of the traffic light to be marked;

[0126] S3: Calculate the viewing cones of all cameras according to the data packet;

[0127] S4: Based on the viewing cone, obtaining pictures containing the traffic light to be labeled at different viewing angles within a preset area;

[0128] S5: inputting the obtained pictures into a preset picture model respectively to obtain the attribute information of the traffic light to be marked;

[0129] S6: Fusion determines the attribute information of the traffic light to be labeled, and obtains backfill information of the traffic light to be labeled;

[0130] S7: Projecting the outer frame of the traffic light to be labeled onto pictures containing the traffic light to be labeled at different viewing angles, and marking the backfill information on the traffic light to be labeled.

[0131] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0132] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A method for marking traffic lights on a three-dimensional map, characterized in that: The following steps are included: Get the position of the outer frame of the traffic light to be marked; Based on the position of the outer frame, a data packet containing different viewing angles of the traffic light to be labeled is obtained; based on the data packet, the view cones of all cameras are calculated; based on the view cone, pictures containing the traffic light to be labeled at different viewing angles located in a preset area are obtained, including searching for all pictures and corresponding parameters in an area with the position of the outer frame of the traffic light to be labeled as the center and a preset distance as the radius; for each picture searched, the view cone of the corresponding camera in the world coordinate system is calculated according to the corresponding parameters; based on the view cone, pictures of different viewing angles containing the outer frame of the traffic light to be labeled at the position located in the view cone are filtered; Input the obtained pictures into a preset picture model respectively to obtain the attribute information of the traffic light to be marked; Fusion determines the attribute information of the traffic light to be labeled, and obtains backfill information of the traffic light to be labeled; The outer frame of the traffic light to be marked is projected onto pictures containing the traffic light to be marked at different viewing angles, and the backfill information is marked on the traffic light to be marked.

2. The method for marking traffic lights on a three-dimensional map according to claim 1, characterized in that: The step of integrating and judging the attribute information of the traffic light to be labeled and obtaining the backfill information of the traffic light to be labeled includes: By voting, the number of identical attribute information of the traffic lights to be marked is counted respectively; The attribute information of the traffic lights to be labeled with the largest number is used as the backfill information of the traffic lights to be labeled.

3. The method for marking traffic lights on a three-dimensional map according to claim 1, characterized in that: The training steps of the preset image model include: Based on the same traffic light, collect pictures from different perspectives, different shooting positions and different time points; Mark the outer frame, lit light frame, light shape, light color, number of lights and light arrangement of the traffic light in the picture; The outer frame of the traffic light, the lit light frame, the light shape, the light color, the number of lights and the light arrangement information are input into the artificial intelligence model and trained to obtain a picture model.

4. The method for marking traffic lights on a three-dimensional map according to claim 1, characterized in that: The step of projecting the outer frame of the traffic light to be labeled onto the pictures containing the traffic light to be labeled at different viewing angles comprises: Detecting pictures containing the traffic lights to be labeled from different perspectives, and obtaining reference frames of at least two traffic lights to be labeled on each picture; Calculate the degree of overlap based on the outer frame of the traffic light to be marked and the reference frame, and determine whether the degree of overlap is less than or equal to a preset threshold; If the degree of overlap is less than or equal to a preset threshold, the reference frame is retained as the outer frame of the traffic light to be marked.

5. The method for marking traffic lights on a three-dimensional map according to claim 4, characterized in that: The following steps are also included: If the degree of overlap is greater than a preset threshold, the reference frame is discarded.

6. The method for marking traffic lights on a three-dimensional map according to any one of claims 1 to 5, characterized in that: The attribute information includes shape information, color information, quantity information and sorting information.

7. A device for marking traffic lights on a three-dimensional map, characterized in that: include, A reference module is used to obtain the position of the outer frame of the traffic light to be marked; A data packet module, used for acquiring data packets containing different viewing angles of the traffic light to be marked based on the position of the outer frame; A frustum module, used for calculating the frustums of all cameras according to the data packet; A picture module is used to obtain pictures containing the traffic light to be labeled at different viewing angles within a preset area based on the view cone, including searching for all pictures and corresponding parameters within an area with a position of the outer frame of the traffic light to be labeled as the center and a preset distance as the radius; for each picture searched, the view cone of the corresponding camera in the world coordinate system is calculated according to the corresponding parameters; Based on the viewing cone, filtering pictures of different perspectives of the outer frame of the traffic light to be marked at the position within the viewing cone; An attribute module, used for inputting the obtained pictures into a preset picture model respectively to obtain attribute information of the traffic light to be marked; A fusion judgment module, used for fusion judgment of the attribute information of the traffic light to be labeled, and obtaining backfill information of the traffic light to be labeled; The marking module is used to project the outer frame of the traffic light to be marked onto pictures containing the traffic light to be marked at different viewing angles, and mark the backfill information on the traffic light to be marked.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images