High-precision map matching-based automatic driving traffic light perception method and device

Abstract

The application discloses an automatic driving traffic light perception method and device based on high-precision map matching, and belongs to the field of automatic driving. The method comprises the following steps: obtaining 3D position information of each lamp group frame and indication direction information of each lamp group frame from a high-precision map according to a current frame identification image; projecting the 3D position information of each lamp group frame into the current frame identification image to obtain each envelope frame corresponding to each lamp group frame; matching each envelope frame with each lamp group frame identification frame in the current frame identification image; determining the type of each lamp group frame by using the corresponding indication direction information of the lamp group frame for each pair of envelope frame and lamp group frame identification frame that is matched successfully; and obtaining the time sequence state of the same lamp group frame of the type according to the type of the lamp group frame and the time sequence state of each same lamp group frame. The application has real-time performance and high accuracy in the state perception of traffic lights.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and in particular to an autonomous driving traffic light perception method and device based on high-precision map matching. Background Technology

[0002] With the rise of artificial intelligence, autonomous driving has become a hot technology in the automotive industry. Just as humans pay attention to traffic light information while driving, autonomous driving relies even more heavily on traffic light information. With this information, autonomous vehicles can better interact with the road, making the design of a reliable traffic light perception system crucial.

[0003] Traditional map-based methods for detecting traffic lights rely on accurate GPS measurements and manually labeled traffic light information to obtain more accurate prior knowledge. Upon approaching the traffic light, geometric transformations are used to obtain candidate regions, which are then classified. Alternatively, offline saliency maps are generated using GPS, and vehicle-mounted camera parameters are used to triangulate the area where the traffic light appears upon approach. Convolutional neural networks and template matching are then employed to detect the traffic light category. However, this approach is overly dependent on sensor equipment, resulting in high costs for achieving the same effect, and requires a large amount of manually labeled traffic light information, making it time-consuming and labor-intensive. Summary of the Invention

[0004] To address the problems of existing technologies, such as high equipment dependence, excessive cost for achieving the same effect, need for a large amount of manual data annotation, and poor real-time matching of traffic lights, this application mainly provides an autonomous driving traffic light perception method and device based on high-precision map matching.

[0005] One technical solution adopted in this application is: providing an autonomous driving traffic light perception method based on high-precision map matching, which includes:

[0006] Using time-series images captured by an onboard camera, target detection is performed to obtain a current frame labeled image after the light group boxes of each traffic light in the current frame image are labeled using light group box labeling frames.

[0007] The state information of each light group frame in the detected time sequence image is automatically learned to obtain the time sequence state of each of the same light group frames corresponding one-to-one in different frame images.

[0008] The current frame identifier image obtains the 3D position information of each light group frame and the indication direction information of each light group frame from the high-precision map;

[0009] Project the 3D position information of each light group frame onto the current frame identifier image to obtain each envelope frame corresponding to each light group frame.

[0010] matching each envelope frame with each lamp group frame in the current frame identification image;

[0011] for each pair of envelope frame and lamp group frame that is matched successfully, determining the type of the lamp group frame by using the corresponding indication direction information of the lamp group frame; and

[0012] obtaining the time sequence state of the same lamp group frame of the type according to the type of the lamp group frame and the time sequence state of each same lamp group frame.

[0013] Another technical solution adopted by the present application is to provide an automatic driving traffic light perception device based on high-precision map matching, which comprises:

[0014] a module for performing target detection on the time sequence image collected by the vehicle-mounted camera to obtain a current frame identification image in which each group of traffic lights in a current frame image in the time sequence image is identified by using lamp group frame identification frames;

[0015] a module for automatically learning the state information of each lamp group frame in the detected time sequence image to obtain the time sequence state of each same lamp group frame which is one-to-one corresponding between different frame images;

[0016] a module for obtaining the 3D position information of each lamp group frame and the indication direction information of each lamp group frame from the high-precision map by the current frame identification image;

[0017] a module for projecting the 3D position information of each lamp group frame into the current frame identification image to obtain each envelope frame corresponding to each lamp group frame;

[0018] a module for matching each envelope frame with each lamp group frame in the current frame identification image;

[0019] a module for determining the type of the lamp group frame by using the corresponding indication direction information of the lamp group frame for each pair of envelope frame and lamp group frame that is matched successfully; and

[0020] a module for obtaining the time sequence state of the same lamp group frame of the type according to the type of the lamp group frame and the time sequence state of each same lamp group frame.

[0021] Another technical solution adopted by the present application is to provide a current intersection driving state decision method in automatic driving, which comprises:

[0022] performing target detection on the time sequence image collected by the vehicle-mounted camera to obtain a current frame identification image in which each group of traffic lights in a current frame image in the time sequence image is identified by using lamp group frame identification frames;

[0023] The state information of each lamp group frame in the detected time sequence image is automatically learned to obtain the time sequence state of each same lamp group frame corresponding one by one between different frame images.

[0024] The 3D position information of each lamp group frame and the indication direction information of each lamp group frame are obtained from the high-definition map according to the current frame identification image;

[0025] The 3D position information of each lamp group frame is projected into the current frame identification image to obtain each envelope frame corresponding to each lamp group frame;

[0026] Each envelope frame is matched with each lamp group frame identification frame in the current frame identification image;

[0027] For each pair of envelope frame and lamp group frame identification frame matched successfully, the type of the lamp group frame is determined by using the corresponding indication direction information of the corresponding lamp group frame;

[0028] The time sequence state of the same lamp group frame of the type is obtained according to the type of the lamp group frame and the time sequence state of each same lamp group frame; and

[0029] The passing state of the current intersection is obtained according to the time sequence state of the same lamp group frame of the type, and the driving state of the autonomous vehicle is decided according to the passing state.

[0030] Another technical solution adopted by the present application is to provide a computer readable storage medium storing computer instructions, the computer instructions being operated to execute the automatic driving traffic light perception method based on high-definition map matching in scheme one.

[0031] Another technical solution adopted by the present application is to provide a computer device including a processor and a memory, the memory storing computer instructions, the computer instructions being operated to execute the automatic driving traffic light perception method based on high-definition map matching in scheme one.

[0032] The technical solution of the present application can achieve the beneficial effects that the present application designs an automatic driving traffic light perception method and device based on high-definition map matching. The method combines high-definition map and time sequence state of lamp group frame, without manually labeled traffic light information, so that the vehicle can match the time sequence state of the current lane and the corresponding traffic light in real time during automatic driving, and then make a timely decision, improve the accuracy and real-time performance, and reduce the equipment cost. BRIEF DESCRIPTION OF DRAWINGS

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of a specific implementation of an autonomous driving traffic light perception method based on high-precision map matching according to this application;

[0035] Figure 2 This is a schematic diagram of a specific implementation of an autonomous driving traffic light perception device based on high-precision map matching according to this application.

[0036] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0037] The preferred embodiments of this application will now be described in detail with reference to the accompanying drawings, so that the advantages and features of this application can be more easily understood by those skilled in the art, thereby providing a clearer and more definite definition of the scope of protection of this application.

[0038] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0039] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0040] Figure 1This paper illustrates a specific implementation of an autonomous driving traffic light perception method based on high-precision map matching, as proposed in this application. Figure 1 The specific implementation shown includes an autonomous driving traffic light perception method based on high-precision map matching, comprising:

[0041] Step S101: Using the time-series images captured by the vehicle-mounted camera, target detection is performed to obtain the current frame identification image after the light group frames of each group of traffic lights in the current frame image of the time-series image are identified by the light group frame identification boxes.

[0042] In this embodiment, the time-series image is marked with light group frame identifiers. By using the ID information corresponding to the light group frame identifiers, it is easy to track the position of the light group frames in the time-series image. This makes it easy to identify light group frames belonging to the same target and also facilitates matching with high-precision maps.

[0043] exist Figure 1 In the specific implementation shown, the autonomous driving traffic light perception method based on high-precision map matching further includes:

[0044] Step S102: Automatically learn the state information of each light group frame in the detected time sequence image to obtain the time sequence state of each same light group frame corresponding to each other in different frame images.

[0045] In this embodiment, the number of light group frames in the identification timing image is not uniform. This application focuses on the light group frames that can be detected within a certain time sequence. A light group frame in the identification timing image can only be in one state, which can only be one of red, yellow, green, or off. This is the state information of the light group frame.

[0046] In an optional embodiment of this application, the method of automatically learning the state information of each light group frame in the detected time-series image to obtain the time-series state of each same light group frame corresponding to each other in different frame images, further includes: tracking each light group frame in the detected time-series image to obtain the one-to-one correspondence between each light group frame in different frame images, thereby determining each same light group frame corresponding to each other in different frame images; and automatically learning the state information of each light group frame in the detected time-series image to obtain the time-series state of each same light group frame.

[0047] In this embodiment, there is a correlation between the light group frames in different frame images. The purpose of tracking is to combine the state information of the correlated light group frames together in order to obtain the timing state of the same light group frame. The timing state includes states such as constant red, constant yellow, yellow flashing, constant green, green flashing, and off.

[0048] This application obtains a temporal state prediction model by training it on state information from different frame images. The trained temporal state prediction model can efficiently determine the temporal state of the same light group frame in a temporal image. This temporal state prediction model can be seen as the first application in the field of traffic light perception, breaking through the limitations of existing technologies. Furthermore, the temporal state prediction model can improve the efficiency and real-time performance of traffic light perception.

[0049] In one example of this application, based on the ID of the light group frame in the current frame identification image and the color and on / off information of the light group frame, the state information of the light group frames belonging to the same traffic light target in multiple frame identification time-series images, including the current frame identification image, is obtained and a state vector is formed; the time-series state prediction model performs prediction and reasoning on the state vector to obtain the time-series state of the light group frame in the time-series image.

[0050] exist Figure 1 In the specific implementation shown, the autonomous driving traffic light perception method based on high-precision map matching further includes:

[0051] Step S103: The current frame identifier image obtains the 3D position information of each light group frame and the indication direction information of each light group frame from the high-precision map.

[0052] In this embodiment, the directional information includes directions such as left turn, straight ahead, right turn, and U-turn. A coordinate system is established in the high-precision map using the current autonomous vehicle, and the 3D position information of the current intersection light fixture frame is calculated. This 3D position information includes not only the coordinates of the light fixture frame but also the distance and direction between the light fixture frame and the current autonomous vehicle.

[0053] There are several types of traffic lights on the market. One type guides motor vehicles, consisting of a group of three plain circles (red, green, and yellow) forming a light group frame. Another type guides vehicles within a lane to proceed according to the instructions, consisting of a group of cross and arrow patterns forming a light group frame. A third type guides motor vehicles to proceed in the indicated direction, consisting of a group of three lights (red, yellow, and green) containing arrow patterns forming a light group frame. A fourth type alerts vehicles and pedestrians to be cautious when crossing, consisting of a group of continuously flashing yellow lights forming a light group frame. A fifth type guides non-motorized vehicles or pedestrians, consisting of a group of three lights (red, yellow, and green) containing bicycle or person patterns forming a light group frame.

[0054] In one example of this application, the high-precision map indicates that there are three traffic lights 100 meters ahead of the current vehicle at an intersection, with the three traffic lights indicating left turn, straight ahead, and right turn respectively.

[0055] In one optional embodiment of this application, the 3D position information is the three-dimensional coordinates corresponding to the eight corner points of the lamp frame.

[0056] In this embodiment, in reality, the light group frame corresponding to the traffic light at the current intersection is a three-dimensional object. The light group frame can be abstracted as a cuboid, with each cuboid having eight corner points. The 3D position information includes the three-dimensional coordinates of the eight corner points of the cuboid. Under the current vehicle coordinate system centered on the autonomous vehicle, this corresponds to 3D coordinates in the real world, facilitating projection into the timing image of the signage.

[0057] In one example of this application, an autonomous vehicle needs to focus on only one traffic light, i.e., only one envelope frame. However, there may be multiple light group frame identifiers in the identification timing image. It is necessary to match the envelope frame with the light group frame identifiers one by one and identify the light group frame identifiers that match the envelope frame so that the vehicle can pass according to the timing status of the traffic lights.

[0058] In one example of this application, because real-world traffic lights have height and thickness, they can be represented as cuboids in a high-precision map. In the current vehicle coordinate system centered on the autonomous vehicle, each of the eight corner points of the traffic light has x, y, and z axis data. These eight corner points can be projected onto the plane of the time-series image. The outer envelope of these eight corner points is then obtained, resulting in a rectangular bounding box. This bounding box is matched against the detected light group bounding boxes in the time-series image to determine which light group bounding box it matches. Combined with the directional information of the traffic lights in the high-precision map, the traffic status in that direction can be determined.

[0059] exist Figure 1 In the specific implementation shown, the autonomous driving traffic light perception method based on high-precision map matching further includes:

[0060] Step S104: Project the 3D position information of each light group frame onto the current frame identifier image to obtain each envelope frame corresponding to each light group frame.

[0061] In this embodiment, the eight corner points in the 3D position information of the light group frame together form a cuboid. Projecting it onto the current frame identifier image yields an envelope, which facilitates subsequent matching with the traffic lights at the current intersection.

[0062] In one optional embodiment of this application, the 3D position information of each light group frame is projected onto the current frame identifier image to obtain each envelope frame corresponding to each light group frame, including: projecting the 3D position information of each light group frame onto the current frame identifier image through the calibration parameters of the vehicle camera to obtain the outer envelope of each light group frame in the current frame identifier image; and obtaining each corresponding envelope frame based on each outer envelope.

[0063] In this embodiment, the calibration parameters of the vehicle-mounted camera are equivalent to a transformation matrix with specific transformation relationships. The coordinates of the light group frame in the high-precision map can be projected onto the marker time sequence image. The outer envelope of the projection is taken from the marker time sequence image to obtain the projected envelope frame, which serves as the basis for subsequent recognition.

[0064] exist Figure 1 In the specific implementation shown, the autonomous driving traffic light perception method based on high-precision map matching further includes:

[0065] Step S105: Match each envelope box with the identification boxes of each light group in the current frame identification image.

[0066] In this embodiment, the process of matching the envelope frame and the lamp group frame identifier frame is equivalent to calculating the degree of overlap between the two. The purpose is to link the information of the lamp group frame in the identifier time sequence image with the lamp group frame in the real world.

[0067] In an optional embodiment of this application, matching each envelope box with each light group frame identifier box in the current frame identifier image includes: performing a global optimal match between each envelope box and each light group frame identifier box in the current frame identifier image, solving for the system deviation, and considering the envelope box and the light group frame identifier box to be successfully matched when the system deviation is less than the error threshold.

[0068] In this embodiment, by searching for the global optimum between each envelope frame and each light group frame in the current frame's identifier image, a fixed error deviation can be tolerated to a certain extent.

[0069] In one example of this application, the accuracy map includes positioning error and mapping error, and the vehicle-mounted camera includes calibration error. For all results of detection of a frame of temporal image, whether it is positioning error, mapping error, or calibration error, all three are a constant systematic error.

[0070] exist Figure 1 In the specific implementation shown, the autonomous driving traffic light perception method based on high-precision map matching further includes:

[0071] Step S106: For each successfully matched pair of envelope frames and lamp group frame identifier frames, the type of the lamp group frame is determined using the corresponding indicator direction information of the lamp group frame.

[0072] In this embodiment, the types include left turn lights, right turn lights, straight-ahead lights, and / or U-turn lights. It determines which light group frame in the current frame's identifier image successfully matches the envelope frame, and based on the direction information in the corresponding light group frame, determines that the light group frame contains one or more of the following: left turn lights, right turn lights, straight-ahead lights, and / or U-turn lights. This method is highly real-time, reliable, and accurate.

[0073] In one example of this application, the shape of traffic lights can be identified through matching, such as left-turn arrows and U-turn arrows. Of course, if the shape of the traffic light is only circular, then the directional information of the high-precision map is combined with the timing status of the straight-ahead light at the current intersection to determine the traffic situation at the current intersection. When the straight-ahead light is green, a right turn is allowed but a left turn is not allowed.

[0074] In an optional embodiment of this application, for each successfully matched pair of envelope frames and light group frame identifier frames, the type of the light group frame is determined by using the indication direction information corresponding to the corresponding light group frame. The method further includes: loading the light group frame corresponding to each successfully matched pair of envelope frames and light group frame identifier frames, along with the indication direction information corresponding to the corresponding light group frame, onto the status information corresponding to the corresponding light group frame to obtain the type of the corresponding light group frame.

[0075] In this embodiment, in the time-series image, the corresponding direction information from the high-precision map is loaded onto the status information of the successfully matched light group frame, and the analysis is performed to obtain the type of the corresponding light group frame, thereby identifying the type of the same light group frame in the time-series image.

[0076] exist Figure 1 In the specific implementation shown, the autonomous driving traffic light perception method based on high-precision map matching further includes:

[0077] Step S107: Based on the type of lamp group frame and the timing status of each lamp group frame of the same type, obtain the timing status of the lamp group frame of the same type.

[0078] In this embodiment, the types of light group frames can be obtained by matching the high-precision map, and the time sequence status of the same light group frame in the time sequence image can be obtained by combining the time sequence status of the same light group frame of various types.

[0079] This application detects light group frames in time-series images and marks them with identification boxes at corresponding positions in each frame. This prevents target loss and records the state information of the light group frames based on their corresponding ID information. It finds identical light group frames in the time-series images and predicts their temporal state using their state information. By matching the position and direction information of the light group frames in the high-precision map with the same light group frames in the time-series images, it obtains the type and temporal state of the light group frames at the current intersection. This application demonstrates high real-time performance and decision-making capabilities in autonomous driving decision-making.

[0080] Figure 2 This paper illustrates a specific implementation of an autonomous driving traffic light perception device based on high-precision map matching according to this application.

[0081] exist Figure 2 In the specific implementation shown, the autonomous driving traffic light perception device based on high-precision map matching mainly includes:

[0082] Module 201 is used to perform target detection using time-series images captured by an on-board camera, and to obtain a current frame labeled image after labeling the light group frames of each group of traffic lights in the current frame image of the time-series image with light group frame labeling frames.

[0083] Module 202 is used to automatically learn the state information of each light group frame in the detected time sequence image to obtain the time sequence state of each of the same light group frames corresponding one-to-one in different frame images.

[0084] Module 203 is used to obtain the 3D position information of each light group frame and the indication direction information of each light group frame from the high-precision map in the current frame identifier image.

[0085] Module 204 is used to project the 3D position information of each light group frame onto the current frame identifier image to obtain each envelope frame corresponding to each light group frame.

[0086] Module 205 is used to match each envelope box with the identification box of each light group in the current frame identification image;

[0087] Module 206 is used to determine the type of light group frame for each successfully matched pair of envelope frames and light group frame identifier frames, using the corresponding indicator direction information of the light group frame; and

[0088] Module 207 is used to obtain the timing status of the same type of lamp group frame based on the type of lamp group frame and the timing status of each lamp group frame.

[0089] In this embodiment, the temporal state of light group frames in the time-series image is conveniently, efficiently, and in real-time detected, determining whether the light group frame is red, green, yellow, flashing green, or flashing yellow, facilitating the autonomous vehicle's judgment of traffic lights. Using relevant information from high-precision maps, the light group frames in the time-series image are matched with the light group frames at the current intersection in the real world. Based on the current driving situation, it is determined which light group frame requires current attention, and the state of the light group frame requiring attention is determined, facilitating the autonomous vehicle's decision-making.

[0090] In one example of this application, the application can use an object detection model to detect whether there are light group frames in a time series image, and mark the light group frames in the time series image with light group frame identifiers. Time series images with light group frame identifiers can easily track the same light group frame in consecutive time series images, and also facilitate the recording of the state of the light group frame.

[0091] For example, in a single frame of a time sequence image, there are three light group frames: one is red, one is green, and the other is off. In the next frame of the time sequence image, the same light group frames are tracked using the light group frame identifier boxes, and their respective states are recorded.

[0092] The autonomous driving traffic light perception device based on high-precision map matching provided in this application can be used to execute the autonomous driving traffic light perception method based on high-precision map matching described in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0093] In one specific embodiment of this application, the functional modules of the autonomous driving traffic light perception device based on high-precision map matching can be directly in hardware, in software modules executed by a processor, or in a combination of both.

[0094] Software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in this art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium.

[0095] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor can be a microprocessor, but alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration. Alternatively, the storage medium can be integrated with the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in the user terminal. Alternatively, the processor and storage medium can reside as discrete components in the user terminal.

[0096] In another specific embodiment of this application, a method for determining the current intersection driving state in autonomous driving includes:

[0097] Using time-series images captured by an onboard camera, target detection is performed to obtain a current frame labeled image after the light group boxes of each traffic light in the current frame image are labeled using light group box labeling frames.

[0098] The state information of each light group frame in the detected time sequence image is automatically learned to obtain the time sequence state of each of the same light group frames corresponding one-to-one in different frame images.

[0099] The current frame identifier image obtains the 3D position information of each light group frame and the indication direction information of each light group frame from the high-precision map;

[0100] Project the 3D position information of each light group frame onto the current frame identifier image to obtain each envelope frame corresponding to each light group frame.

[0101] Match each envelope box with the corresponding light group frame identifier box in the current frame identifier image;

[0102] For each pair of successfully matched envelope frames and light group frame identifier frames, the type of the light group frame is determined using the corresponding indicator direction information of the light group frame.

[0103] Based on the type of light fixture frame and the timing status of each frame within the same type, the timing status of the same type of light fixture frame is obtained; and

[0104] Based on the timing status of the same type of light group frame, the current traffic status of the intersection is obtained, and a decision is made on the driving status of the autonomous vehicle based on the traffic status.

[0105] In this embodiment, the temporal state of the light group frame is combined with the high-precision map, which improves the realism of traffic light perception, as well as its accuracy and real-time performance.

[0106] In another specific embodiment of this application, a computer-readable storage medium is provided, which stores computer instructions that are operated to perform the autonomous driving traffic light perception method based on high-precision map matching or the current intersection driving state decision method in autonomous driving in any embodiment.

[0107] In another specific embodiment of this application, a computer device is provided, which includes a processor and a memory, the memory storing computer instructions that are operated to execute the autonomous driving traffic light perception method based on high-precision map matching or the current intersection driving state decision method in autonomous driving in any embodiment.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for traffic light perception in autonomous driving based on high-precision map matching, characterized in that, include: Using time-series images captured by an onboard camera, target detection is performed to obtain a current frame labeled image after the light group frames of each traffic light in the current frame image of the time-series image are labeled using light group frame labeling frames. The state information of each light group frame in the detected time sequence image is automatically learned to obtain the time sequence state of each same light group frame that corresponds one-to-one between different frame images. The current frame identifier image obtains the 3D position information of each of the light group frames and the indication direction information of each of the light group frames from the high-precision map; The 3D position information of each of the light group frames is projected into the current frame identifier image to obtain each envelope frame corresponding to each of the light group frames. Match each of the envelope boxes with each of the lamp group frame identifier boxes in the current frame identifier image; For each pair of successfully matched envelope frames and lamp group frame identifier frames, the type of the lamp group frame is determined using the corresponding indicator direction information. as well as Based on the type of the lamp group frame and the timing state of each lamp group frame of the same type, the timing state of the lamp group frame of the same type is obtained; The step of automatically learning the state information of each light group frame in the detected time-series image to obtain the time-series state of each corresponding light group frame in different frame images further includes: Track each of the light group frames in the detected time sequence image to obtain the one-to-one correspondence between each of the light group frames in different frame images, and then determine each of the same light group frames that correspond one-to-one in different frame images. The state information of each light group frame in the detected time sequence image is automatically learned to obtain the time sequence state of each light group frame.

2. The autonomous driving traffic light perception method based on high-precision map matching as described in claim 1, characterized in that, The step of projecting the 3D position information of each of the light group frames into the current frame identifier image to obtain each envelope frame corresponding to each of the light group frames includes: The 3D position information of each of the light group frames is projected into the current frame identifier image through the calibration parameters of the vehicle camera to obtain the outer envelope of each of the light group frames in the current frame identifier image; Based on each of the outer envelopes, the corresponding envelope frames are obtained.

3. The autonomous driving traffic light perception method based on high-precision map matching as described in claim 1, characterized in that, The step of matching each of the envelope frames with each of the light group frame identifier frames in the current frame identifier image includes: A global optimal match is performed between each of the envelope boxes and each of the light group frame identifier boxes in the current frame identifier image, and the system deviation is calculated. When the system deviation is less than the error threshold, the envelope box is considered to be successfully matched with the light group frame identifier box.

4. The autonomous driving traffic light perception method based on high-precision map matching as described in claim 1, characterized in that, The step of determining the type of light group frame using the corresponding indicator direction information for each successfully matched pair of envelope frames and light group frame identifier frames also includes: The matching envelope frames and lamp group frame identifier frames, along with the indicator direction information corresponding to the corresponding lamp group frames, are loaded onto the status information corresponding to the corresponding lamp group frames to obtain the type of the corresponding lamp group frames.

5. The autonomous driving traffic light perception method based on high-precision map matching as described in claim 1, characterized in that, The 3D position information refers to the three-dimensional coordinates of the eight corner points of the light assembly frame.

6. An autonomous driving traffic light sensing device based on high-precision map matching, characterized in that, include: This module is used to perform target detection using time-series images captured by an onboard camera, and to obtain a current frame labeled image after the light group frames of each group of traffic lights in the current frame image are labeled using light group frame labeling frames. A module for automatically learning the state information of each of the lamp group frames in the detected time-series images to obtain the time-series state of each of the same lamp group frames corresponding one-to-one in different frame images. A module for obtaining 3D position information of each of the light group frames and indication direction information of each of the light group frames from a high-precision map in the current frame identification image. A module for projecting the 3D position information of each of the light group frames into the current frame identifier image to obtain each envelope frame corresponding to each of the light group frames; A module for matching each of the envelope frames with each of the lamp group frame identifier frames in the current frame identifier image; A module for determining the type of a light group frame by using the corresponding indicator direction information for each successfully matched pair of envelope frames and light group frame identifier frames; as well as A module for obtaining the timing state of the same type of lamp group frame based on the type of lamp group frame and the timing state of each lamp group frame. The step of automatically learning the state information of each light group frame in the detected time-series image to obtain the time-series state of each corresponding light group frame in different frame images further includes: Track each of the light group frames in the detected time sequence image to obtain the one-to-one correspondence between each of the light group frames in different frame images, and then determine each of the same light group frames that correspond one-to-one in different frame images. The state information of each light group frame in the detected time sequence image is automatically learned to obtain the time sequence state of each light group frame.

7. A method for determining the current driving state at an intersection in autonomous driving, characterized in that, include: Using time-series images captured by an onboard camera, target detection is performed to obtain a current frame labeled image after the light group frames of each traffic light in the current frame image of the time-series image are labeled using light group frame labeling frames. The state information of each light group frame in the detected time sequence image is automatically learned to obtain the time sequence state of each same light group frame that corresponds one-to-one between different frame images. The current frame identifier image obtains the 3D position information of each of the light group frames and the indication direction information of each of the light group frames from the high-precision map; The 3D position information of each of the light group frames is projected into the current frame identifier image to obtain each envelope frame corresponding to each of the light group frames. Match each of the envelope boxes with each of the lamp group frame identifier boxes in the current frame identifier image; For each pair of successfully matched envelope frames and lamp group frame identifier frames, the type of the lamp group frame is determined using the corresponding indicator direction information of the corresponding lamp group frame; Based on the type of the lamp group frame and the timing state of each lamp group frame of the same type, the timing state of the same lamp group frame of the same type is obtained. as well as Based on the timing status of the same type of light group frames, the current traffic status of the intersection is obtained, and a decision is made on the driving status of the autonomous vehicle based on the traffic status; The step of automatically learning the state information of each light group frame in the detected time-series image to obtain the time-series state of each corresponding light group frame in different frame images further includes: Track each of the light group frames in the detected time sequence image to obtain the one-to-one correspondence between each of the light group frames in different frame images, and then determine each of the same light group frames that correspond one-to-one in different frame images. The state information of each light group frame in the detected time sequence image is automatically learned to obtain the time sequence state of each light group frame.

8. A computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are operated to perform the autonomous driving traffic light perception method based on high-precision map matching as described in any one of claims 1-5.

9. A computer device comprising a processor and a memory storing computer instructions, wherein the processor operates the computer instructions to perform the autonomous driving traffic light perception method based on high-precision map matching according to any one of claims 1-5.

Images