Query-based automatic driving traffic light detection method, system, medium and device

Abstract

The application provides a query-based automatic driving traffic light detection method, system, medium and equipment, the method comprises the following steps: step S1: constructing a query unit; step S2: a detection unit detects the current intersection traffic light in real time; step S3: the update unit updates the traffic light detection result by using the constructed query unit, and the update object is the traffic light of unknown type direction and the traffic light of type direction detection error. The application provides a method for optimizing traffic light detection by querying information, effectively reduces the probability of final result error caused by problems such as too far distance and detection model error, and the establishment and modification process of the query module is simple, only needs to mark the collected data simply, and only needs to replace the marking information of the corresponding intersection for modification, which greatly reduces the production and modification cost compared with high-precision map detection.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically, to a query-based method, system, medium, and device for detecting traffic lights in autonomous driving systems. Background Technology

[0002] For urban roads, traffic light recognition has always been a challenge in ensuring the safe and efficient operation of autonomous vehicles. Traffic light detection actually involves two tasks: traffic light detection and traffic light recognition. Many existing traffic light detection technologies use panoramic images seen by cameras for detection. This method has a high false detection rate and cannot be matched one-to-one with actual lanes, which may pose a fatal danger to autonomous driving.

[0003] The invention document with publication number CN113177522A discloses a method for traffic light detection and recognition in autonomous driving scenarios, including the following steps: (1) processing traffic light image data; (2) mapping the coordinates of the traffic light area on the high-precision map onto the traffic light image and obtaining the ROI of the traffic light area; (3) detecting the traffic light area and individual traffic lights; (4) determining whether the detected traffic lights are correct and cropping the individual traffic light area; (5) classifying the cropped individual traffic lights; (6) fusing the detection results and classification results, and obtaining the lane information corresponding to each traffic light through the high-precision map; (7) outputting the traffic light status of all lanes. This method mainly uses a high-precision map to obtain the specific spatial location information of the traffic lights that need to be focused on in the current lane of the current scene, and then projects them onto the image through a pre-calibrated projection matrix, and uses this location as a reference for traffic light detection. The production of high-precision maps is relatively complex, and traffic lights are associated with lanes. When obtaining the location of traffic lights, it is necessary to first determine the current lane. Finally, the obtained traffic light map coordinates and size are projected onto the image to extract the ROI. This process involves complex dependencies, a long map creation cycle, and significant modification difficulties.

[0004] Therefore, a new solution is needed to improve the above-mentioned technical problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a query-based method, system, medium, and device for detecting traffic lights in autonomous driving systems.

[0006] According to the present invention, an autonomous driving traffic light detection method based on query is provided, the method comprising the following steps:

[0007] Step S1: Construct query units;

[0008] Step S2: The detection unit detects the traffic lights at the current intersection in real time;

[0009] Step S3: The update unit uses the constructed query unit to update the traffic light detection results. The update objects are traffic lights with unknown type directions and traffic lights with incorrect type direction detection.

[0010] Preferably, step S1 includes the following steps:

[0011] Step S1.1: The data collection vehicle collects road information for a preset road segment. The road information includes the location information of each traffic light intersection and the corresponding image information. The data is collected at preset distance intervals. If the resolution of the image sensor is too high, the resolution is reduced and saved to reduce the disk space occupied by the map.

[0012] Step S1.2: Cluster the collected road information data according to location, and set the cluster center as the search index of the current intersection. Each index corresponds to multiple images.

[0013] Step S1.3: Manually annotate the collected image information, including the 2D frame, direction, and type of each traffic light in the current image, and whether it belongs to the current intersection;

[0014] Step S1.4: Traverse the labeling results of each intersection, count all traffic light directions and types, and save the results to the current intersection information.

[0015] Preferably, step S2 uses an image sensor to acquire an image and detects the traffic lights contained in the image. The detection content includes the position, color, direction, and type of the traffic lights. The detection method is traditional image detection or a deep learning model.

[0016] Preferably, step S3 includes the following steps:

[0017] Step S3.1: Use GPS to obtain the current vehicle location information, and use the query unit to search for the image sequence corresponding to the nearest neighbor location, as well as the direction and type stored in it;

[0018] Step S3.2: Use the currently acquired image to compare features with the images in the sequence queried in step S3.1, find the frame image most similar to the current image, and use a matching algorithm to pair the detected traffic lights with the queried traffic lights;

[0019] Step S3.3: Iterate through the traffic lights that were successfully paired in step S3.2, and directly assign the type and direction of the query light to the matched detection light.

[0020] The present invention also provides a query-based traffic light detection system for autonomous driving, the system comprising the following modules:

[0021] Module M1: Constructs query units;

[0022] Module M2: The detection unit detects the traffic lights at the current intersection in real time;

[0023] Module M3: The update unit uses the constructed query unit to update the traffic light detection results. The update objects are traffic lights with unknown type directions and traffic lights with incorrect type direction detection.

[0024] Preferably, module M1 includes the following modules:

[0025] Module M1.1: The data collection vehicle collects road information for a preset road segment. The road information includes the location information of each traffic light intersection and the corresponding image information. The data is collected at preset distance intervals. If the resolution of the image sensor is too high, the resolution is reduced and saved to reduce the disk space occupied by the map.

[0026] Module M1.2: Clusters the collected road information data by location, with the cluster center set as the search index of the current intersection, and each index corresponding to multiple images;

[0027] Module M1.3: Manually annotates the collected image information, including the 2D frame, direction, and type of each traffic light in the current image, and whether it belongs to the current intersection;

[0028] Module M1.4: Iterate through the labeling results of each intersection, count all traffic light directions and types, and save the results to the current intersection information.

[0029] Preferably, the module M2 uses an image sensor to acquire images and detects traffic lights contained in the images. The detection content includes the position, color, direction, and type of the traffic lights. The detection system is a traditional image detection system or uses a deep learning model.

[0030] Preferably, module M3 includes the following modules:

[0031] Module M3.1: Uses GPS to obtain the current vehicle location information, and uses a query unit to search for the image sequence corresponding to the nearest neighbor location, as well as the direction and type stored in it;

[0032] Module M3.2: Compares the features of the currently acquired image with the images in the sequence queried in Module M3.1, finds the frame image most similar to the current image, and uses a matching algorithm to pair the detected traffic lights with the queried traffic lights;

[0033] Module M3.3: Iterates through the traffic lights that were successfully paired in module M3.2, and directly assigns the type and direction of the query light to the matched detection light.

[0034] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the query-based autonomous driving traffic light detection method described above.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the query-based autonomous driving traffic light detection method described above.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] This invention proposes a method for optimizing traffic light detection by querying information, which effectively reduces the probability of incorrect final results due to problems such as excessive distance and detection model errors. Furthermore, the process of establishing and modifying the query module is simple. Establishment only requires simple annotation of the collected data, and modification only requires replacing the annotation information of the corresponding intersection. Compared with detection by high-precision maps, this greatly reduces the production and modification costs. Attached Figure Description

[0038] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0039] Figure 1 This is a schematic diagram of the main flow of the present invention;

[0040] Figure 2 This is a flowchart illustrating step S1 of the present invention.

[0041] Figure 3 This is a flowchart illustrating step S3 of the present invention.

[0042] Figure 4 This is a flowchart illustrating the process of updating traffic light detection results using the constructed query in the update module of this invention. Detailed Implementation

[0043] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0044] Example 1:

[0045] According to the present invention, an autonomous driving traffic light detection method based on query is provided, the method comprising the following steps:

[0046] Step S1: Construct query units;

[0047] Step S1.1: The data collection vehicle collects road information for a preset road segment. The road information includes the location information of each traffic light intersection and the corresponding image information. The data is collected at preset distance intervals. If the resolution of the image sensor is too high, the resolution is reduced and saved to reduce the disk space occupied by the map.

[0048] Step S1.2: Cluster the collected road information data according to location, and set the cluster center as the search index of the current intersection. Each index corresponds to multiple images.

[0049] Step S1.3: Manually annotate the collected image information, including the 2D frame, direction, and type of each traffic light in the current image, and whether it belongs to the current intersection;

[0050] Step S1.4: Traverse the labeling results of each intersection, count all traffic light directions and types, and save the results to the current intersection information.

[0051] Step S2: The detection unit detects the traffic lights at the current intersection in real time; it acquires images using an image sensor and detects the traffic lights contained in the images. The detection content includes the position, color, direction, and type of the traffic lights. The detection method is traditional image detection or the use of a deep learning model.

[0052] Step S3: The update unit uses the constructed query unit to update the traffic light detection results. The update objects are traffic lights with unknown type directions and traffic lights with incorrect type direction detection.

[0053] Step S3.1: Use GPS to obtain the current vehicle location information, and use the query unit to search for the image sequence corresponding to the nearest neighbor location, as well as the direction and type stored in it;

[0054] Step S3.2: Use the currently acquired image to compare features with the images in the sequence queried in step S3.1, find the frame image most similar to the current image, and use a matching algorithm to pair the detected traffic lights with the queried traffic lights;

[0055] Step S3.3: Iterate through the traffic lights that were successfully paired in step S3.2, and directly assign the type and direction of the query light to the matched detection light.

[0056] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the query-based autonomous driving traffic light detection method described above.

[0057] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the query-based autonomous driving traffic light detection method described above.

[0058] The present invention also provides a query-based autonomous driving traffic light detection system, which can be implemented by executing the process steps of the query-based autonomous driving traffic light detection method. That is, those skilled in the art can understand the query-based autonomous driving traffic light detection method as a preferred embodiment of the query-based autonomous driving traffic light detection system.

[0059] Example 2:

[0060] The present invention also provides a query-based traffic light detection system for autonomous driving, the system comprising the following modules:

[0061] Module M1: Constructs query units;

[0062] Module M1.1: The data collection vehicle collects road information for a preset road segment. The road information includes the location information of each traffic light intersection and the corresponding image information. The data is collected at preset distance intervals. If the resolution of the image sensor is too high, the resolution is reduced and saved to reduce the disk space occupied by the map.

[0063] Module M1.2: Clusters the collected road information data by location, with the cluster center set as the search index of the current intersection, and each index corresponding to multiple images;

[0064] Module M1.3: Manually annotates the collected image information, including the 2D frame, direction, and type of each traffic light in the current image, and whether it belongs to the current intersection;

[0065] Module M1.4: Iterate through the labeling results of each intersection, count all traffic light directions and types, and save the results to the current intersection information.

[0066] Module M2: The detection unit detects the traffic lights at the current intersection in real time; it uses an image sensor to acquire images and detects the traffic lights contained in the images. The detection content includes the position, color, direction and type of the traffic lights. The detection system can be a traditional image detection system or a deep learning model.

[0067] Module M3: The update unit uses the constructed query unit to update the traffic light detection results. The update objects are traffic lights with unknown type directions and traffic lights with incorrect type direction detection.

[0068] Module M3.1: Uses GPS to obtain the current vehicle location information, and uses a query unit to search for the image sequence corresponding to the nearest neighbor location, as well as the direction and type stored in it;

[0069] Module M3.2: Compares the features of the currently acquired image with the images in the sequence queried in Module M3.1, finds the frame image most similar to the current image, and uses a matching algorithm to pair the detected traffic lights with the queried traffic lights;

[0070] Module M3.3: Iterates through the traffic lights that were successfully paired in module M3.2, and directly assigns the type and direction of the query light to the matched detection light.

[0071] Example 3:

[0072] For traffic light detection in autonomous driving, due to the characteristics of camera sensor imaging, traffic lights appear smaller in images the farther away they are. In such cases, only color is relatively easy to obtain, while direction and type (bicycle lights, pedestrian lights, vehicle lights, etc.) are difficult to detect. Therefore, it is crucial to pre-determine which types of lights are present at the current intersection, which can solve the problem of not being able to determine whether the current direction is passable in some situations. This invention proposes a pre-established intersection query module and an adaptive query result update method, which can effectively improve the accuracy of traffic light detection.

[0073] This invention proposes a query-based traffic light detection method for autonomous driving, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0074] The query module is constructed as follows:

[0075] The data collection vehicle gathers road information for preset road sections, including the location information and corresponding image information of each traffic light intersection. Data can be collected at regular intervals (e.g., saving information every 10 meters). If the image sensor resolution is too high, the resolution can be reduced to save the image and decrease the disk space occupied by the map. Furthermore, to minimize the difference between the image used and the collected image, data can be collected multiple times at different time intervals.

[0076] The collected data is clustered by location, and the cluster center is set as the search index of the current intersection. Each index corresponds to multiple images.

[0077] The collected image information is manually labeled, including the 2D frame, direction, type, and whether it belongs to the current intersection for each traffic light in the current image (used to exclude interfering targets from other intersections, often appearing in two very close intersections).

[0078] Iterate through the labeling results for each intersection, and count the directions and types of all traffic lights that appear. Save the results to the current intersection information.

[0079] The detection module monitors the traffic lights at the current intersection in real time.

[0080] This process utilizes images acquired by an image sensor to detect traffic lights, identifying their position, color, direction, and type. Detection methods can range from traditional image detection techniques, such as template matching, to deep learning models, such as YOLO.

[0081] The update module updates traffic light detection results using the constructed query. The main update targets are traffic lights with unknown direction types and those with incorrect direction type detection. The specific process is as follows: Figure 4 .

[0082] The system uses GPS to obtain the current vehicle location information and uses a query module to search for the image sequence corresponding to the nearest neighbor location, as well as the direction and type stored in it.

[0083] The currently acquired image is compared with the images in the query sequence sequentially based on their features to find the frame most similar to the current image. Then, a matching algorithm is used to pair the detected traffic lights with the queried traffic lights.

[0084] Iterate through the traffic lights that have been successfully matched in the detection module, and directly assign the type and direction of the query light to the matched detection light.

[0085] This invention uses the intersection information annotations established by the query module to perform traversal image detection and updates. The establishment and modification process of the query module is simple. Establishment only requires annotating the collected data, and modification only requires replacing the annotation information of the corresponding intersection, which reduces the cost of traffic light detection for autonomous driving.

[0086] Those skilled in the art can understand this embodiment as a more specific description of Embodiment 1 and Embodiment 2.

[0087] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, enabling the system and its various devices, modules, and units to function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

[0088] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A query-based traffic light detection method for autonomous driving, characterized in that, The method includes the following steps: Step S1: Construct query units; Step S2: The detection unit detects the traffic lights at the current intersection in real time; Step S3: The update unit uses the constructed query unit to update the traffic light detection results. The update objects are traffic lights with unknown type directions and traffic lights with incorrect type direction detection. Step S1 includes the following steps: Step S1.1: The data collection vehicle collects road information for a preset road segment. The road information includes the location information of each traffic light intersection and the corresponding image information. The data is collected at preset distance intervals. If the resolution of the image sensor is too high, the resolution is reduced and saved to reduce the disk space occupied by the map. Step S1.2: Cluster the collected road information data according to location, and set the cluster center as the search index of the current intersection. Each index corresponds to multiple images. Step S1.3: Manually annotate the collected image information, including the 2D frame, direction, and type of each traffic light in the current image, and whether it belongs to the current intersection; Step S1.4: Traverse the labeling results of each intersection, count all traffic light directions and types, and save the results to the current intersection information; Step S3 includes the following steps: Step S3.1: Use GPS to obtain the current vehicle location information, and use the query unit to search for the image sequence corresponding to the nearest neighbor location, as well as the direction and type stored in it; Step S3.2: Use the currently acquired image to compare features with the images in the sequence queried in step S3.1, find the frame image most similar to the current image, and use the matching algorithm to pair the detected traffic lights with the queried traffic lights; Step S3.3: Iterate through the traffic lights that were successfully paired in step S3.2, and directly assign the type and direction of the query light to the matched detection light.

2. The query-based traffic light detection method for autonomous driving according to claim 1, characterized in that, Step S2 uses an image sensor to acquire an image and detects the traffic lights contained in the image. The detection content includes the position, color, direction and type of the traffic lights. The detection method is traditional image detection or a deep learning model.

3. A query-based traffic light detection system for autonomous driving, characterized in that, The system includes the following modules: Module M1: Constructs query units; Module M2: The detection unit detects the traffic lights at the current intersection in real time; Module M3: The update unit uses the constructed query unit to update the traffic light detection results. The update objects are traffic lights with unknown type directions and traffic lights with incorrect type direction detection. Module M1 includes the following modules: Module M1.1: The data collection vehicle collects road information for a preset road segment. The road information includes the location information of each traffic light intersection and the corresponding image information. The data is collected at preset distance intervals. If the resolution of the image sensor is too high, the resolution is reduced and saved to reduce the disk space occupied by the map. Module M1.2: Clusters the collected road information data by location, with the cluster center set as the search index of the current intersection, and each index corresponding to multiple images; Module M1.3: Manually annotates the collected image information, including the 2D frame, direction, and type of each traffic light in the current image, and whether it belongs to the current intersection; Module M1.4: Iterate through the labeling results of each intersection, count all traffic light directions and types, and save the results to the current intersection information; Module M3 includes the following modules: Module M3.1: Uses GPS to obtain the current vehicle location information, and uses a query unit to search for the image sequence corresponding to the nearest neighbor location, as well as the direction and type stored in it; Module M3.2: Compares the features of the currently acquired image with the images in the sequence queried in Module M3.1, finds the frame image most similar to the current image, and uses a matching algorithm to pair the detected traffic lights with the queried traffic lights; Module M3.3: Iterates through the traffic lights that were successfully paired in module M3.2, and directly assigns the type and direction of the queried light to the matched detection light.

4. The query-based autonomous driving traffic light detection system according to claim 3, characterized in that, The module M2 uses an image sensor to acquire images and detects traffic lights contained in the images. The detection content includes the position, color, direction and type of the traffic lights. The detection system can be a traditional image detection system or a deep learning model.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the query-based traffic light detection method for autonomous driving as described in any one of claims 1 to 2.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the steps of the query-based traffic light detection method for autonomous driving as described in any one of claims 1 to 2.

Images