Traffic light perception method and device based on historical frame memory automatic learning
By using an automatic learning method based on historical frame memory, traffic light images are acquired and tracked in real time, and then matched with high-precision maps. This solves the problems of accuracy and real-time performance in traffic light perception in traditional autonomous driving, enabling autonomous vehicles to make accurate decisions at intersections.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- MOMENTA (SUZHOU) TECHNOLOGY CO LTD
- Filing Date
- 2021-12-29
- Publication Date
- 2026-07-03
AI Technical Summary
Traditional traffic light perception solutions for autonomous driving have poor recognition accuracy and real-time performance, and cannot accurately match lanes, affecting the vehicle's intersection decision-making during autonomous driving.
By acquiring real-time images from vehicle-mounted cameras, the system detects and tracks traffic light frames. Using a historical frame memory automatic learning method, it establishes a one-to-one correspondence between light frames in different frame images. This correspondence is then combined with a high-precision map for matching, enabling real-time perception and decision-making regarding traffic lights.
It improves the accuracy and real-time performance of traffic light perception, ensuring that autonomous vehicles make correct decisions at intersections and enhancing the safety and reliability of autonomous driving.
Smart Images
Figure CN116416273B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving, and in particular to a traffic light perception method and device based on automatic learning from historical frame memory. Background Technology
[0002] With the rise of artificial intelligence, autonomous driving has become a hot technology in the automotive industry. Just as humans pay attention to traffic light information while driving, autonomous driving relies even more heavily on traffic light information for vehicles to better interact with the road. Therefore, designing a reliable traffic light perception system is crucial.
[0003] Traditional traffic light perception solutions in autonomous driving primarily rely on color recognition of images taken of traffic lights to determine which light is currently on. However, this method suffers from poor accuracy and real-time performance, and it also fails to accurately match traffic lights with actual lanes, impacting the vehicle's decision-making at intersections during autonomous driving. Summary of the Invention
[0004] To address the issues of poor accuracy and real-time performance in existing traffic light recognition technologies, this application provides a traffic light sensing method and apparatus based on automatic learning from historical frame memory.
[0005] One technical solution adopted in this application is: providing a traffic light perception method based on automatic learning from historical frame memory, which includes:
[0006] Real-time acquisition of time-series images captured by vehicle-mounted cameras;
[0007] Real-time detection of the light group frames of each traffic light in the current frame of the time-series image;
[0008] The state of each light group frame in the detected current frame image is judged in real time to obtain the state information of each light group frame in the current frame image. The state information of each light group frame in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on.
[0009] Real-time tracking of each light group frame in the detected current frame image is performed to obtain the one-to-one correspondence between each light group frame in different frame images, thereby determining the individual light group frames that correspond one-to-one in different frame images; and
[0010] By utilizing the state information of each of the same light group frames in a preset number of historical frames adjacent to the current frame image, as well as the state information of the corresponding light group frames in the current frame image, real-time automatic learning is performed to obtain the current temporal state of each of the same light group frames.
[0011] Another technical solution adopted in this application is: providing a traffic light sensing device based on automatic learning from historical frame memory, which includes:
[0012] Module for real-time acquisition of time-series images captured by vehicle-mounted cameras;
[0013] This module is used for real-time detection of the light group frames of each traffic light in the current frame of a time-series image.
[0014] This module is used to determine the state of each light group frame in the detected current frame image in real time and obtain the state information of each light group frame in the current frame image. The state information of each light group frame in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on.
[0015] This module is used to track each light group frame in the detected current frame image in real time, so as to obtain the one-to-one correspondence between each light group frame in different frame images, and then determine the corresponding light group frames in different frame images.
[0016] This module is used to automatically learn in real time, using the state information of each of the same light group frames in a preset number of historical frames adjacent to the current frame image, as well as the state information of the corresponding light group frames in the current frame image, to obtain the current time sequence state of each of the same light group frames.
[0017] Another technical solution adopted in this application is: providing a method for determining the current intersection driving state in autonomous driving, which includes:
[0018] Real-time acquisition of time-series images captured by vehicle-mounted cameras;
[0019] Real-time detection of the light group frames of each traffic light in the current frame of the time sequence image, and marking the light group frame identifier frame at the corresponding position on the time sequence image;
[0020] The state of each light group frame in the detected current frame image is judged in real time to obtain the state information of each light group frame in the current frame image. The state information of each light group frame in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on.
[0021] Real-time tracking of each light group frame in the detected current frame image is performed to obtain the one-to-one correspondence between each light group frame in different frame images, and then the corresponding light group frames in different frame images are determined.
[0022] By utilizing the state information of each of the same light group frames in the preset number of historical frames adjacent to the current frame image, as well as the state information of the corresponding light group frames in the current frame image, real-time automatic learning is performed to obtain the current temporal state of each of the same light group frames.
[0023] The directional information of the light group frame identifiers in the current frame image, matched with the light group frames in the high-precision map, is loaded into the status information of the corresponding light group frames to obtain the current traffic status of the intersection; and
[0024] Decisions are made regarding the driving status of autonomous vehicles based on traffic conditions.
[0025] Another technical solution adopted in this application is to provide a computer-readable storage medium storing computer instructions that are operated to execute the traffic light perception method based on historical frame memory automatic learning in Solution 1.
[0026] Another technical solution adopted in this application is: providing a computer device, which includes a processor and a memory, the memory storing computer instructions, which are operated to execute the traffic light perception method based on historical frame memory automatic learning in Solution 1.
[0027] The beneficial effects of the technical solution of this application are as follows: This application designs a traffic light perception method and device based on historical frame memory and automatic learning. This method realizes the perception result output of autonomous vehicles for traffic lights by memorizing the results of historical frame images and automatically learning the results of the current frame image, and makes correct decisions when passing through traffic light intersections, with real-time performance and high accuracy. Attached Figure Description
[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0029] Figure 1 This is a schematic diagram of a specific implementation of a traffic light perception method based on automatic learning of historical frame memory according to this application;
[0030] Figure 2 This is a schematic diagram of a specific implementation of a traffic light sensing device based on automatic learning from historical frame memory, as described in this application.
[0031] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0032] The preferred embodiments of this application will now be described in detail with reference to the accompanying drawings, so that the advantages and features of this application can be more easily understood by those skilled in the art, thereby providing a clearer and more definite definition of the scope of protection of this application.
[0033] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.
[0034] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0035] Figure 1 This illustration shows a specific implementation of a traffic light perception method based on historical frame memory automatic learning according to this application. Figure 1 In the specific implementation shown, the traffic light perception method based on automatic learning from historical frame memory includes:
[0036] Step S101: Acquire time-series images captured by the vehicle-mounted camera in real time.
[0037] In this embodiment, within the field of autonomous driving, a target detection model is used to detect and identify light group bounding boxes, facilitating target tracking in subsequent time-series images of the current frame. The number of current light group bounding boxes corresponds to the number of current light group bounding box identifiers in the current time-series image, making it easy to record each light group bounding box.
[0038] exist Figure 1In the specific implementation shown, the traffic light perception method based on automatic learning from historical frame memory further includes:
[0039] Step S102: Real-time detection of the light group frames of each group of traffic lights in the current frame image of the time sequence image.
[0040] In this embodiment, the light group frame in the detection time sequence image is implemented to prevent the target from being lost, which would reduce the accuracy of traffic light perception.
[0041] In an optional embodiment of this application, real-time detection of the light group frames of each group of traffic lights in the current frame image of the time sequence image includes: marking a light group frame identifier frame at a corresponding position on the time sequence image to indicate the light group frame.
[0042] In this embodiment, the light group frame identifier has a corresponding ID, which can prevent the target from being lost.
[0043] exist Figure 1 In the specific implementation shown, the traffic light perception method based on automatic learning of historical frame memory further includes: step S103, which involves real-time judgment of the state of each light group frame in the detected current frame image to obtain the state information of each light group frame in the current frame image, wherein the state information of each light group frame in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on.
[0044] In this embodiment, the traffic light status of the light group frame in the current frame is judged in real time to obtain the status information corresponding to the light group frame in the current frame. Based on the ID corresponding to the light group frame identifier, the status information corresponding to the current light group frame in the current frame time sequence image is recorded. The status information includes the color information and on / off information corresponding to the current light group frame. This not only obtains important information for traffic light perception, but also prevents target loss.
[0045] It's necessary to determine the color and on / off information of the traffic lights within each traffic light group frame. Specifically, this involves identifying the positions of the red, green, and yellow lights within the frame, as well as their respective colors and on / off states. For example, a red light within a frame might have both red on and off states, and a green light might have both green on and off states. In practice, due to LED delays in the traffic lights or camera exposure times, it's possible for two different colored lights within the same frame to briefly illuminate simultaneously. Furthermore, three different colored lights within a single frame generally won't illuminate simultaneously. For instance, this might occur when red is off, yellow is off, and green is on while traffic is flowing, or when the yellow light is flashing, all lights might be off or yellow might be on.
[0046] exist Figure 1 In the specific implementation shown, the traffic light perception method based on automatic learning from historical frame memory further includes:
[0047] Step S104: Real-time tracking of each light group frame in the detected current frame image is performed to obtain the one-to-one correspondence between each light group frame in different frame images, thereby determining the corresponding light group frames in different frame images.
[0048] In this embodiment, the light group frames are correlated between different frame images. The purpose of real-time tracking is to combine the state information of the correlated light group frames together in order to obtain the timing state of the corresponding light group frames.
[0049] In one optional embodiment of this application, each light group frame in the detected current frame image is tracked in real time to obtain the one-to-one correspondence between each light group frame in different frame images, and then the corresponding light group frames in different frame images are determined. This includes: calculating the intersection-union ratio between the light group frame identifier frame of each light group frame in the marked current frame image and the light group frame identifier frame of each light group frame in the previous frame image. When the intersection-union ratio is greater than a predetermined overlap threshold, the light group frame corresponding to the corresponding light group frame identifier frame is determined to be the same light group frame.
[0050] In this embodiment, determining the same light group frame not only provides a basis for traffic light status perception, but also prevents target loss.
[0051] In one example of this application, the intersection-union ratio (IoU) of the current light group frame identifier in the current frame temporal image and the previous light group frame identifier in the previous frame temporal image is calculated and obtained. When the IoU is greater than a predetermined overlap threshold, it is determined that the current light group frame identifier is associated with the previous light group frame identifier, and the same light group frame belonging to the same traffic light target in the current frame temporal image and the previous frame temporal image is determined. To prevent target loss in single-frame temporal image detection, the same light group frame belonging to the same traffic light target is determined. When each frame temporal image arrives, it is tracked with the previous frame temporal image to ensure that the target is tracked in each frame.
[0052] In this example, the correlation between time-series images is determined based on the Intersection over Union (IOU). IOU is the ratio of the area of the intersection (intersection) of two light group frame identifiers in two consecutive time-series images to the area of their union. A stronger correlation between two time-series images indicates a greater overlap in area and number of identifiers between the current and previous light group frames.
[0053] In one example of this application, the ID information of the light group frame identifier can also be used for tracking. In reality, the light group frame detected in the previous time sequence image is located at one position in the world coordinate system, and the light group frame detected in the current time sequence image is located at another position in the world coordinate system. Therefore, it is necessary to determine whether the light group frames at these two positions represent the same traffic light. This can be achieved by using the ID information of the light group frame identifier to track the target, which can ensure that the target is not lost in the single-frame time sequence image detection.
[0054] In one example of this application, the state information of the same light group frame is arranged in a time sequence to obtain a corresponding state vector. In each frame of the time sequence image, the state information of the same light group frame is represented by one numerical value, and the state vector is represented by one or two numerical values. In a frame of the time sequence image, each type of the same light group frame exists in only one color on / off state, represented by four numerical values: red, yellow, green, and off, represented by 1, 2, 3, and 4 respectively. The state vector is used to determine the time sequence state of the traffic lights. Since the time sequence state includes red, green, yellow, flashing green, flashing yellow, or off, the state vector contains only one or two numerical values. Using numerical values to represent the state information and the state vector makes the state of the traffic lights easy to identify.
[0055] exist Figure 1 In the specific implementation shown, the traffic light perception method based on automatic learning from historical frame memory further includes:
[0056] Step S105: Using the state information of each of the same light group frames in the preset number of historical frames adjacent to the current frame image, and the state information of the corresponding light group frames in the current frame image, real-time automatic learning is performed to obtain the current temporal state of each of the same light group frames.
[0057] In this embodiment, the temporal state information of the same light group frame is obtained from a preset number of historical frame images and the current frame image, and the current temporal state of the same light group frame is obtained through learning. Based on the state information in the preset number of historical frame images, only the state information in the current frame image needs to be added, making the recognition of the temporal state of the same light group frame real-time and efficient.
[0058] In one optional embodiment of this application, the preset number is the number of frames of time-series images captured by the vehicle-mounted camera within a continuous time period of 1.5 seconds to 2 seconds.
[0059] In this embodiment, the time-series image is an image acquired in chronological order. The acquisition frequency, measured in Hertz, is converted to a value in milliseconds per frame or seconds per frame. The preset number is equal to a continuous time interval of 1.5 to 2 seconds divided by the value in milliseconds per frame or seconds per frame.
[0060] This application automatically learns from the state information of historical and current frame images to train a temporal state prediction model, which is used to predict the temporal state of light group frames. The temporal state prediction model is a long short-term memory network model; its long-term memory can remember information from dozens of temporal images, while its short-term memory can remember information from a few frames. The preset number can range from 30 to 40, or it can be determined based on the number of frames in which the temporal state prediction model accurately identifies the temporal state of the light group frame during actual training. The temporal state prediction model does not impose specific requirements on the preset number. This application can improve the real-time performance and accuracy of the solution.
[0061] In one optional embodiment of this application, the state information of each light group frame in the current frame image is stored and memorized in real time, and the state information of each of the same light group frames in the preset number of historical frame images adjacent to the current frame image is obtained.
[0062] In this embodiment, the storage capacity is limited and can only store the state information of a preset number of historical frame images that are adjacent to the current frame image.
[0063] In an optional embodiment of this application, the indicator direction information of the light group frame identifier frame of each light group frame in the current frame image and the light group frame whose position matches the high-precision map is loaded onto the status information of the light group frame corresponding to the light group frame identifier frame to obtain the current traffic status of the intersection.
[0064] In this embodiment, by combining the traffic light information in the real world from the high-precision map with the traffic lights in the current frame time sequence image, the state of different types of traffic lights in different time sequences can be obtained, which facilitates vehicles to make decisions at intersections and improves accuracy.
[0065] In one example of this application, the temporal state prediction model stores state information corresponding to a preset number of time-series images. When the cumulative number of state information entries for the same light group frame in the temporal state prediction model exceeds the preset number, the model stores the state information corresponding to the latest time-series image and releases the previously entered state information sequentially according to the order of input time. The long short-term memory of the temporal state prediction model can process large amounts of data quickly in real time.
[0066] In a specific example of this application, assuming the preset number is N, and the time-series state prediction model already stores the state information of N frames of time-series images, then when the state information of the (N+1)th frame of the sequence image is input and stored, the state information of the first frame of the sequence image is released at the same time; when the state information of the Mth frame of the sequence image is input and stored, the state information of the MNth frame of the sequence image is released at the same time.
[0067] In one example of this application, a certain amount of data can be used to train a temporal state prediction model, enabling the model to accurately predict the temporal state of traffic lights. Using an object detection model, the traffic light bounding boxes in N consecutive frames of temporal images captured by an onboard camera are detected, and these boxes are identified using bounding box identifiers, where N is a natural number not less than 2. The traffic light state of each bounding box is determined, obtaining the color and on / off information of each light within the bounding box. The state information of the bounding box in each frame is obtained based on its ID. The bounding boxes in the current frame of the N consecutive frames are tracked against those in the previous frame to identify the same bounding box belonging to the same target. The state information of the same bounding box in the N consecutive frames is used to train the temporal state prediction model, resulting in a well-trained model.
[0068] In this example, a tracking algorithm is used to map the same light group frame in N consecutive time-series images to the light group frame in the real world, and the state information corresponding to the same light group frame in N consecutive time-series images is organized into a state vector. The state vector is then input into the state prediction model for prediction and memory training, and finally the time-series state prediction model is obtained.
[0069] It should be noted that the tracking algorithm is the same as the tracking algorithm. By tracking the target position in N consecutive frames, a short trajectory tracklet is obtained, and then the correlation between two time-series images is found.
[0070] In one example of this application, the state information in the state vector is input into the state prediction model in batches to obtain the number of temporal images required for the state prediction model to identify the temporal state of the light group frame. The state information in the state vector can be input into the state prediction model frame by frame or several consecutive frames at once. The state prediction model can predict the number of temporal states of the light group frame that need to be determined based on the actual training situation.
[0071] This application achieves high real-time performance by processing the current frame image in real time, memorizing the state information of the current frame image, calculating the correlation degree of the state information of the same light group frame, and thus obtaining the temporal state of the same light group frame. By memorizing the results of historical frame images and automatically learning the results of the current frame image, the autonomous vehicle can output the perception results of traffic lights and make correct decisions when passing through traffic light intersections, exhibiting both real-time performance and high accuracy.
[0072] Figure 2 This illustration shows a specific implementation of a traffic light sensing device based on historical frame memory automatic learning, as described in this application. Figure 2In the specific implementation shown, the traffic light sensing device based on automatic learning from historical frame memory includes:
[0073] Module 201 is a module used to acquire time-series images captured by the vehicle-mounted camera in real time;
[0074] Module 202 is used to detect the light group frames of each group of traffic lights in the current frame image of the time sequence image in real time;
[0075] Module 203 is used to make real-time judgments on the state of each light group frame in the detected current frame image and obtain the state information of each light group frame in the current frame image. The state information of each light group frame in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on.
[0076] Module 204 is used to track each light group frame in the detected current frame image in real time, so as to obtain the one-to-one correspondence between each light group frame in different frame images, and then determine the module of each same light group frame that corresponds one-to-one in different frame images.
[0077] Module 205 is used to perform real-time automatic learning by utilizing the state information of each of the same light group frames in a preset number of historical frame images adjacent to the current frame image, as well as the state information of the corresponding light group frames in the current frame image, to obtain the current temporal state of each of the same light group frames.
[0078] In this embodiment, the current frame image is detected, judged, and tracked in real time, laying a solid foundation for subsequent processing; the timing state of traffic lights in the time sequence image is predicted, which has high real-time performance and accuracy in vehicle autonomous driving.
[0079] The traffic light sensing device based on automatic learning of historical frame memory provided in this application can be used to execute the traffic light sensing method based on automatic learning of historical frame memory described in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.
[0080] In one specific embodiment of this application, the functional modules of the traffic light sensing device based on historical frame memory automatic learning can be directly in hardware, in software modules executed by a processor, or in a combination of both.
[0081] Software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in this art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium.
[0082] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor can be a microprocessor, but alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration. Alternatively, the storage medium can be integrated with the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in the user terminal. Alternatively, the processor and storage medium can reside as discrete components in the user terminal.
[0083] In another specific embodiment of this application, a method for determining the current intersection driving state in autonomous driving includes:
[0084] Real-time acquisition of time-series images captured by vehicle-mounted cameras;
[0085] Real-time detection of the light group frames of each traffic light in the current frame of the time sequence image, and marking the light group frame identifier frame at the corresponding position on the time sequence image;
[0086] The state of each light group frame in the detected current frame image is judged in real time to obtain the state information of each light group frame in the current frame image. The state information of each light group frame in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on.
[0087] Real-time tracking of each light group frame in the detected current frame image is performed to obtain the one-to-one correspondence between each light group frame in different frame images, and then the corresponding light group frames in different frame images are determined.
[0088] By utilizing the state information of each of the same light group frames in the preset number of historical frames adjacent to the current frame image, as well as the state information of the corresponding light group frames in the current frame image, real-time automatic learning is performed to obtain the current temporal state of each of the same light group frames.
[0089] The directional information of the light group frame identifiers in the current frame image, matched with the light group frames in the high-precision map, is loaded into the status information of the corresponding light group frames to obtain the current traffic status of the intersection; and
[0090] Decisions are made regarding the driving status of autonomous vehicles based on traffic conditions.
[0091] In this embodiment, the vehicle's current driving status refers to determining, based on the automatically planned path, which direction the autonomous vehicle needs to travel at the intersection. If a left turn is required, the timing of the left turn signal is used to determine the autonomous vehicle's passage status, facilitating decision-making and improving the safety of autonomous driving.
[0092] In another specific embodiment of this application, a computer-readable storage medium is provided, which stores computer instructions that are operated to perform the traffic light perception method based on automatic learning of historical frame memory in any embodiment.
[0093] In another specific embodiment of this application, a computer device includes a processor and a memory, the memory storing computer instructions that are operated to perform the traffic light perception method based on historical frame memory automatic learning in any embodiment.
[0094] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0095] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0096] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A traffic light perception method based on automatic learning from historical frame memory, characterized in that, include: Real-time acquisition of time-series images captured by vehicle-mounted cameras; Real-time detection of the light group frames of each group of traffic lights in the current frame image of the time sequence image, wherein the real-time detection of the light group frames of each group of traffic lights in the current frame image of the time sequence image includes: marking the light group frame identification frame at the corresponding position on the time sequence image to represent the light group frame. The state of each light group frame in the detected current frame image is judged in real time to obtain the state information of each light group frame in the current frame image. The state information of each light group frame in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on. Real-time tracking of each light group frame in the detected current frame image is performed to obtain a one-to-one correspondence between each light group frame in different frame images, thereby determining the corresponding identical light group frames in different frame images. The real-time tracking of each light group frame in the detected current frame image to obtain a one-to-one correspondence between each light group frame in different frame images, thereby determining the corresponding identical light group frames in different frame images, includes: calculating the intersection-over-union ratio (IoU) between the light group frame identifier frames of each light group frame marked in the current frame image and the light group frame identifier frames of each light group frame in the previous frame image; when the IoU is greater than a predetermined overlap threshold, the light group frame corresponding to the corresponding light group frame identifier frame is determined to be the same light group frame; and By using the state information of each of the same light group frames in a preset number of historical frames adjacent to the current frame image, and the state information of the corresponding light group frames in the current frame image, real-time automatic learning is performed to obtain the current temporal state of each of the same light group frames.
2. The traffic light perception method based on historical frame memory automatic learning as described in claim 1, characterized in that, Also includes: The state information of each light group frame in the current frame image is stored and remembered in real time, and the state information of each of the same light group frames in the historical frame images adjacent to the current frame image is obtained.
3. The traffic light perception method based on historical frame memory automatic learning as described in claim 1, characterized in that, The preset number is the number of frames of the time-series image captured by the vehicle-mounted camera within a continuous time period of 1.5 seconds to 2 seconds.
4. The traffic light perception method based on historical frame memory automatic learning as described in claim 1, characterized in that, Also includes: The indicator direction information of the light group frame identifier frame of each light group frame in the current frame image and the light group frame whose position is matched in the high-precision map are loaded into the status information of the light group frame corresponding to the light group frame identifier frame to obtain the current traffic status of the intersection.
5. A traffic light sensing device based on automatic learning from historical frame memory, characterized in that, include: Module for real-time acquisition of time-series images captured by vehicle-mounted cameras; A module for real-time detection of the light group frames of each group of traffic lights in the current frame image of the time sequence image, wherein the real-time detection of the light group frames of each group of traffic lights in the current frame image of the time sequence image includes: marking a light group frame identifier frame at a corresponding position on the time sequence image to indicate the light group frame. A module for real-time determination of the state of each of the light group frames in the detected current frame image, and obtaining the state information of each of the light group frames in the current frame image, wherein the state information of each of the light group frames in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on. The module is used to perform real-time tracking of each of the light group frames in the detected current frame image to obtain a one-to-one correspondence between each of the light group frames in different frame images, and then determine each of the same light group frames that correspond one-to-one in different frame images. The step of performing real-time tracking of each of the light group frames in the detected current frame image to obtain a one-to-one correspondence between each of the light group frames in different frame images and then determining each of the same light group frames that correspond one-to-one in different frame images includes: calculating the intersection-union ratio between the light group frame identifier frame of each of the light group frames marked in the current frame image and the light group frame identifier frame of each of the light group frames in the previous frame image. When the intersection-union ratio is greater than a predetermined overlap threshold, the light group frame corresponding to the corresponding light group frame identifier frame is determined to be the same light group frame. A module for automatically learning in real time, using the state information of each of the same light group frames in a preset number of historical frames adjacent to the current frame image, and the state information of the corresponding light group frames in the current frame image, to obtain the current temporal state of each of the same light group frames.
6. A method for determining the current intersection driving state in autonomous driving, characterized in that, include: Real-time acquisition of time-series images captured by vehicle-mounted cameras; Real-time detection of the light group frames of each group of traffic lights in the current frame image of the time sequence image, and marking the light group frame identification frame at the corresponding position on the time sequence image, wherein the real-time detection of the light group frames of each group of traffic lights in the current frame image of the time sequence image includes: marking the light group frame identification frame at the corresponding position on the time sequence image. The state of each light group frame in the detected current frame image is judged in real time to obtain the state information of each light group frame in the current frame image. The state information of each light group frame in the current frame image includes whether the light group frame is on or off, and the color when the light group frame is on. Real-time tracking of each light group frame in the detected current frame image is performed to obtain a one-to-one correspondence between each light group frame in different frame images, thereby determining each corresponding identical light group frame in different frame images. The real-time tracking of each light group frame in the detected current frame image to obtain a one-to-one correspondence between each light group frame in different frame images, thereby determining each corresponding identical light group frame in different frame images, includes: calculating the intersection-union ratio (IUGR) between the light group frame identifier frames of each light group frame in the marked current frame image and the light group frame identifier frames of each light group frame in the previous frame image. When the IUGR is greater than a predetermined overlap threshold, the light group frame corresponding to the corresponding light group frame identifier frame is determined to be the identical light group frame. By utilizing the state information of each of the same light group frames in a preset number of historical frames adjacent to the current frame image, as well as the state information of the corresponding light group frames in the current frame image, real-time automatic learning is performed to obtain the current temporal state of each of the same light group frames. The direction information of the light group frame identifiers of each light group frame in the current frame image, matched with the light group frames in the high-precision map, is loaded onto the status information of the light group frame corresponding to the light group frame identifier to obtain the current traffic status of the intersection; and Based on the traffic conditions, decisions are made regarding the driving status of the autonomous vehicles.
7. A computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are operated to perform the traffic light perception method based on automatic learning of historical frame memory as described in any one of claims 1-4.
8. A computer device comprising a processor and a memory storing computer instructions, wherein the processor operates the computer instructions to perform the traffic light perception method based on historical frame memory automatic learning according to any one of claims 1-4.