Lane line identification method and device based on time sequence aerial view, non-temporary computer readable storage medium and vehicle

Abstract

The invention discloses a lane line recognition method and device based on a time sequence aerial view, a non-temporary computer readable storage medium and a vehicle, and the method comprises the steps: obtaining an image in a preset range in front of the vehicle, and carrying out the feature extraction of the image, and obtaining the features of a plurality of frames of images; storing the extracted multi-frame image features into a preset image feature storage queue according to a time sequence; respectively obtaining bird's-eye view features of a historical frame and bird's-eye view features of a current frame in the preset image feature storage queue; fusing the bird's-eye view feature of the historical frame and the bird's-eye view feature of the current frame to obtain an updated bird's-eye view feature of the historical frame; and carrying out lane line identification by adopting the updated aerial view features of the historical frames. According to the method, the aerial view features with the time sequence information are introduced to update the aerial view features of the historical frames, and powerful support is provided for improving the prediction performance under the condition that the lane lines are shielded.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology, and specifically relates to a lane line recognition method, device, non-temporary computer-readable storage medium and vehicle based on a time-series bird's-eye view. Background Art

[0002] In autonomous driving scenarios, using deep learning to estimate lane lines using 2D images provided by a single or multiple cameras is a low-cost, high-performance method. However, previous methods often rely on single-frame images for discrimination, which lacks relevant information in continuous video sequences, resulting in the inability to rely on historical frames for auxiliary identification and correction of lane line occlusion problems.

[0003] BEV is short for Bird's Eye View, which is a perspective or coordinate system (3D) used to describe the perceived world. Converting image space to BEV space can better represent the 3D information of the apparent environment. Lane line perception based on BEV space is the current mainstream solution. Depth information is crucial for the perception of 3D lane lines, so some work explores how to better use depth information for recognition. For example, LLS explores the use of convolutional networks to extract depth information from images, obtains BEV features based on depth information, and then performs subsequent recognition. However, these methods require additional information input, which will consume computing resources. In addition to the exploration of depth information, some work has begun to study how to better extract and associate features in image space and time series space to model more robust BEV features. For example, BEVFormer uses a ring view of 6 perspectives at four moments as input, and uses DeformableDETR to extract features, while Persformer uses the front view as input and constructs BEV features in a pyramidal manner.

[0004] However, the multi-view images input by the BEVFormer solution will bring about the problem of high computing resource overhead, and when the input samples of two frames vary greatly, it is difficult to align the same position points through motion information, which will introduce alignment errors and affect the learning of the network. In actual application scenarios, from a temporal perspective, the lane lines are often blocked in a certain frame, and the processing logic of Persformer is to predict each frame separately. Therefore, when the lane lines of a certain frame are blocked or change quickly, it is difficult to identify the current lane lines because previous historical information cannot be used to assist. Summary of the invention

[0005] The technical problem to be solved by the embodiments of the present invention is to provide a lane line recognition method, device, non-temporary computer-readable storage medium and vehicle based on a time-series bird's-eye view to improve the prediction performance when the lane line is blocked.

[0006] In order to solve the above technical problems, the present invention provides a lane line recognition method based on a time-series bird's-eye view, comprising:

[0007] Acquire an image within a preset range in front of the vehicle, and perform feature extraction on the image to obtain features of multiple frames of images;

[0008] The extracted multi-frame image features are stored in a preset image feature storage queue in time sequence;

[0009] Respectively obtaining the bird's-eye view features of the historical frames and the bird's-eye view features of the current frame in the preset image feature storage queue;

[0010] The bird's-eye view features of the historical frame are merged with the bird's-eye view features of the current frame to obtain updated bird's-eye view features of the historical frame;

[0011] Lane line recognition is performed using the bird's-eye view features of the updated historical frame.

[0012] Preferably, when the extracted multi-frame image features are stored in a preset image feature storage queue in time sequence, the method further includes:

[0013] Determining whether there is no bird's-eye view feature of the historical frame in the preset image feature storage queue;

[0014] If there is no bird's-eye view feature of the historical frame, the current frame is taken as the historical frame, and the bird's-eye view feature of the current frame is extracted as the bird's-eye view feature of the historical frame;

[0015] If there are bird's-eye view features of historical frames, the bird's-eye view features of the current frame are extracted.

[0016] Preferably, when the current frame stored in the preset image feature storage queue is the initial frame, the initial frame is taken as the historical frame, and the bird's-eye view features are extracted from the image features of the initial frame to obtain the bird's-eye view features of the historical frame; when the current frame stored in the preset image feature storage queue is not the initial frame, the previous frame of the current frame is taken as the historical frame, and the bird's-eye view features are extracted from the image features of the previous frame to obtain the bird's-eye view features of the historical frame.

[0017] Preferably, the bird's-eye view features of the historical frame and the bird's-eye view features of the current frame are fused, specifically: the bird's-eye view features of the historical frame and the bird's-eye view features of the current frame are fused using DeformableDETR, wherein the bird's-eye view features of the current frame are used as the query vector, and the bird's-eye view features of the historical frame are used as the key vector and the value vector.

[0018] Preferably, after obtaining the updated bird's-eye view features of the historical frame, it also includes: determining whether the bird's-eye view features of the current frame are the bird's-eye view features of the last frame in the preset image feature storage queue, if so, using the currently obtained updated bird's-eye view features of the historical frame for subsequent lane line recognition, if not, continuing to extract BEV features and update BEV features of the historical frame.

[0019] The present invention also provides a non-transitory computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the non-transitory computer-readable storage medium is located to execute the lane line recognition method based on the time-series bird's-eye view.

[0020] The present invention also provides a lane line recognition device based on a time-series bird's-eye view, comprising:

[0021] An image feature extraction module is used to extract features from the image acquired within a preset range in front of the vehicle to obtain features of multiple frames of images;

[0022] A storage module, used for storing the extracted multi-frame image features into a preset image feature storage queue in time sequence;

[0023] A bird's-eye view feature acquisition module, used to respectively acquire the bird's-eye view features of the historical frames and the bird's-eye view features of the current frame in the preset image feature storage queue;

[0024] A bird's-eye view feature fusion module, used to fuse the bird's-eye view features of the historical frame with the bird's-eye view features of the current frame to obtain updated bird's-eye view features of the historical frame;

[0025] A recognition module is used to perform lane line recognition using the bird's-eye view features of the updated historical frame.

[0026] Preferably, when the storage module stores the extracted multi-frame image features into the preset image feature storage queue in time sequence, it is also used to: determine whether there is no bird's-eye view feature of the historical frame in the preset image feature storage queue;

[0027] The bird's-eye view feature acquisition module is also used to: if there is no bird's-eye view feature of the historical frame, take the current frame as the historical frame and extract the bird's-eye view feature of the current frame as the bird's-eye view feature of the historical frame; if there is a bird's-eye view feature of the historical frame, extract the bird's-eye view feature of the current frame.

[0028] Preferably, the bird's-eye view feature acquisition module is also used for: when the current frame stored in the preset image feature storage queue is the initial frame, taking the initial frame as the historical frame, extracting the bird's-eye view features from the image features of the initial frame, and obtaining the bird's-eye view features of the historical frame; when the current frame stored in the preset image feature storage queue is not the initial frame, taking the previous frame of the current frame as the historical frame, extracting the bird's-eye view features from the image features of the previous frame, and obtaining the bird's-eye view features of the historical frame.

[0029] Preferably, the bird's-eye view feature fusion module is specifically used to: fuse the bird's-eye view features of the historical frame and the bird's-eye view features of the current frame using DeformableDETR, wherein the bird's-eye view features of the current frame are used as the query vector, and the bird's-eye view features of the historical frame are used as the key vector and the value vector.

[0030] Preferably, the device also includes an iteration module, which is used to determine whether the bird's-eye view features of the current frame are the bird's-eye view features of the last frame in the preset image feature storage queue after obtaining the updated bird's-eye view features of the historical frame. If so, the bird's-eye view features of the currently obtained updated historical frame are used for subsequent lane line recognition; if not, the bird's-eye view feature acquisition module continues to acquire the corresponding bird's-eye view features, and the bird's-eye view feature fusion module fuses the bird's-eye view features.

[0031] The present invention also provides a vehicle, comprising the lane line recognition device based on the time-series bird's-eye view.

[0032] The implementation of the present invention has the following beneficial effects: by introducing bird's-eye view features with time series information to update the bird's-eye view features of the historical frame, when the current lane line is obscured, the area similar to the current feature can be found from the historical features according to the historical time information, and the lane line information in the area is used to assist the current lane line recognition, providing strong support for improving the prediction performance when the lane line is obscured. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 It is a flowchart of a lane line recognition method based on a time-series bird's-eye view according to an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of the structure of BEVFormer.

[0036] Figure 3 It is a schematic diagram of the process principle of a lane line recognition method based on a time-series bird's-eye view according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following descriptions of the embodiments refer to the accompanying drawings to illustrate specific embodiments in which the present invention may be implemented.

[0038] Please refer to Figure 1 As shown, the first embodiment of the present invention provides a lane line recognition method based on a time-series bird's-eye view, including:

[0039] Acquire an image within a preset range in front of the vehicle, and perform feature extraction on the image to obtain features of multiple frames of images;

[0040] The extracted multi-frame image features are stored in a preset image feature storage queue in time sequence;

[0041] Respectively obtaining the bird's-eye view features of the historical frames and the bird's-eye view features of the current frame in the preset image feature storage queue;

[0042] The bird's-eye view features of the historical frame are merged with the bird's-eye view features of the current frame to obtain updated bird's-eye view features of the historical frame;

[0043] Lane line recognition is performed using the bird's-eye view features of the updated historical frame.

[0044] It can be seen from the above steps that the embodiment of the present invention introduces a bird's-eye view feature with time series information to update the bird's-eye view feature of the historical frame, so that when the current lane line is blocked, the area similar to the current feature can be found from the historical features according to the historical time information, and the lane line information in the area is used to assist the current lane line recognition.

[0045] Please refer to Figure 2As shown, the following is a brief introduction to BEVFormer (BEV+Transformer). Its main structure is divided into three parts: the backbone network Backbone, the spatial cross-attention module SCA (Spatial Cross-Attention Module), and the temporal self-attention module TSA (Temporal Self-Attention Module). Among them, the Temporal Self-Attention Module performs self-attention learning through the learnable BEV query vector (learnable BEV query) and the historical BEV feature (History BEV) to obtain the BEV feature (BEV feature) that integrates the temporal information. The BEV feature is then input into the spatial cross-attention module SCA as the query vector query, and the multi-camera feature (Multi-CamFeature) is input into the spatial cross-attention module SCA as the value vector Value, and cross-attention learning is performed to obtain the updated BEV feature, and finally the downstream task head is connected.

[0046] A set of queries that can learn BEV are defined in advance and are denoted as Where H and W represent the spatial size of BEV. N reference points are selected in the real world, and then the coordinates on BEV are obtained by scaling. Finally, the input query of SCA is the BEV feature corresponding to the reference point, and the Value is the Multi-Cam Feature. Then, the reference point is back-projected onto the image through the internal and external parameters of the camera to obtain the corresponding position information on the image, and the corresponding feature is obtained for cross attention. The crossattention adopts the deformable attention method. The input query of the TSA module is the learnable BEV feature corresponding to the reference point, and the input Value is the initial BEV query or the History BEV aligned with the current BEV query using ego-motion. The sum of the two results is the output of TSA, where the specific time information is obtained by setting the query to the stack of the History BEV and the current query features, and then using DeformableDETR to learn the offset and weight of each reference point, and then calculating the correlation with the historical features.

[0047] It is understandable that the time series information of BEVFormer is essentially a stacking of historical information on the BEV query, and the stacked features are used to retrieve the features of the relevant areas in the historical frames. On the one hand, this approach lacks physical meaning because the query and BEV feature are not high-dimensional features of the same dimension; on the other hand, since the input video is a sequence of time-series fragments, if the sequence is incoherent, it will lead to inconsistent features. The present invention is to improve these problems.

[0048] Please combine Figure 3 As shown, first, after obtaining the image within the preset range in front of the vehicle, the image can be preprocessed: the input image is randomly rotated, fixed-size scaled, etc. through the inverse perspective transformation module IPM (Inverse Perspective Mapping); the rotation angle and scaling size are set according to actual needs.

[0049] When extracting features from the image, as an example, this embodiment uses the EfficientNet network as the backbone to extract image features. EfficientNet is a highly scalable and efficient neural network structure that optimizes model efficiency and accuracy by balancing the depth, width, and resolution of the network. It is mainly used for image classification or image feature extraction tasks. This embodiment can also use other backbones to extract image features, which is not specifically limited in this embodiment.

[0050] Since the images acquired within the preset range in front of the vehicle have time sequence information (for example, 16 frames of images are obtained within 20 seconds, and each frame of image corresponds to a moment), the extracted multi-frame image features also have time sequence information accordingly; the extracted multi-frame image features are stored in the preset image feature storage queue in time sequence. The preset image feature storage queue has a preset length (for example, 5 frames), so when the extracted image features are stored in the preset image feature storage queue in time sequence, it is also determined whether the length of the preset image feature storage queue has been met, and if not, it continues to be stored. The length of the preset image feature storage queue is customizable. For example, if the scene in which the vehicle is located changes quickly, the difference between the image features of the previous and next two frames is large, so the length of the preset image feature storage queue can be set longer; if the scene in which the vehicle is located changes slowly, the difference between the image features of the previous and next two frames is small, so the length of the preset image feature storage queue can be set shorter; of course, it can also be set in combination with the vehicle-side computing power, etc.

[0051] Next, determine whether the preset image feature storage queue does not have BEV features of historical frames. It is understandable that if the process has just started, before the extracted image features are stored in the preset image feature storage queue in time sequence, there are no image features in the preset image feature storage queue, that is, there are no BEV features of historical frames; therefore, the first frame of image features stored (that is, the initial frame of image features of the entire process) will be used as a historical frame, and BEV features will be extracted from it (for example, using DeformableDETR to extract), and the BEV features of the historical frame will be obtained (History BEV Features). When subsequent image features are stored in the preset image feature storage queue in time sequence, for example, when the second frame of image features is stored, the first frame of image features in the preset image feature storage queue has become a historical frame, and its BEV features have been extracted, that is, the preset image feature storage queue already has BEV features of historical frames, then the aforementioned second frame of image will be used as the current frame, and BEV features will also be extracted from it to obtain the BEV features of the current frame (Current BEV Features). The embodiment of the present invention adopts DeformableDETR to extract BEV features for the purpose of saving computational overhead and balancing optimal performance. Of course, other methods can also be used to extract BEV features, such as depth estimation methods such as LSS or methods such as DETR, which are not limited in this embodiment.

[0052] Therefore, when the extracted image features are stored in a preset image feature storage queue in time sequence, the present embodiment further includes:

[0053] Determining whether there is no bird's-eye view feature of the historical frame in the preset image feature storage queue;

[0054] If there is no bird's-eye view feature of the historical frame, the current frame is taken as the historical frame, and the bird's-eye view feature of the current frame is extracted as the bird's-eye view feature of the historical frame;

[0055] If there are bird's-eye view features of historical frames, the bird's-eye view features of the current frame are extracted.

[0056] As mentioned above, when the first frame image features are stored in the preset image feature storage queue, there are no image features in the preset image feature storage queue, so the judgment result is that there are no bird's-eye view features of the historical frame. At this time, the BEV features of the currently stored first frame image features are extracted to obtain the BEV features of the historical frame; when the first frame image features have been stored in the preset image feature storage queue and the second frame image features are stored in time sequence, the BEV features of the historical frame already exist in the preset image feature storage queue (obtained by BEV feature extraction of the first frame image features), so the judgment result is that there are bird's-eye view features of the historical frame, then the BEV features of the currently stored second frame image features are extracted to obtain the BEV features of the current frame.

[0057] After obtaining the bird's-eye view features of the historical frame and the bird's-eye view features of the current frame, the bird's-eye view features of the historical frame and the bird's-eye view features of the current frame are fused to obtain the updated bird's-eye view features of the historical frame. Specifically, the two BEV features are fused using DeformableDETR, where the BEV features of the current frame are used as the query vector Q (query), and the BEV features of the historical frame are used as the key vector K (key) and the value vector V (value). In addition, the Pooling layer or Transformer can be used to fuse the two features: for example, the Pooling layer is used to align the scales of the two features, and then stack them on the channel, or directly add the elements; or the self-attention mechanism in the Transformer is used to calculate the similarity between the two bird's-eye view features, and the two bird's-eye view features are weightedly fused according to the weight of the similarity.

[0058] Furthermore, this embodiment also determines whether the BEV feature of the current frame is the BEV feature of the last frame in the preset image feature storage queue. If so, the bird's-eye view feature of the currently updated historical frame is used for the subsequent lane line recognition process. If not, BEV feature extraction and BEV feature update of the historical frame are continued.

[0059] According to the above process, when the image features of the third frame are stored in time sequence, the image features of the first frame and the second frame are already stored in the preset image feature storage queue. At this time, the second frame is used as the historical frame, and the BEV features extracted from the image features of the second frame are used as the BEV features of the historical frame. The third frame currently stored is used as the current frame, and the BEV features of the third frame image features are extracted to obtain the BEV features of the current frame; that is, when the image features of the current frame are stored in time sequence, if the current frame is not the initial frame, the previous frame is used as the historical frame, and the BEV features extracted from the previous frame are used as the BEV features of the historical frame. On the one hand, this embodiment updates the BEV features of the historical frame through the BEV features of the previous and next frames with different time sequence information. The BEV features of the previous and next frames have actual physical meanings and represent the corresponding physical space where the vehicle is located; on the other hand, through this cyclic update mechanism, the BEV features of the historical frame can be continuously updated to provide support for downstream lane line recognition.

[0060] Corresponding to the lane line recognition method based on the time-series bird's-eye view described in the aforementioned embodiment 1 of the present invention, embodiment 2 of the present invention provides that the present invention also provides a non-transitory computer-readable storage medium, and the non-transitory computer-readable storage medium includes a stored computer program; wherein, when the computer program is running, it controls the device where the non-transitory computer-readable storage medium is located to execute the lane line recognition method based on the time-series bird's-eye view.

[0061] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the device.

[0062] The processor may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor. The processor is the control center of the device, and various parts of the device are connected using various interfaces and lines.

[0063] The memory mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, and a flash card (Flash Card), etc., or the memory can also be other volatile solid-state storage devices.

[0064] It should be noted that the above-mentioned device may include but is not limited to a processor and a memory, which can be understood by those skilled in the art.

[0065] Corresponding to the lane line recognition method based on the time-series bird's-eye view described in the first embodiment of the present invention, the third embodiment of the present invention provides a lane line recognition device based on the time-series bird's-eye view, including:

[0066] An image feature extraction module is used to extract features from the image acquired within a preset range in front of the vehicle to obtain features of multiple frames of images;

[0067] A storage module, used for storing the extracted multi-frame image features into a preset image feature storage queue in time sequence;

[0068] A bird's-eye view feature acquisition module, used to respectively acquire the bird's-eye view features of the historical frames and the bird's-eye view features of the current frame in the preset image feature storage queue;

[0069] A bird's-eye view feature fusion module, used to fuse the bird's-eye view features of the historical frame with the bird's-eye view features of the current frame to obtain updated bird's-eye view features of the historical frame;

[0070] A recognition module is used to perform lane line recognition using the bird's-eye view features of the updated historical frame.

[0071] Preferably, when the storage module stores the extracted multi-frame image features into the preset image feature storage queue in time sequence, it is also used to: determine whether there is no bird's-eye view feature of the historical frame in the preset image feature storage queue;

[0072] The bird's-eye view feature acquisition module is also used to: if there is no bird's-eye view feature of the historical frame, take the current frame as the historical frame and extract the bird's-eye view feature of the current frame as the bird's-eye view feature of the historical frame; if there is a bird's-eye view feature of the historical frame, extract the bird's-eye view feature of the current frame.

[0073] Preferably, the bird's-eye view feature acquisition module is also used for: when the current frame stored in the preset image feature storage queue is the initial frame, taking the initial frame as the historical frame, extracting the bird's-eye view features from the image features of the initial frame, and obtaining the bird's-eye view features of the historical frame; when the current frame stored in the preset image feature storage queue is not the initial frame, taking the previous frame of the current frame as the historical frame, extracting the bird's-eye view features from the image features of the previous frame, and obtaining the bird's-eye view features of the historical frame.

[0074] Preferably, the bird's-eye view feature fusion module is specifically used to: fuse the bird's-eye view features of the historical frame and the bird's-eye view features of the current frame using DeformableDETR, wherein the bird's-eye view features of the current frame are used as the query vector, and the bird's-eye view features of the historical frame are used as the key vector and the value vector.

[0075] Preferably, the device also includes an iteration module, which is used to determine whether the bird's-eye view features of the current frame are the bird's-eye view features of the last frame in the preset image feature storage queue after obtaining the updated bird's-eye view features of the historical frame. If so, the bird's-eye view features of the currently obtained updated historical frame are used for subsequent lane line recognition; if not, the bird's-eye view feature acquisition module continues to acquire the corresponding bird's-eye view features, and the bird's-eye view feature fusion module fuses the bird's-eye view features.

[0076] Corresponding to the lane line recognition device based on a time-series bird's-eye view described in the aforementioned third embodiment of the present invention, a fourth embodiment of the present invention provides a vehicle, including the lane line recognition device based on a time-series bird's-eye view described in the third embodiment of the present invention.

[0077] For the working principle and process of the above embodiment, please refer to the description of the above embodiment of the present invention, which will not be repeated here.

[0078] From the above description, it can be seen that compared with the prior art, the beneficial effect of the present invention is that: by introducing a bird's-eye view feature with time series information to update the bird's-eye view feature of the historical frame, when the current lane line is obscured, the area similar to the current feature can be found from the historical features according to the historical time information, and the lane line information in the area is used to assist the current lane line recognition, providing strong support for improving the prediction performance when the lane line is obscured.

[0079] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A lane line recognition method based on a time-series bird's-eye view, characterized in that: include: Acquire an image within a preset range in front of the vehicle, and perform feature extraction on the image to obtain features of multiple frames of images; The extracted multi-frame image features are stored in a preset image feature storage queue in time sequence; Respectively obtaining the bird's-eye view features of the historical frames and the bird's-eye view features of the current frame in the preset image feature storage queue; The bird's-eye view features of the historical frame are merged with the bird's-eye view features of the current frame to obtain updated bird's-eye view features of the historical frame; Lane line recognition is performed using the bird's-eye view features of the updated historical frame.

2. The method according to claim 1, characterized in that When the extracted multi-frame image features are stored in a preset image feature storage queue in time sequence, the following steps are also included: Determining whether there is no bird's-eye view feature of the historical frame in the preset image feature storage queue; If there is no bird's-eye view feature of the historical frame, the current frame is taken as the historical frame, and the bird's-eye view feature of the current frame is extracted as the bird's-eye view feature of the historical frame; If there are bird's-eye view features of historical frames, the bird's-eye view features of the current frame are extracted.

3. The method according to claim 2, characterized in that When the current frame stored in the preset image feature storage queue is the initial frame, the initial frame is taken as the historical frame, and the bird's-eye view features are extracted from the image features of the initial frame to obtain the bird's-eye view features of the historical frame; when the current frame stored in the preset image feature storage queue is not the initial frame, the previous frame of the current frame is taken as the historical frame, and the bird's-eye view features are extracted from the image features of the previous frame to obtain the bird's-eye view features of the historical frame.

4. The method according to claim 1, characterized in that: The bird's-eye view features of the historical frame and the bird's-eye view features of the current frame are fused, specifically: the bird's-eye view features of the historical frame and the bird's-eye view features of the current frame are fused using DeformableDETR, wherein the bird's-eye view features of the current frame are used as query vectors, and the bird's-eye view features of the historical frame are used as key vectors and value vectors.

5. The method according to claim 1, characterized in that After obtaining the updated bird's-eye view features of the historical frame, it also includes: determining whether the bird's-eye view features of the current frame are the bird's-eye view features of the last frame in the preset image feature storage queue, if so, using the currently obtained updated bird's-eye view features of the historical frame for subsequent lane line recognition, if not, continuing to extract BEV features and update BEV features of the historical frame.

6. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium includes a stored computer program; wherein, when the computer program is run, it controls the device where the non-transitory computer-readable storage medium is located to execute the lane line recognition method based on the time-series bird's-eye view as described in any one of claims 1 to 5.

7. A lane line recognition device based on a time-series bird's-eye view, characterized in that: include: An image feature extraction module is used to extract features from the image acquired within a preset range in front of the vehicle to obtain features of multiple frames of images; A storage module, used for storing the extracted multi-frame image features into a preset image feature storage queue in time sequence; A bird's-eye view feature acquisition module, used to respectively acquire the bird's-eye view features of the historical frames and the bird's-eye view features of the current frame in the preset image feature storage queue; A bird's-eye view feature fusion module, used to fuse the bird's-eye view features of the historical frame with the bird's-eye view features of the current frame to obtain updated bird's-eye view features of the historical frame; A recognition module is used to perform lane line recognition using the bird's-eye view features of the updated historical frame.

8. The device according to claim 7, characterized in that When the storage module stores the extracted multi-frame image features into the preset image feature storage queue in time sequence, it is also used to: determine whether there is no bird's-eye view feature of the historical frame in the preset image feature storage queue; The bird's-eye view feature acquisition module is also used for: if there is no bird's-eye view feature of the historical frame, taking the current frame as the historical frame, and extracting the bird's-eye view feature of the current frame as the bird's-eye view feature of the historical frame; If there are bird's-eye view features of historical frames, the bird's-eye view features of the current frame are extracted.

9. The device according to claim 8, characterized in that The bird's-eye view feature acquisition module is also used for: when the current frame stored in the preset image feature storage queue is the initial frame, the initial frame is used as the historical frame, and the bird's-eye view features are extracted from the image features of the initial frame to obtain the bird's-eye view features of the historical frame; when the current frame stored in the preset image feature storage queue is not the initial frame, the previous frame of the current frame is used as the historical frame, and the bird's-eye view features are extracted from the image features of the previous frame to obtain the bird's-eye view features of the historical frame.

10. The device according to claim 7, characterized in that The bird's-eye view feature fusion module is specifically used to: fuse the bird's-eye view features of the historical frame and the bird's-eye view features of the current frame using DeformableDETR, wherein the bird's-eye view features of the current frame are used as query vectors, and the bird's-eye view features of the historical frame are used as key vectors and value vectors.

11. The device according to claim 7, characterized in that It also includes an iteration module, which is used to determine whether the bird's-eye view features of the current frame are the bird's-eye view features of the last frame in the preset image feature storage queue after obtaining the updated bird's-eye view features of the historical frame. If so, the bird's-eye view features of the currently obtained updated historical frame are used for subsequent lane line recognition; if not, the bird's-eye view feature acquisition module continues to acquire the corresponding bird's-eye view features, and the bird's-eye view feature fusion module fuses the bird's-eye view features.

12. A vehicle, characterized in that: It comprises a lane line recognition device based on a time-series bird's-eye view as described in any one of claims 7 to 11.

Images