BEV perception model training method and device based on spatiotemporal knowledge double distillation

Abstract

The application relates to a BEV perception model training method and device based on space-time knowledge double distillation, and relates to the field of automatic driving, wherein the method comprises the following steps: constructing a space fusion teacher model for generating space BEV features containing space geometric information; constructing a time sequence fusion teacher model for generating dynamic space BEV features containing time sequence information; constructing a pure visual student model for generating first space BEV features, fusing the first space BEV features before and after into second space BEV features with time sequence information, generating a current final BEV feature, and outputting a 3D detection result based on the current final BEV feature; acquiring space distillation loss, time sequence distillation loss and self detection loss, and performing distillation training on the student model by combining the three kinds of losses. The scheme not only maintains low cost and high efficiency, but also realizes performance comparable to complex multi-modal methods.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, and in particular to a method and apparatus for training a BEV perception model based on spatiotemporal knowledge double distillation. Background Technology

[0002] In the ongoing development of autonomous driving technology, accurate three-dimensional perception of the vehicle's surrounding environment is a crucial foundation for its safe and efficient operation. Among numerous perception solutions, the Bird's-Eye-View (BEV) representation method has become the mainstream research paradigm due to its unique advantages. The BEV representation method can intuitively display the spatial layout and motion state of objects in a scene and can seamlessly integrate with downstream planning and control modules, providing vital environmental information support for autonomous driving systems.

[0003] Currently, there are two main technical approaches in the field of BEV perception: multimodal fusion and pure vision.

[0004] Multimodal fusion methods, such as BEVFusion, which utilize LiDAR and cameras, fully leverage the complementary advantages of LiDAR's precise geometric information and camera's rich semantic information, achieving remarkable performance results in key tasks such as 3D object detection. However, these methods have several drawbacks. On the one hand, hardware costs are extremely high; LiDAR and other equipment are expensive, increasing the overall construction cost of the perception system. On the other hand, the synchronization, calibration, and fusion of multi-sensor data are extremely complex. This not only requires high-precision techniques to ensure data accuracy but also places extremely high demands on practical deployment, posing a significant challenge to the actual implementation of autonomous driving systems. The root of these problems lies in the fact that while current models offer powerful performance, they have significant limitations in cost control and complexity management.

[0005] Pure vision-based BEV perception methods, such as BEVFormer, rely solely on onboard cameras for information, offering significant advantages in terms of low cost and ease of deployment. However, due to the lack of direct depth information, they often lag behind multimodal methods in terms of positioning accuracy and robustness. This is because the performance of current pure vision models highly depends on the accuracy of depth prediction, and image information acquired solely through cameras inherently lacks depth information, making it difficult to meet the high requirements of autonomous driving in terms of positioning accuracy and robustness when facing complex environments and diverse scenarios.

[0006] Knowledge distillation has been widely applied in 2D vision tasks, but it is still in the exploratory stage in the field of 3D object detection, especially in cross-modal BEV perception tasks. Currently proposed knowledge distillation frameworks include general cross-modal distillation frameworks, foreground distillation based on LiDAR-guided masks, distillation methods based on improved LiDAR-guided masks, and self-distillation schemes. However, current methods typically focus only on a single aspect of knowledge transfer, failing to fully explore the potential of cross-modal knowledge transfer and limiting further improvements in model performance.

[0007] Therefore, ensuring the high performance of autonomous driving perception systems while effectively reducing their cost and complexity, particularly addressing the high cost and complexity of multimodal fusion technology and the shortcomings of pure vision technology in depth information acquisition, has become a key challenge that urgently needs to be overcome in the field of autonomous driving. Meanwhile, further exploring and optimizing the application of knowledge distillation in cross-modal BEV perception tasks, and uncovering the possibilities of knowledge transfer in multiple aspects, offers a new possibility for achieving pure vision-based multimodal perception capabilities. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a training method and apparatus for a BEV perception model based on spatiotemporal knowledge double distillation.

[0009] To achieve the above-mentioned objectives, this invention provides a BEV perception model training method based on spatiotemporal knowledge double distillation, comprising the following steps: S1. Construct a spatial fusion teacher model to fuse LiDAR point cloud data and multi-view RGB images and generate spatial BEV features containing spatial geometric information; S2. Construct a temporal fusion teacher model, which uses a temporal self-attention mechanism to fuse the current multi-view RGB image and the spatial BEV features of the previous time step to generate dynamic spatial BEV features containing temporal information. S3. Construct a pure vision student model, generate first spatial BEV features based on multi-view RGB images, fuse the first spatial BEV features generated at previous and subsequent times into the current second spatial BEV features with temporal information, generate the current final BEV features based on the second spatial BEV features, and output 3D detection results based on the current final BEV features. S4. Obtain the spatial distillation loss between the spatial BEV features and the first spatial BEV features, obtain the temporal distillation loss between the dynamic spatial BEV features and the current second spatial BEV features with temporal information, obtain the self-detection loss of the student model, and combine the three obtained losses to perform distillation training on the student model.

[0010] According to one aspect of the present invention, the spatial fusion teacher model is the BEVFusion model, and the temporal fusion teacher model is the BEVFormer model.

[0011] According to one aspect of the present invention, in step S3, the step of constructing a pure vision student model, the student model includes: an image backbone network, a view transformation module, a temporal fusion module, a BEV encoder, and a detection head; The image backbone network is used to receive multi-view RGB images and extract image features; The view transformation module is used to receive image features and generate first spatial BEV features; The temporal fusion module is used to fuse the first spatial BEV features generated at previous and subsequent times into the current second spatial BEV feature with temporal information. The BEV encoder generates the current final BEV feature based on the current second spatial BEV feature with temporal information; The detection head outputs 3D detection results based on the current final BEV features.

[0012] According to one aspect of the invention, the image backbone network is a ResNet network; The view transformation module is the View Transformer module.

[0013] According to one aspect of the present invention, in step S4, the step of obtaining the spatial distillation loss between the spatial BEV features and the first spatial BEV features, the spatial distillation loss is obtained using a foreground-aware distillation loss, and is expressed as follows: ; in, Indicates space distillation loss, The foreground mask represents the foreground region in the image. This indicates element-wise multiplication. This represents the spatial BEV features generated by the spatial fusion teacher model. This indicates that the student model generates first-space BEV features based on multi-view RGB images. This represents the Euclidean norm.

[0014] According to one aspect of the invention, in step S4, the step of obtaining the temporal distillation loss between the dynamic spatial BEV feature and the currently temporally information-based second spatial BEV feature is based on the L2 distance between the dynamic spatial BEV feature and the currently temporally information-based second spatial BEV feature, and is expressed as: ; in, Indicates the time-series distillation loss. express Dynamic spatial characteristics of BEV at any given moment express The second-space BEV feature at time step, i.e. the second-space BEV feature with temporal information at the current time step.

[0015] According to one aspect of the present invention, in step S4, the step of performing distillation training on the student model using the three obtained losses, a total loss function for distillation training of the student model is obtained based on spatial distillation loss, temporal distillation loss, and self-detection loss, to complete the distillation training of the student model; wherein, the total loss function is expressed as: ; in, Represents the total loss function. Indicates the loss from its own testing. Indicates the loss of space distillation used to balance the space. hyperparameters, Represented by equilibrium time-series distillation loss Hyperparameters.

[0016] To achieve the above-mentioned objectives, the present invention provides a BEV perception model training device based on spatiotemporal knowledge double distillation, comprising: a first teacher model construction module, used to construct a spatial fusion teacher model, and to fuse LiDAR point cloud data and multi-view RGB images based on the spatial fusion teacher model to generate spatial BEV features containing spatial geometric information; The second teacher model construction module is used to construct a temporal fusion teacher model and, based on the temporal self-attention mechanism of the temporal fusion teacher model, fuse the current multi-view RGB image and the spatial BEV features of the previous time step to generate dynamic spatial BEV features containing temporal information. The student model building module is used to build a pure vision student model, generate first spatial BEV features from multi-view RGB images based on the student model, fuse the first spatial BEV features generated at previous and next time steps into the current second spatial BEV features with temporal information, generate the current final BEV features based on the second spatial BEV features, and output 3D detection results based on the current final BEV features. The distillation training module is used to obtain the spatial distillation loss between the spatial BEV features and the first spatial BEV features, the temporal distillation loss between the dynamic spatial BEV features and the current second spatial BEV features with temporal information, and the self-detection loss of the student model. The student model is then trained by distillation using the three obtained losses.

[0017] According to one aspect of the invention, it further includes: The visual image acquisition module is used to acquire multi-view RGB images and uses the acquired multi-view RGB images as input to the first teacher model construction module, the second teacher model construction module, and the student model construction module. The point cloud acquisition module is used to acquire LiDAR point cloud data, and the acquired LiDAR point cloud data is used as input to the first teacher model construction module.

[0018] The beneficial effects of this invention are as follows: According to one aspect of the present invention, by targeting the learning of spatial fusion and temporal fusion capabilities from spatial fusion teacher models and temporal fusion teacher models respectively, the student model of this scheme achieves an excellent balance between performance and efficiency.

[0019] According to one aspect of the present invention, the student model is set up with a double distillation method based on a spatial fusion teacher model and a temporal fusion teacher model, which allows the student model to absorb knowledge from two different but complementary dimensions. This maximizes performance and ensures accurate and reliable output results while maintaining lightweight design. Furthermore, the student model of this approach can simulate the complex perceptual capabilities of a multimodal temporal model using only visual image input.

[0020] According to one aspect of the present invention, the student model of this scheme implements pure visual BEV perception, which achieves performance comparable to complex multimodal methods while maintaining low computational cost and high inference speed (FPS).

[0021] According to one aspect of the present invention, by minimizing spatial distillation loss, the student model is incentivized to generate first spatial BEV features that are highly consistent with the output of the spatially integrated teacher model in the foreground region, thereby indirectly learning the spatial structure knowledge provided by LiDAR point cloud data, effectively improving the student model's spatial perception ability.

[0022] According to one aspect of the present invention, during the distillation training process, the student model is effectively guided to simulate the final output of the temporal fusion teacher model. This not only forces the student model to learn how to aggregate historical information, but more importantly, it can further learn how the temporal fusion teacher model uses spatiotemporal context to infer the position, posture and motion state of objects during the fusion process, thereby effectively improving the perceptual robustness and accuracy of the student model in dynamic scenes.

[0023] According to one aspect of the present invention, the three objectives are optimized and jointly achieved based on the constructed loss function, thereby enabling the student model to absorb multimodal and temporal information to the greatest extent while maintaining lightweight design, thus achieving a significant performance improvement.

[0024] According to one aspect of the present invention, this approach provides a new direction for the development of autonomous driving perception systems: namely, bridging the gap between low-cost sensors and high-performance perception through advanced knowledge distillation technology, and extending this framework to more perception tasks to achieve efficient and economical autonomous driving perception effects. Attached Figure Description

[0025] Figure 1 This is a structural diagram of the BEV perception model training method based on spatiotemporal knowledge double distillation of the present invention. Figure 2 This is a double distillation framework diagram of the BEV perception model training method based on spatiotemporal knowledge double distillation of the present invention. Detailed Implementation

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0027] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The embodiments cannot be described in detail here, but the embodiments of the present invention are not limited to the following embodiments.

[0028] Combination Figure 1 and Figure 2 As shown, according to one embodiment of the present invention, a BEV perception model training method based on spatiotemporal knowledge double distillation includes the following steps: S1. Construct a spatial fusion teacher model to fuse LiDAR point cloud data and multi-view RGB images and generate spatial BEV features containing spatial geometric information; S2. Construct a temporal fusion teacher model, which uses a temporal self-attention mechanism to fuse the current multi-view RGB image and the spatial BEV features of the previous time step to generate dynamic spatial BEV features containing temporal information. S3. Construct a pure vision student model, generate first spatial BEV features based on multi-view RGB images, fuse the first spatial BEV features generated at previous and subsequent time steps into the current second spatial BEV features with temporal information, generate the current final BEV features based on the second spatial BEV features, and output 3D detection results based on the current final BEV features. S4. Obtain the spatial distillation loss between the spatial BEV features and the first spatial BEV features, obtain the temporal distillation loss between the dynamic spatial BEV features and the current second spatial BEV features with temporal information, obtain the self-detection loss of the student model, and combine the three obtained losses to perform distillation training on the student model.

[0029] According to one embodiment of the present invention, the spatial fusion teacher model is a BEVFusion model. Based on the advantages of the BEVFusion model in multimodal BEV feature fusion, LiDAR point cloud data and multi-view RGB images are efficiently fused in a unified BEV space to generate high-quality spatial BEV features rich in accurate spatial geometric information. Thus, the distillation training process of the student model is guided by the BEVFusion model, enabling the student model to generate first spatial BEV features with similar spatial geometric information from multi-view RGB images alone.

[0030] like Figure 2 As shown, in this embodiment, the spatial fusion teacher model includes: a first RGB image input module, a LiDAR point cloud input module, a fusion encoder, a BEV encoder, and a fusion BEV feature output module; wherein, the first RGB image input module is used to input multi-view RGB images; the LiDAR point cloud input module is used to input LiDAR point cloud data; the fusion encoder can use voxel segmentation to fuse and align image feature points in the multi-view RGB images with point cloud points in the LiDAR point cloud data, and output the fusion result; the BEV encoder performs BEV encoding based on the output fusion result; the fusion BEV feature output module receives the encoding structure output by the BEV encoder, splices it, and outputs spatial BEV features containing spatial geometric information.

[0031] According to one embodiment of the present invention, the temporal fusion teacher model is a BEVFormer model. Based on the expertise of the BEVFormer model in processing temporal information and utilizing spatiotemporal attention, it achieves the ability to associate information in processing long video sequences and capturing object motion information. Thus, the distillation training process of the student model is guided by the BEVFormer model, and the student model achieves the performance of effectively aggregating historical information and performing dynamic target detection.

[0032] like Figure 2As shown, in this embodiment, the temporal fusion teacher model includes: a second RGB image input module, a backbone network, a historical BEV feature output module, a BEV feature query vector module, a BEVFormer module, and a dynamic feature output module; wherein, the first RGB image input module is used to input multi-view RGB images; the backbone network extracts multi-view image features based on the input multi-view RGB images, the historical BEV feature output module is used to output the spatial BEV features of the previous moment, the BEV feature query vector module is used to output the BEV feature query vector, the BEVFormer module fuses the multi-view image features and the spatial BEV features of the previous moment based on the BEV feature query vector, and the dynamic feature output module outputs dynamic spatial BEV features containing temporal information based on the fusion result of the BEVFormer module.

[0033] Through the above settings, the student model is double-distilled based on the spatial fusion teacher model and the temporal fusion teacher model, enabling the student model to absorb knowledge from two different but complementary dimensions. This maximizes performance and ensures accurate and reliable output results while maintaining lightweight design.

[0034] like Figure 2 As shown, according to one embodiment of the present invention, in step S3, the step of constructing a pure vision student model includes: an image backbone network, a view transformation module, a temporal fusion module, a BEV encoder, and a detection head; wherein, the image backbone network is used to receive multi-view RGB images and extract image features; the view transformation module is used to receive image features and generate first spatial BEV features; the temporal fusion module is used to fuse the first spatial BEV features generated at previous and subsequent time steps into a second spatial BEV feature with temporal information; wherein, in the temporal fusion module, the first spatial BEV feature at the previous time step and the first spatial BEV feature at the current time step are aligned and then stitched together in the same channel dimension to achieve a simple and efficient effect of fusing BEV features. The BEV encoder generates the current final BEV feature based on the current second spatial BEV feature with temporal information, wherein the BEV encoder can be implemented using a 2D CNN network structure.

[0035] In another implementation, the time-series fusion module may selectively fuse the second spatial BEV feature with time-series information generated in the previous moment with the first spatial BEV feature at the current moment to generate the second spatial BEV feature with time-series information. Thus, in the time-series fusion module, by collecting the second spatial BEV feature with time-series information generated in the previous moment, aligning the second spatial BEV feature of the previous moment with the first spatial BEV feature of the current moment, and then stitching them together in the same channel dimension, the same simple and efficient effect of fusing BEV features can be achieved, and richer time-series information can also be fused.

[0036] According to one embodiment of the present invention, the image backbone network is a ResNet network; the view transformation module is a ViewTransformer module.

[0037] According to one embodiment of the present invention, in step S4, the spatial distillation loss between the spatial BEV features and the first spatial BEV features is obtained using a foreground-aware distillation loss. This distillation only applies to features in the foreground region (i.e., the region containing the object) to avoid interference from background noise and to focus more on object features crucial to the detection task. The spatial distillation loss is then expressed as: ; in, Indicates space distillation loss, A foreground mask representing the foreground region in the image (generated from the detection results of the spatial fusion teacher model or ground truth labels). This indicates element-wise multiplication. This represents the spatial BEV features generated by the spatial fusion teacher model. This indicates that the student model generates first-space BEV features based on multi-view RGB images. This represents the Euclidean norm.

[0038] Furthermore, by minimizing the spatial distillation loss, the student model is incentivized to generate first spatial BEV features that are highly consistent with the output of the teacher model in the foreground region and spatially integrated. This enables the student model to indirectly learn the spatial structure knowledge provided by the LiDAR point cloud data, thereby effectively improving the student model's spatial perception ability.

[0039] According to one embodiment of the present invention, in step S4, the step of obtaining the temporal distillation loss between the dynamic spatial BEV features and the current second spatial BEV features with temporal information, is based on temporal fusion distillation to enable the student model to learn the powerful temporal information processing capabilities of the temporal fusion teacher model. This allows the student model to aggregate information from historical BEV features, achieving the effects of velocity estimation and occluded object detection. Specifically, for the temporal fusion teacher model, in processing the current moment... When processing data, it will simultaneously utilize the current multi-view RGB image and the previous moment's data. The spatial BEV characteristics are used to generate the current moment through its powerful spatiotemporal attention module. The dynamic spatial BEV characteristic representation is denoted as Correspondingly, based on the fact that the temporal fusion module in the student model can receive the first spatial BEV features (denoted as ) generated from the current image by the view transformation module of the student model, the model can receive these features. ) and the first-space BEV characteristics of the previous moment (denoted as ) After fusion, the second spatial BEV feature with temporal information is output (denoted as...). Therefore, during distillation training, the student model's own temporal fusion module will output the current second-space BEV features with temporal information (denoted as...). Semantically and structurally approximate the dynamic space BEV features output by the temporal fusion teacher model (denoted as...). ); In another implementation, based on the fact that the temporal fusion module in the student model can receive the first spatial BEV features (denoted as ) generated from the current image by the view transformation module of the student model, the feature is further supported by setting the temporal fusion module in the student model. ) and the second-space BEV features with temporal information generated in the previous time step (denoted as ) After fusion, the second spatial BEV feature with temporal information is output (denoted as...). Therefore, during distillation training, the temporal fusion module of the student model itself can also output the second-space BEV features (denoted as...) that currently possess temporal information. Semantically and structurally approximate the dynamic space BEV features output by the temporal fusion teacher model (denoted as...). ).

[0040] Therefore, the temporal distillation loss is obtained based on the L2 distance between the dynamic spatial BEV features and the current temporally information-rich second spatial BEV features, and it is expressed as: ; in, Indicates the time-series distillation loss. express Dynamic spatial BEV characteristics at time (i.e., the current time). express The second-space BEV feature at time step, i.e. the second-space BEV feature with temporal information at the current time step.

[0041] With the above settings, during the distillation training process, the student model is effectively guided to simulate the final output of the temporal fusion teacher model. This not only forces the student model to learn how to aggregate historical information, but more importantly, it can further learn how the temporal fusion teacher model uses spatiotemporal context to infer the position, posture and motion state of objects during the fusion process, thereby effectively improving the student model's perceptual robustness and accuracy in dynamic scenes.

[0042] According to one embodiment of the present invention, in step S4, the step of performing distillation training on the student model by combining the three obtained losses, a total loss function for distillation training of the student model is obtained based on spatial distillation loss, temporal distillation loss, and self-detection loss, so as to complete the distillation training of the student model; wherein, the total loss function is expressed as: ; in, Represents the total loss function. Indicates the loss from its own testing. Indicates the loss of space distillation used to balance the space. hyperparameters, Represented by equilibrium time-series distillation loss Hyperparameters.

[0043] With the above settings, the three objectives are optimized and jointly achieved based on the constructed loss function, so that the student model can absorb multimodal and temporal information to the greatest extent while maintaining its lightweight nature, thus achieving a significant performance improvement.

[0044] According to one embodiment of the present invention, a training device for a BEV perception model based on spatiotemporal knowledge double distillation is provided, comprising: a first teacher model construction module, used to construct a spatial fusion teacher model, and to fuse LiDAR point cloud data and multi-view RGB images based on the spatial fusion teacher model to generate spatial BEV features containing spatial geometric information; a second teacher model construction module, used to construct a temporal fusion teacher model, and to fuse the current multi-view RGB image and the spatial BEV features of the previous moment based on the temporal self-attention mechanism of the temporal fusion teacher model to generate dynamic spatial BEV features containing temporal information; and a student model construction module, used to construct a purely visual student model. The system generates first spatial BEV features from multi-view RGB images based on the student model, merges the first spatial BEV features generated at different time points into a second spatial BEV feature with temporal information, generates the current final BEV feature based on the second spatial BEV feature, and outputs the 3D detection result based on the current final BEV feature. The distillation training module is used to obtain the spatial distillation loss between the spatial BEV features and the first spatial BEV features, the temporal distillation loss between the dynamic spatial BEV features and the current second spatial BEV feature with temporal information, and the self-detection loss of the student model. The student model is then trained by distillation using the three obtained losses.

[0045] In another implementation, the second spatial BEV feature can be generated by fusing the second spatial BEV feature with temporal information generated at the previous time with the first spatial BEV feature at the current time; the generation method is the same as the aforementioned method, and will not be repeated here.

[0046] According to one embodiment of the present invention, the BEV perception model training device based on spatiotemporal knowledge double distillation further includes: a visual image acquisition module, used to acquire multi-view RGB images, and using the acquired multi-view RGB images as input to the first teacher model construction module, the second teacher model construction module, and the student model construction module; and a point cloud acquisition module, used to acquire LiDAR point cloud data, and using the acquired LiDAR point cloud data as input to the first teacher model construction module.

[0047] Specific limitations regarding the BEV perception model training device based on spatiotemporal knowledge double distillation can be found in the limitations of the BEV perception model training method based on spatiotemporal knowledge double distillation mentioned above, and will not be repeated here. Each module in the aforementioned BEV perception model training device based on spatiotemporal knowledge double distillation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0048] In this embodiment, the memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0049] In this embodiment, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0050] The above description is merely an example of a specific solution of the present invention. For any devices and structures not described in detail herein, it should be understood that they are implemented using common devices and methods already available in the art.

[0051] The above description is merely one embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for a BEV perception model based on spatiotemporal knowledge double distillation, characterized in that, Includes the following steps: S1. Construct a spatial fusion teacher model to fuse LiDAR point cloud data and multi-view RGB images and generate spatial BEV features containing spatial geometric information; S2. Construct a temporal fusion teacher model, which uses a temporal self-attention mechanism to fuse the current multi-view RGB image and the spatial BEV features of the previous time step to generate dynamic spatial BEV features containing temporal information. S3. Construct a pure vision-based student model, generate first spatial BEV features based on multi-view RGB images, fuse the first spatial BEV features generated at previous and subsequent time steps into a second spatial BEV feature with temporal information, generate the current final BEV feature based on the second spatial BEV feature, and output 3D detection results based on the current final BEV feature; wherein, the student model includes: an image backbone network, a view transformation module, a temporal fusion module, a BEV encoder, and a detection head; The image backbone network is used to receive multi-view RGB images and extract image features; The view transformation module is used to receive image features and generate first spatial BEV features; The temporal fusion module is used to fuse the first spatial BEV features generated at previous and subsequent times into the current second spatial BEV feature with temporal information. The BEV encoder generates the current final BEV feature based on the current second spatial BEV feature with temporal information; The detection head outputs 3D detection results based on the current final BEV features; S4. Obtain the spatial distillation loss between the spatial BEV features and the first spatial BEV features, obtain the temporal distillation loss between the dynamic spatial BEV features and the current second spatial BEV features with temporal information, obtain the self-detection loss of the student model, and combine the three obtained losses to perform distillation training on the student model; wherein, the spatial distillation loss is obtained using the foreground-aware distillation loss, and it is expressed as: in, Indicates space distillation loss, The foreground mask represents the foreground region in the image. This indicates element-wise multiplication. This represents the spatial BEV features generated by the spatial fusion teacher model. This indicates that the student model generates first-space BEV features based on multi-view RGB images. Denotes the Euclidean norm; The temporal distillation loss is obtained based on the L2 distance between the dynamic spatial BEV features and the current temporally information-rich second spatial BEV features, and it is expressed as: in, Indicates the time-series distillation loss. express Dynamic spatial characteristics of BEV at any given moment express The second-space BEV feature at time step, i.e. the second-space BEV feature with temporal information at the current time step.

2. The BEV perception model training method based on spatiotemporal knowledge double distillation according to claim 1, characterized in that, The spatial fusion teacher model is the BEVFusion model, and the temporal fusion teacher model is the BEVFormer model.

3. The BEV perception model training method based on spatiotemporal knowledge double distillation according to claim 2, characterized in that, The image backbone network is a ResNet network; The view transformation module is the View Transformer module.

4. The BEV perception model training method based on spatiotemporal knowledge double distillation according to claim 3, characterized in that, In step S4, the step of performing distillation training on the student model using the three obtained losses involves obtaining a total loss function for distillation training of the student model based on spatial distillation loss, temporal distillation loss, and self-detection loss, thereby completing the distillation training of the student model; wherein, the total loss function is expressed as: in, Represents the total loss function. Indicates the loss from its own testing. Indicates the loss of space distillation used to balance the space. hyperparameters, Represented by equilibrium time-series distillation loss Hyperparameters.

5. A training device for a BEV perception model based on spatiotemporal knowledge double distillation, characterized in that, include: The first teacher model construction module is used to construct a spatial fusion teacher model, and based on the spatial fusion teacher model, fuse LiDAR point cloud data and multi-view RGB images to generate spatial BEV features containing spatial geometric information. The second teacher model construction module is used to construct a temporal fusion teacher model and, based on the temporal self-attention mechanism of the temporal fusion teacher model, fuse the current multi-view RGB image and the spatial BEV features of the previous time step to generate dynamic spatial BEV features containing temporal information. The student model construction module is used to construct a pure vision student model, generate first spatial BEV features from multi-view RGB images based on the student model, fuse the first spatial BEV features generated at different time steps into a second spatial BEV feature with temporal information, generate the current final BEV feature based on the second spatial BEV feature, and output 3D detection results based on the current final BEV feature; wherein, the student model includes: an image backbone network, a view transformation module, a temporal fusion module, a BEV encoder, and a detection head; The image backbone network is used to receive multi-view RGB images and extract image features; The view transformation module is used to receive image features and generate first spatial BEV features; The temporal fusion module is used to fuse the first spatial BEV features generated at previous and subsequent times into the current second spatial BEV feature with temporal information. The BEV encoder generates the current final BEV feature based on the current second spatial BEV feature with temporal information; The detection head outputs 3D detection results based on the current final BEV features; The distillation training module is used to obtain the spatial distillation loss between the spatial BEV features and the first spatial BEV features, the temporal distillation loss between the dynamic spatial BEV features and the current second spatial BEV features with temporal information, and the self-detection loss of the student model. The student model is then trained using a combination of these three losses. The spatial distillation loss is obtained using a foreground-aware distillation loss, and is expressed as follows: in, Indicates space distillation loss, The foreground mask represents the foreground region in the image. This indicates element-wise multiplication. This represents the spatial BEV features generated by the spatial fusion teacher model. This indicates that the student model generates first-space BEV features based on multi-view RGB images. Denotes the Euclidean norm; The temporal distillation loss is obtained based on the L2 distance between the dynamic spatial BEV features and the current temporally information-rich second spatial BEV features, and it is expressed as: in, Indicates the time-series distillation loss. express Dynamic spatial characteristics of BEV at any given moment express The second-space BEV feature at time step, i.e. the second-space BEV feature with temporal information at the current time step.

6. The BEV perception model training device based on spatiotemporal knowledge double distillation according to claim 5, characterized in that, Also includes: The visual image acquisition module is used to acquire multi-view RGB images and uses the acquired multi-view RGB images as input to the first teacher model construction module, the second teacher model construction module, and the student model construction module. The point cloud acquisition module is used to acquire LiDAR point cloud data, and the acquired LiDAR point cloud data is used as input to the first teacher model construction module.

Images