Three-dimensional target detection method capable of learning pose alignment and three-dimensional axial attention

Abstract

The invention discloses a three-dimensional target detection method capable of learning pose alignment and three-dimensional axial attention, and relates to the technical field of computer vision and intelligent traffic. The method comprises the following steps: receiving continuously collected multi-frame aerial view point cloud data, wherein the multi-frame point cloud data at least comprises a current frame point cloud and one or more frame historical frame point clouds; carrying out voxel division and feature coding on each frame of point cloud to obtain a corresponding single frame of aerial view BEV feature map; the method comprises the following steps: constructing a two-dimensional SE (2) pose transformation matrix from each historical frame to a current frame based on the self-parking pose information of a carrier in a preset time window, extracting rotation and translation parameters from the pose transformation matrix, and encoding the rotation and translation parameters into a spatial distribution motion feature map. According to the method, the performance and robustness of three-dimensional target detection in a complex dynamic scene can be improved by utilizing multi-frame time sequence information, and the method is clear in structure, easy to expand and suitable for the fields of automatic driving, intelligent transportation and the like.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, specifically to a 3D target detection method with learnable pose alignment and 3D axial attention. Background Technology

[0002] With the development of autonomous driving and intelligent transportation systems, 3D target detection technology based on LiDAR point clouds has attracted widespread attention. Existing single-frame point cloud 3D detection methods typically project or encode the point cloud into bird's-eye view BEV features, and then use a 2D convolutional network for feature extraction and target prediction, which has the advantages of simple structure and high inference efficiency. However, single-frame methods can only utilize information at the current moment, and their detection performance is limited for distant small targets, partially occluded targets, and fast-moving targets.

[0003] To improve detection accuracy, some methods incorporate multi-frame point cloud temporal information. By projecting consecutive frame point clouds onto a unified coordinate system and fusing features, the spatiotemporal representation of the target is enhanced. Existing techniques typically employ pose-based geometric grid sampling to rigidly align historical frame features, or directly concatenate multi-frame BEV features along the channel or time dimension before inputting them into a convolutional network for implicit fusion. However, existing techniques suffer from the following problems: Pure geometric grid sampling relies on the accuracy of pose estimation, cannot learn to compensate for pose errors and residual alignment errors caused by dynamic target motion, and often uses bilinear interpolation, which can easily lead to feature blurring. Simple feature concatenation or fixed weighted averaging cannot adequately model temporal relationships and long-distance spatial dependencies, and it is difficult to adaptively adjust the contribution of historical information under different scenarios and target motion patterns. While some Transformer-based spatiotemporal feature fusion methods can improve accuracy, their complex model structures, large number of parameters and computational demands place high demands on real-time performance and hardware resources, making them unsuitable for deployment on vehicle platforms.

[0004] To this end, this invention proposes a 3D target detection method that can learn pose alignment and 3D axial attention. Summary of the Invention

[0005] The purpose of this invention is to provide a 3D target detection method with learnable pose alignment and 3D axial attention. By introducing a self-supervised / assisted-supervised temporal alignment warp module and a 3D axial attention mechanism at the BEV feature level, the detection performance and robustness in complex scenes are effectively improved without significantly increasing inference complexity.

[0006] According to a first aspect of the present invention, in order to achieve the above-mentioned objective, the present invention provides the following technical solution: a three-dimensional target detection method capable of learning pose alignment and three-dimensional axial attention, comprising the following steps: Receive continuously acquired multi-frame bird's-eye view point cloud data, wherein the multi-frame point cloud data includes at least the current frame point cloud and one or more historical frame point clouds. Each frame of point cloud is divided into voxels and its features are encoded to obtain the corresponding single-frame bird's-eye view BEV feature map. Based on the vehicle pose information of the carrier within a preset time window, a two-dimensional SE(2) pose transformation matrix is ​​constructed from each historical frame to the current frame, and rotation and translation parameters are extracted from the pose transformation matrix. The rotation and translation parameters are then encoded into a spatially distributed motion feature map. The BEV feature maps of the point clouds of each historical frame are concatenated with the corresponding motion feature maps in the channel dimension and then input into a pre-built learnable convolutional alignment module, which outputs historical frame BEV feature maps aligned with the coordinate system of the current frame. Stack the current frame BEV feature map and the aligned historical frame BEV feature maps along the time dimension to form a temporal BEV feature body; The temporal BEV feature body is input into a pre-constructed three-dimensional axial attention module, and axial attention operations are performed sequentially or alternately along the time axis and spatial axis to obtain the temporally fused BEV feature body; Select the BEV feature map corresponding to the current frame time step from the temporally fused BE feature body, input it into the pre-constructed target detection head, and output the target category, three-dimensional position, size and orientation parameters of the current frame.

[0007] Furthermore, the multi-frame bird's-eye view point cloud data is continuously collected within a preset time window by a carrier equipped with a three-dimensional sensor.

[0008] Furthermore, each frame of point cloud is divided into voxels and its features are encoded to obtain the corresponding single-frame bird's-eye view BEV feature map, as follows: (31) Divide the three-dimensional space into uniform grid cylinders on the horizontal plane, and assign the points in each frame of the point cloud to the corresponding grid cylinders according to their spatial coordinates; (32) Enhance the point cloud data in each non-empty grid cylinder, calculate the offset of each point relative to the geometric center of the grid cylinder and the offset relative to the group center, and form the expanded point feature vector; (33) Use a multilayer perceptron to extract features from the expanded point feature vectors, and combine batch normalization and nonlinear activation functions to obtain high-dimensional point features; (34) Perform max pooling on all point features within each grid cylinder to extract the global features of the grid cylinder and generate a pseudo image composed of grid cylinder features, namely the single-frame bird's-eye view BEV feature map.

[0009] Furthermore, the two-dimensional SE(2) pose transformation matrix from each historical frame to the current frame is constructed as follows: (41) Represent the vehicle pose of the current frame and the historical frame as four-dimensional homogeneous transformation matrices, denoted as the current pose matrix and the historical pose matrix, respectively. (42) Calculate the inverse of the current pose matrix and multiply it with the historical pose matrix to obtain the relative pose matrix, as follows: Let the vehicle pose matrix of the current frame be... The vehicle pose matrix of the historical frame is Calculate the relative pose matrix from the current frame to the historical frames. : in, The matrix representing the inverse of the current pose matrix; To describe the transformation relationship between the historical frame coordinate system and the current frame coordinate system Homogeneous transformation matrix; (43) Extract the translation components on the horizontal plane and the rotation angles around the vertical axis from the relative pose matrix, and construct the corresponding two-dimensional planar rigid body transformation matrix SE(2) based on the translation components and rotation angles.

[0010] Furthermore, the learnable convolutional alignment module includes: Feature building units are used to extract r from the two-dimensional SE(2) pose transformation matrix. 11 , r 12 , r 21 , r 22 , t x ,t y Six parameters are used, and each parameter is expanded into a single-channel feature map with the same spatial size as the BEV feature map, which are then stitched together to form a six-channel motion feature map. The feature concatenation unit is used to concatenate the historical frame BEV feature map with the motion feature map in the channel dimension to obtain the input feature tensor. The convolution alignment unit is used to apply at least one layer of two-dimensional convolution, normalization and nonlinear activation operations to the input feature tensor in sequence, and output the historical frame BEV feature map aligned with the current frame coordinate system.

[0011] Furthermore, during the training phase of the learnable convolutional alignment module, a teacher-student auxiliary supervision mechanism based on geometric grid sampling is constructed to enable the learnable convolutional alignment module to quickly converge to a reasonable alignment state in the early stages of training. The specific steps are as follows: (61) The BEV feature map of the historical frame is geometrically transformed by the feature alignment method based on geometric grid sampling to obtain the teacher alignment feature map; (62) The difference between the student alignment feature map output by the learnable convolutional alignment module and the teacher alignment feature map is measured to form an auxiliary loss term; (63) The auxiliary loss term is weighted and summed with the main loss of target detection according to the preset weight, and used as the total training loss for backpropagation, so as to guide the learnable convolutional alignment module to learn the initial alignment capability close to geometric alignment, and further compensate for the pose estimation error and the alignment deviation caused by dynamic targets; wherein, the preset weight gradually decays with the training stage, or is set to zero after the preset training rounds.

[0012] Furthermore, the three-dimensional axial attention module includes a cascaded stack of multiple axial attention units, with the input of each axial attention unit being the output of the previous axial attention unit, in order to progressively deepen the modeling capability of temporal and spatial correlations. The three-dimensional axial attention unit receives a temporal BEV feature volume of size (C, T, H, W), where C is the number of channels, T is the number of time frames, and H and W are the spatial dimensions. Each layer of the three-dimensional axial attention unit includes the following cascaded components: (71) The time axis attention submodule is used to perform self-attention calculation on the feature sequence on the time axis at a fixed spatial location to model the temporal dependency between multiple frames; (72) Spatial axis attention submodule, used to perform self-attention calculation on feature sequences along the row and column directions at fixed time steps to model long-distance spatial dependencies; The spatial axis attention submodule includes one-dimensional self-attention branches executed along the row direction and along the column direction of the BEV feature map, respectively: The row-direction attention branch is used to input the feature sequences at different row indices as one-dimensional sequences into the self-attention computation under the condition of fixed time steps and column indices; The column-direction attention branch is used to input the feature sequences at different column indices as one-dimensional sequences into the self-attention computation under the condition of fixed time steps and row indices. The outputs of the two branches are superimposed on the input features via residuals. (73) Residual connection and gating unit, used to superimpose axial attention output and input features in residual form, and adjust the attention enhancement intensity through learnable gating parameters; (74) In the one-dimensional self-attention calculation of the time-axis attention submodule and the spatial-axis attention submodule: The query vector is generated based on an unsampling one-dimensional feature sequence to maintain feature resolution and detail representation. The key vector and value vector are generated based on a one-dimensional feature sequence downsampled along the corresponding axis.

[0013] Furthermore, before performing attention modeling on the temporal BEV feature volume, the three-dimensional axial attention module also includes a channel dimensionality reduction unit, which is used to reduce the number of channels C to a preset bottleneck channel number C'. After completing the operations of the temporal axis attention submodule and the spatial axis attention submodule, as well as the optional feedforward network operation, the channel dimensionality increase unit restores the number of channels from C' to C, thereby reducing the number of parameters and memory overhead in attention calculation.

[0014] Furthermore, the target detection head is a three-dimensional target detection head based on bird's-eye view features, comprising: The center point prediction branch, size regression branch, and orientation regression branch are used to predict the center position of the target and its height, length, width, and orientation parameters on the BEV plane.

[0015] According to a second aspect of the present invention, the present invention provides a multi-frame bird's-eye view point cloud target detection system for implementing the three-dimensional target detection method with learnable pose alignment and three-dimensional axial attention described in the first aspect, comprising: Receive continuously acquired multi-frame bird's-eye view point cloud data, wherein the multi-frame point cloud data includes at least the current frame point cloud and one or more historical frame point clouds. The data acquisition module is used to perform voxel division and feature encoding on each frame of point cloud to obtain the corresponding single-frame bird's-eye view BEV feature map. The single-frame feature extraction module is used to construct a two-dimensional SE(2) pose transformation matrix from each historical frame to the current frame based on the vehicle pose information of the carrier within a preset time window, and extract rotation and translation parameters from the pose transformation matrix, and encode the rotation and translation parameters into a spatially distributed motion feature map. The convolutional alignment module is used to concatenate the BEV feature maps of the point clouds of each historical frame with the corresponding motion feature maps in the channel dimension and input them into the pre-built learnable convolutional alignment module, and output the historical frame BEV feature maps aligned with the coordinate system of the current frame. The temporal feature construction module is used to stack the current frame BEV feature map and the aligned historical frame BEV feature maps along the time dimension to form a temporal BEV feature body. The three-dimensional axial attention module inputs the temporal BEV feature body into the three-dimensional axial attention module, and performs axial attention operations sequentially or alternately along the time axis and spatial axis to obtain the temporally fused BEV feature body; The detection head module is used to select the BEV feature map corresponding to the current frame time step from the temporally fused BE feature body, input it into the pre-constructed target detection head, and output the target category, three-dimensional position, size and orientation parameters of the current frame. The Geometric Alignment Teacher Module is used to generate teacher alignment features through geometric raster sampling; The loss calculation module is used to calculate the sum of the main loss for object detection and the auxiliary loss for convolution alignment.

[0016] The present invention has at least the following beneficial effects: 1. This invention introduces a learnable convolutional alignment module at the bird's-eye view feature level, encoding the vehicle's pose parameters into a spatial motion feature map. The module then adaptively learns the alignment transformation from historical frames to the current frame through a convolutional network. Compared to the rigid transformation of traditional geometric grid sampling, this module overcomes the feature blurring limitations caused by bilinear interpolation. Building upon geometric alignment, it further learns to compensate for pose estimation errors and residual deviations caused by dynamic target motion, significantly improving alignment accuracy and robustness in complex dynamic scenes.

[0017] 2. This invention constructs a teacher-student assisted supervision mechanism based on geometric grid sampling, which provides reliable initialization guidance for the learnable alignment module during the training phase, enabling it to quickly converge to a reasonable alignment state. This mechanism adopts a weight reduction or round-zeroing strategy to effectively inject geometric prior knowledge in the early stages of training, while avoiding excessive constraints on the model in the later stages, preserving the learning space for the network to autonomously explore better alignment strategies, and achieving a balance between geometric constraints and flexible learning.

[0018] 3. This invention employs three-dimensional axial attention to efficiently model the spatiotemporal dependencies of temporal BEV feature volumes. By decomposing the three-dimensional attention into one-dimensional self-attention operations on the time axis, spatial row axis, and spatial column axis, the number of model parameters and computational complexity are significantly reduced. Compared with global self-attention, it significantly reduces memory usage and computing power requirements. Furthermore, this module can effectively capture long-term temporal dependencies across frames and long-distance spatial associations in the BEV plane, adaptively adjusting the feature contribution weights of different historical frames and different spatial locations, thereby enhancing the ability to identify distant small targets, partially occluded targets, and fast-moving targets.

[0019] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the method described in this invention; Figure 2 This is a schematic diagram of the overall architecture of the system described in this invention; Figure 3 This is a schematic diagram of constructing a two-dimensional SE(2) transformation based on three-dimensional pose and performing geometric alignment on the BEV plane in this invention; Figure 4 This is a schematic diagram of the BEV-level EgoMotionConvWarp module structure in this invention; Figure 5 This is a schematic diagram of the teacher-student assisted supervision training process in this invention; Figure 6 This is a schematic diagram of the three-dimensional axial attention module in this invention; Figure 7 This is a schematic diagram of AP / APH of the method of the present invention under L1 difficulty on Waymo Open Dataset (training on a 20% subset of the training set); Figure 8 This is a schematic diagram of AP / APH of the method of the present invention under L2 difficulty on Waymo Open Dataset (training on a 20% subset of the training set). Detailed Implementation

[0021] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0022] Example 1: Please see Figures 1-8 This invention provides a technical solution: a 3D target detection method with learnable pose alignment and 3D axial attention, comprising the following steps: S1. Receive continuously acquired multi-frame bird's-eye view point cloud data, the multi-frame point cloud data including at least the current frame point cloud and one or more historical frame point clouds; The multi-frame bird's-eye view point cloud data is continuously acquired by the vehicle-mounted LiDAR in an autonomous driving scenario within a preset time window. The time window length is, for example, 3 or 4 frames, where frame 0 is the current frame and frames 1 to T-1 are historical frames. The system first preprocesses each frame of point cloud, including coordinate cropping, voxelization, Pillar feature encoding, etc., to obtain a single-frame BEV feature map of size (C0, H0, W0). S2. Perform voxel segmentation and feature encoding on each frame of point cloud to obtain the corresponding single-frame bird's-eye view BEV feature map, as follows: (S21) Divide the three-dimensional space into uniform grid pillars on the horizontal plane, and assign the points in each frame of the point cloud to the corresponding grid pillars according to their spatial coordinates; (S22) Enhance the point cloud data in each non-empty grid cylinder, calculate the offset of each point relative to the geometric center of the grid cylinder and the offset relative to the group center, and form the expanded point feature vector; (S23) Use a multilayer perceptron (MLP) to extract features from the expanded point feature vectors, and combine batch normalization and nonlinear activation functions to obtain high-dimensional point features; (S24) Perform max pooling on all point features within each grid cylinder to extract the global features of the cylinder and generate a pseudo image composed of grid cylinder features, i.e., the single-frame bird's-eye view BEV feature map. S3. Based on the vehicle's position and pose information collected within a preset time window by the vehicle odometer or inertial navigation system, construct a two-dimensional SE(2) pose transformation matrix from each historical frame to the current frame, extract rotation and translation parameters from the pose transformation matrix, and encode the rotation and translation parameters into a spatially distributed motion feature map. Specifically, the two-dimensional SE(2) pose transformation matrix from each historical frame to the current frame is constructed as follows: (S31) Represent the vehicle pose of the current frame and the historical frame as four-dimensional homogeneous transformation matrices, denoted as the current pose matrix and the historical pose matrix, respectively. (S32) Calculate the inverse of the current pose matrix and multiply it with the historical pose matrix to obtain the relative pose matrix, as follows: Let the vehicle pose matrix of the current frame be... The vehicle pose matrix of the historical frame is Calculate the relative pose matrix from the current frame to the historical frames. : in, The matrix representing the inverse of the current pose matrix; To describe the transformation relationship between the historical frame coordinate system and the current frame coordinate system Homogeneous transformation matrix; (S33) Extract the translation component on the horizontal plane and the rotation angle around the vertical axis from the relative pose matrix, and construct the corresponding two-dimensional planar rigid body transformation matrix SE(2) based on the translation component and the rotation angle. Specifically, such as Figure 3As shown, the vehicle's pose is described using a four-dimensional homogeneous transformation matrix, meaning that the pose at each moment is represented by a 4×4 matrix containing three-dimensional rotation and translation information. The system first records the poses corresponding to the current frame and a certain historical frame as the current pose matrix and the historical pose matrix, respectively. The relative poses of the two are obtained by calculating the inverse of the current pose matrix and multiplying it by the historical pose matrix. Then, the translation on the horizontal plane and the rotation angle around the vertical axis are extracted from the relative pose matrix, and a two-dimensional SE(2) transformation matrix describing the planar rigid body motion is constructed accordingly to characterize the relative poses from the historical frame to the current frame on the BEV plane. S4. After concatenating the BEV feature maps of the point clouds of each historical frame with the corresponding motion feature maps in the channel dimension, input them into the pre-built learnable convolutional alignment module, and output the historical frame BEV feature maps aligned with the coordinate system of the current frame. S41. Learnable convolution alignment modules include: Feature building units are used to extract r from the two-dimensional SE(2) pose transformation matrix. 11 , r 12 , r 21 , r 22 , t x ,t y Six parameters are used, and each parameter is expanded into a single-channel feature map with the same spatial size as the BEV feature map, which are then stitched together to form a six-channel motion feature map. The feature concatenation unit is used to concatenate the historical frame BEV feature map with the motion feature map in the channel dimension to obtain the input feature tensor. The convolution alignment unit is used to sequentially apply at least one layer of two-dimensional convolution, normalization and nonlinear activation operations to the input feature tensor, and output a historical frame BEV feature map aligned with the current frame coordinate system; Specifically, such as Figure 4 As shown, the learnable convolutional alignment module (EgoMotionConvWarp) takes the historical frame BEV feature map and the two-dimensional SE(2) transformation matrix as input. First, it extracts six parameters r from the SE(2) matrix. 11 , r 12 ,r 21 , r 22 , t x , t y Each feature map is copied and expanded to a single-channel feature map with the same spatial size (H, W) as the BEV feature map. The six feature maps are then concatenated in the channel dimension to form a six-channel motion feature map. This motion feature map is then concatenated with the historical frame BEV feature map in the channel dimension to form (C+6, H, W) input features. Learnable convolutional alignment modules can include two or more 2D convolutional layers, each followed by batch normalization and a non-linear activation function, such as ReLU or Leaky ReLU. Through end-to-end training, the convolutional network learns to align historical frame BEV features to the current frame coordinate system given prior motion features. Unlike fixed geometric interpolation, this module can adaptively adjust the alignment strategy based on scene structure and target motion patterns. During the training phase, to accelerate the convergence of EgoMotionConvWarp and introduce geometric priors, this invention constructs a teacher-student assisted supervision mechanism. For example... Figure 5 As shown, for the same historical frame BEV features, the system generates teacher-aligned feature maps using pose-based geometric grid sampling on one hand, and student-aligned feature maps using EgoMotionConvWarp on the other. The difference between the two is then calculated, for example, using L1 or L2 loss as auxiliary losses, and linearly weighted and summed with the main loss for object detection. In actual training, the auxiliary loss weight can be set to a value between 0.05 and 0.2 to achieve a balance between geometric prior constraints and network free learning. In a preferred embodiment, the auxiliary loss weight can adopt a strategy of gradually decaying with the training stage (e.g., linear decay or piecewise decay), and can be set to zero after a preset training round. This allows the auxiliary supervision to be mainly used for alignment initialization and stable convergence in the early stage of training, while in the later stage of training, the main loss of object detection drives the model to learn better alignment and fusion strategies, thereby avoiding excessive constraints on the student module when "teacher alignment is not optimal". S5. Stack the current frame BEV feature map and all aligned historical frame BEV feature maps along the time dimension to form a temporal BEV feature body of size (C, T, H, W), and input it into the three-dimensional axial attention module to model the correlation between multiple frames and multiple positions to obtain the fused temporal BEV feature body. S6. Input the temporal BEV feature volume into the pre-built three-dimensional axial attention module, and perform axial attention operations sequentially or alternately along the time axis and spatial axis to obtain the temporally fused BEV feature volume; The three-dimensional axial attention module consists of a cascaded stack of multiple axial attention units. The input of each axial attention unit is the output of the previous axial attention unit, so as to deepen the modeling capability of temporal and spatial correlation layer by layer. The three-dimensional axial attention unit receives a temporal BEV feature volume of size (C, T, H, W), where C is the number of channels, T is the number of time frames, and H and W are the spatial dimensions. Each layer of the three-dimensional axial attention unit includes the following cascaded components: S61. The temporal axis attention submodule is used to perform self-attention calculation on the feature sequence on the temporal axis at a fixed spatial location to model the temporal dependency between multiple frames. S62. Spatial axis attention submodule, used to perform self-attention calculation on feature sequences along row and column directions at fixed time steps to model long-distance spatial dependencies; The spatial axis attention submodule includes one-dimensional self-attention branches executed along the row direction and along the column direction of the BEV feature map, respectively: The row-direction attention branch is used to input the feature sequences at different row indices as one-dimensional sequences into the self-attention computation under the condition of fixed time steps and column indices; The column-direction attention branch is used to input the feature sequences at different column indices as one-dimensional sequences into the self-attention computation under the condition of fixed time steps and row indices. The outputs of the two branches are superimposed on the input features via residuals. S63. Residual connection and gating unit, used to superimpose axial attention output and input features in residual form, and adjust the attention enhancement intensity through learnable gating parameters; S64. In the one-dimensional self-attention calculation of the time-axis attention submodule and the spatial-axis attention submodule: The query vector is generated based on an unsampling one-dimensional feature sequence to maintain feature resolution and detail representation. The key vector and value vector are generated based on a one-dimensional feature sequence downsampled along the corresponding axis. Specifically, such as Figure 6 As shown, the three-dimensional axial attention module receives a temporal BEV feature volume of size (C, T, H, W). First, while keeping the spatial position unchanged, it performs self-attention operation on the temporal feature sequence to model the temporal dependency between the historical frame and the current frame. Then, while keeping the time step unchanged, it performs self-attention operation along the row and column directions respectively to model the long-distance spatial dependency on the BEV plane. In a preferred embodiment, to further reduce the number of parameters and memory overhead in attention computation, the three-dimensional axial attention module first reduces the number of channels from C to the bottleneck number of channels C' (e.g., C' = C / 2) through 1 × 1 convolution or linear transformation before performing attention operations. Then, it sequentially performs temporal axis attention, spatial row axis attention, spatial column axis attention, and optional feedforward network operations on the bottleneck number of channels. Finally, it restores the number of channels from C' to C through channel dimensionality upscaling.

[0023] The temporal axis attention treats the feature vectors at different time steps as a one-dimensional sequence at a fixed spatial location (h, w) and performs self-attention computation. The spatial row axis attention, under the condition of fixed time steps and column indices, treats the features at different row indices in the column as a one-dimensional sequence for attention modeling. The spatial column axis attention, under the condition of fixed time steps and row indices, treats the features at different column indices in the row as a one-dimensional sequence for attention modeling. Thus, while decomposing complex three-dimensional attention, it captures long-range spatial dependencies on the BEV plane.

[0024] To balance feature representation capability and computational efficiency, in one-dimensional axial attention, the query vector is preferably generated based on the unsampled feature sequence to preserve the complete spatial or temporal resolution; while the key vector and value vector are generated based on the feature sequence after downsampling along the corresponding axis, such as retaining only a portion of historical frames on the temporal axis or performing downsampling with a step size of 2 on the spatial axis, thereby reducing the storage and computing power required for attention without sacrificing query resolution.

[0025] To reduce computational complexity, this invention employs axial attention to decompose 3D attention into several 1D sub-attentions, and then fuses the attention results with the original features through residual connections and learnable gating parameters. After stacking several layers of axial attention, a fused feature body that fully models temporal and spatial relationships is obtained. S7. Select the BEV feature map corresponding to the current frame time step from the temporally fused BE feature body, input it into the pre-built target detection head, and output the target category, 3D position, size and orientation parameters of the current frame; The target detection head is a 3D target detection head based on bird's-eye view features, including: The center point prediction branch, size regression branch, and orientation regression branch are used to predict the target center position and its height, length, width, and orientation parameters on the BEV plane. It should be further explained that only the feature map corresponding to the current frame time step is selected from the temporally fused BEV feature body as the input of the target detection head. Historical frames only participate in attention modeling as temporal context and do not output detection results separately. In this way, the detection performance of the current frame can be improved by utilizing historical information without increasing the output branch.

[0026] The experimental results of the present invention will be verified next with reference to specific embodiments: Experimental verification was conducted based on the publicly available autonomous driving LiDAR dataset (WaymoOpenDataset). The experiment used continuous multi-frame point cloud input, generated single-frame BEV feature maps for each frame according to the method described in this invention, aligned historical frame BEV features using EgoMotionConvWarp and stacked them along the time dimension, then performed temporal fusion through a 3D axial attention module, and finally output the 3D detection result of the current frame from the detection head.

[0027] like Figure 7 and Figure 8 As shown, the dataset was trained using a 20% subset of the training set and evaluated on the validation set. The evaluation metrics used were the AP and APH (APH being the AP metric considering orientation) under the L1 / L2 difficulty partitioning commonly used by Waymo. The evaluation results for this embodiment are shown in Table 1.

[0028] Table 1: Detection results of this invention on Waymo (20% training subset) (unit: AP / APH) As can be seen from Table 1, the embodiments of the present invention achieved good detection accuracy in the Vehicle, Pedestrian and Cyclist categories, indicating that the technical route of "learnable BEV pose alignment + three-dimensional axial attention temporal modeling + current frame output" can effectively utilize historical frame information and help improve the stability and robustness of detection results in complex dynamic scenes.

[0029] In summary, this invention introduces a learnable pose alignment module at the BEV feature level, constructs a spatially distributed motion feature map using SE(2) motion parameters, and adaptively learns alignment operations through a convolutional network. This allows for further compensation of pose errors and residual deviations caused by dynamic target motion on the basis of geometric alignment. By constructing a teacher-student auxiliary supervision mechanism based on geometric grid sampling, the convolutional alignment module can quickly converge to a reasonable alignment state in the early stages of training, while preserving space for learning better alignment methods. Three-dimensional axial attention is used to efficiently model the temporal and spatiotemporal features of the time series BEV, which significantly reduces computational complexity compared to global self-attention. At the same time, it captures long-distance dependencies in the time and space dimensions, achieving adaptive adjustment of the contribution of historical information. The "output only the current frame" strategy is adopted, using historical frames as contextual information to participate in attention modeling without independently outputting results. This improves the detection performance of the current frame without increasing the output branches, keeping the system structure simple. In the inference stage, geometric grid sampling is no longer called, relying only on convolutional alignment and axial attention, making it suitable for real-time operation on resource-constrained vehicle platforms.

[0030] Example 2: This embodiment provides a multi-frame bird's-eye view point cloud target detection system to implement the 3D target detection method with learnable pose alignment and 3D axial attention described in Embodiment 1, including: Receive continuously acquired multi-frame bird's-eye view point cloud data, wherein the multi-frame point cloud data includes at least the current frame point cloud and one or more historical frame point clouds. The data acquisition module is used to perform voxel division and feature encoding on each frame of point cloud to obtain the corresponding single-frame bird's-eye view BEV feature map. The single-frame feature extraction module is used to construct a two-dimensional SE(2) pose transformation matrix from each historical frame to the current frame based on the vehicle pose information of the carrier within a preset time window, and extract rotation and translation parameters from the pose transformation matrix, and encode the rotation and translation parameters into a spatially distributed motion feature map. The convolutional alignment module is used to concatenate the BEV feature maps of the point clouds of each historical frame with the corresponding motion feature maps in the channel dimension and input them into the pre-built learnable convolutional alignment module, and output the historical frame BEV feature maps aligned with the coordinate system of the current frame. The temporal feature construction module is used to stack the current frame BEV feature map and the aligned historical frame BEV feature maps along the time dimension to form a temporal BEV feature body. The three-dimensional axial attention module inputs the temporal BEV feature body into the three-dimensional axial attention module, and performs axial attention operations sequentially or alternately along the time axis and spatial axis to obtain the temporally fused BEV feature body; The detection head module is used to select the BEV feature map corresponding to the current frame time step from the temporally fused BE feature body, input it into the pre-constructed target detection head, and output the target category, three-dimensional position, size and orientation parameters of the current frame. The Geometric Alignment Teacher Module is used to generate teacher alignment features through geometric raster sampling; The loss calculation module is used to calculate the sum of the main loss for object detection and the auxiliary loss for convolution alignment.

[0031] It should be noted that during the system's training phase, the teacher BEV features obtained through geometric grid sampling alignment are used as the teacher signal. An auxiliary self-supervised loss is applied to the student BEV features output by the learnable convolutional alignment module, achieving a combination of geometric prior constraints and learnable alignment. The weight of the auxiliary loss can be gradually decayed during the training phase or set to zero after a preset training epoch to avoid over-constraining the model in the later stages of training and limiting its ability to learn better alignment strategies. During the system's inference phase, the geometric alignment teacher module and the convolutional alignment auxiliary loss are turned off. The system only retains the learnable convolutional alignment module and the 3D axial attention module. This design results in low computational overhead and high operating efficiency in the inference process, with reasonable hardware resource requirements, making it very suitable for deployment on resource-constrained in-vehicle edge computing platforms to meet the real-time perception needs of autonomous driving.

[0032] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0033] For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances. When an element is referred to as being "assembled on," "mounted on," "fixed to," or "set on" another element, it may be directly on the other element or there may be an intermediate element present. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element present. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible embodiments.

[0034] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0035] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

Claims

1. A 3D target detection method capable of learning pose alignment and 3D axial attention, characterized in that, Includes the following steps: Receive continuously acquired multi-frame bird's-eye view point cloud data, wherein the multi-frame point cloud data includes at least the current frame point cloud and one or more historical frame point clouds. Each frame of point cloud is divided into voxels and its features are encoded to obtain the corresponding single-frame bird's-eye view BEV feature map. Based on the vehicle's pose information within a preset time window, a two-dimensional SE(2) pose transformation matrix is ​​constructed from each historical frame to the current frame, and rotation and translation parameters are extracted from the pose transformation matrix. The rotation and translation parameters are then encoded into a spatially distributed motion feature map. The BEV feature maps of the point clouds of each historical frame are concatenated with the corresponding motion feature maps in the channel dimension and then input into a pre-built learnable convolutional alignment module, which outputs historical frame BEV feature maps aligned with the coordinate system of the current frame. Stack the current frame BEV feature map and the aligned historical frame BEV feature maps along the time dimension to form a temporal BEV feature body; The temporal BEV feature body is input into a pre-constructed three-dimensional axial attention module, and axial attention operations are performed sequentially or alternately along the time axis and spatial axis to obtain the temporally fused BEV feature body; Select the BEV feature map corresponding to the current frame time step from the temporally fused BE feature body, input it into the pre-constructed target detection head, and output the target category, three-dimensional position, size and orientation parameters of the current frame.

2. The 3D target detection method with learnable pose alignment and 3D axial attention according to claim 1, characterized in that: The multi-frame bird's-eye view point cloud data is continuously collected within a preset time window by a carrier equipped with a three-dimensional sensor.

3. The 3D target detection method with learnable pose alignment and 3D axial attention according to claim 1, characterized in that: For each frame of point cloud, voxel segmentation and feature encoding are performed to obtain the corresponding single-frame bird's-eye view BEV feature map, as follows: (31) Divide the three-dimensional space into uniform grid cylinders on the horizontal plane, and assign the points in each frame of the point cloud to the corresponding grid cylinders according to their spatial coordinates; (32) Enhance the point cloud data in each non-empty grid cylinder, calculate the offset of each point relative to the geometric center of the grid cylinder and the offset relative to the group center, and form the expanded point feature vector; (33) Use a multilayer perceptron to extract features from the expanded point feature vectors, and combine batch normalization and nonlinear activation functions to obtain high-dimensional point features; (34) Perform max pooling on all point features within each grid cylinder to extract the global features of the grid cylinder and generate a pseudo image composed of grid cylinder features, namely the single-frame bird's-eye view BEV feature map.

4. The 3D target detection method with learnable pose alignment and 3D axial attention according to claim 3, characterized in that: Construct the two-dimensional SE(2) pose transformation matrix from each historical frame to the current frame, as follows: (41) Represent the vehicle pose of the current frame and the historical frame as four-dimensional homogeneous transformation matrices, denoted as the current pose matrix and the historical pose matrix, respectively. (42) Calculate the inverse of the current pose matrix and multiply it with the historical pose matrix to obtain the relative pose matrix, as follows: Let the vehicle pose matrix of the current frame be... The vehicle pose matrix of the historical frame is Calculate the relative pose matrix from the current frame to the historical frames. : in, The matrix representing the inverse of the current pose matrix; To describe the transformation relationship between the historical frame coordinate system and the current frame coordinate system Homogeneous transformation matrix; (43) Extract the translation components on the horizontal plane and the rotation angles around the vertical axis from the relative pose matrix, and construct the corresponding two-dimensional planar rigid body transformation matrix SE(2) based on the translation components and rotation angles.

5. The 3D target detection method with learnable pose alignment and 3D axial attention according to claim 4, characterized in that: The learnable convolutional alignment module includes: Feature building units are used to extract r from the two-dimensional SE(2) pose transformation matrix. 11 , r 12 , r 21 , r 22 , t x , t y Six parameters are used, and each parameter is expanded into a single-channel feature map with the same spatial size as the BEV feature map, which are then stitched together to form a six-channel motion feature map. The feature concatenation unit is used to concatenate the historical frame BEV feature map with the motion feature map in the channel dimension to obtain the input feature tensor. The convolution alignment unit is used to apply at least one layer of two-dimensional convolution, normalization and nonlinear activation operations to the input feature tensor in sequence, and output the historical frame BEV feature map aligned with the current frame coordinate system.

6. The 3D target detection method with learnable pose alignment and 3D axial attention according to claim 5, characterized in that: During the training phase of the learnable convolutional alignment module, a teacher-student auxiliary supervision mechanism based on geometric grid sampling is constructed to enable the learnable convolutional alignment module to quickly converge to a reasonable alignment state in the early stage of training. The specific steps are as follows: (61) The BEV feature map of the historical frame is geometrically transformed by the feature alignment method based on geometric grid sampling to obtain the teacher alignment feature map; (62) The difference between the student alignment feature map output by the learnable convolutional alignment module and the teacher alignment feature map is measured to form an auxiliary loss term; (63) The auxiliary loss term is weighted and summed with the main loss of target detection according to the preset weight, and used as the total training loss for backpropagation, so as to guide the learnable convolutional alignment module to learn the initial alignment capability close to geometric alignment, and further compensate for the pose estimation error and the alignment deviation caused by dynamic targets; wherein, the preset weight gradually decays with the training stage, or is set to zero after the preset training rounds.

7. The 3D target detection method with learnable pose alignment and 3D axial attention according to claim 5, characterized in that: The three-dimensional axial attention module includes a cascaded stack of multiple axial attention units. The input of each axial attention unit is the output of the previous axial attention unit, so as to deepen the modeling capability of temporal and spatial correlation layer by layer. The three-dimensional axial attention unit receives a temporal BEV feature volume of size (C, T, H, W), where C is the number of channels, T is the number of time frames, and H and W are the spatial dimensions. Each layer of the three-dimensional axial attention unit includes the following cascaded components: (71) The time axis attention submodule is used to perform self-attention calculation on the feature sequence on the time axis at a fixed spatial location to model the temporal dependency between multiple frames; (72) Spatial axis attention submodule, used to perform self-attention calculation on feature sequences along the row and column directions at fixed time steps to model long-distance spatial dependencies; The spatial axis attention submodule includes one-dimensional self-attention branches executed along the row direction and along the column direction of the BEV feature map, respectively: The row-direction attention branch is used to input the feature sequences at different row indices as one-dimensional sequences into the self-attention computation under the condition of fixed time steps and column indices; The column-direction attention branch is used to input the feature sequences at different column indices as one-dimensional sequences into the self-attention computation under the condition of fixed time steps and row indices. The outputs of the two branches are superimposed on the input features via residuals. (73) Residual connection and gating unit, used to superimpose axial attention output and input features in residual form, and adjust the attention enhancement intensity through learnable gating parameters; (74) In the one-dimensional self-attention calculation of the time-axis attention submodule and the spatial-axis attention submodule: The query vector is generated based on an unsampling one-dimensional feature sequence to maintain feature resolution and detail representation. The key vector and value vector are generated based on a one-dimensional feature sequence downsampled along the corresponding axis.

8. The 3D target detection method with learnable pose alignment and 3D axial attention according to claim 7, characterized in that: Before performing attention modeling on the temporal BEV feature volume, the three-dimensional axial attention module also includes a channel dimensionality reduction unit, which reduces the number of channels C to a preset bottleneck channel number C'. After completing the operations of the temporal axis attention submodule and the spatial axis attention submodule, as well as the optional feedforward network operation, the channel dimensionality increase unit restores the number of channels from C' to C, thereby reducing the number of parameters and memory overhead in attention calculation.

9. The 3D target detection method with learnable pose alignment and 3D axial attention according to claim 8, characterized in that: The target detection head is a 3D target detection head based on bird's-eye view features, including: The center point prediction branch, size regression branch, and orientation regression branch are used to predict the center position of the target and its height, length, width, and orientation parameters on the BEV plane.

10. A multi-frame bird's-eye view point cloud target detection system, used to implement the three-dimensional target detection method with learnable pose alignment and three-dimensional axial attention as described in any one of claims 1 to 9, characterized in that, include: Receive continuously acquired multi-frame bird's-eye view point cloud data, wherein the multi-frame point cloud data includes at least the current frame point cloud and one or more historical frame point clouds. The data acquisition module is used to perform voxel division and feature encoding on each frame of point cloud to obtain the corresponding single-frame bird's-eye view BEV feature map. The single-frame feature extraction module is used to construct a two-dimensional SE(2) pose transformation matrix from each historical frame to the current frame based on the vehicle pose information of the carrier within a preset time window, and extract rotation and translation parameters from the pose transformation matrix, and encode the rotation and translation parameters into a spatially distributed motion feature map. The convolutional alignment module is used to concatenate the BEV feature maps of the point clouds of each historical frame with the corresponding motion feature maps in the channel dimension and input them into the pre-built learnable convolutional alignment module, and output the historical frame BEV feature maps aligned with the coordinate system of the current frame. The temporal feature construction module is used to stack the current frame BEV feature map and the aligned historical frame BEV feature maps along the time dimension to form a temporal BEV feature body. The three-dimensional axial attention module inputs the temporal BEV feature body into the three-dimensional axial attention module, and performs axial attention operations sequentially or alternately along the time axis and spatial axis to obtain the temporally fused BEV feature body; The detection head module is used to select the BEV feature map corresponding to the current frame time step from the temporally fused BE feature body, input it into the pre-constructed target detection head, and output the target category, three-dimensional position, size and orientation parameters of the current frame. The Geometric Alignment Teacher Module is used to generate teacher alignment features through geometric raster sampling; The loss calculation module is used to calculate the sum of the main loss for object detection and the auxiliary loss for convolution alignment.

Images