Vehicle attitude estimation method based on image and point cloud feature multilayer fusion

Abstract

The invention discloses a vehicle attitude estimation method based on image and point cloud feature multilayer fusion, and mainly solves the problems that an existing vehicle attitude estimation method is low in accuracy and poor in robustness, and a model depends on a large amount of label data. The implementation scheme is as follows: 1) acquiring a point cloud and an image data set; 2) constructing a pose estimation model; 3) constructing a model loss function; 4) training a pose estimation model; and 5) reasoning and obtaining an attitude transformation result. According to the vehicle pose estimation model constructed by the method, through a self-supervised learning mode, the learning cost is reduced, and the generalization migration ability of the model is improved. Through homogeneous feature fusion and heterogeneous feature fusion, full fusion of image features and point cloud features at multiple levels is realized, the accuracy of a vehicle pose estimation result is effectively improved under the condition that the model reasoning time is not increased, and the method has a relatively high practical application value.

Description

Technical Field

[0001] The present invention belongs to the technical field of real-time mapping and positioning, and in particular relates to a vehicle posture estimation method based on multi-layer fusion of image and point cloud features. Background Art

[0002] With the development of unmanned driving technology, real-time mapping and positioning has become one of the core technologies of autonomous driving systems. Vehicle posture estimation is crucial in path planning, obstacle detection and control decision-making, especially in complex dynamic environments. Accurate posture information plays a fundamental role in the stable operation of the system. Traditional posture estimation methods usually rely on a single sensor, such as an inertial measurement unit (IMU) or a monocular camera. However, these methods have limitations in high-precision requirements and complex environments, and cannot fully reflect the changes in dynamic environments, resulting in insufficient accuracy and robustness, especially in scenes with low texture or large lighting changes. In recent years, the fusion technology of laser radar (LiDAR) and camera has received widespread attention. LiDAR provides high-precision spatial information through point cloud data, while cameras can provide rich visual information, especially in object recognition and scene understanding. The fusion of the two can improve the accuracy and robustness of posture estimation. However, most of the existing learning-based pose estimation algorithms are based on supervised learning, which usually requires a large amount of labeled data for training, resulting in extremely high costs for obtaining high-quality labeled data. In addition, the generalization ability of supervised models is relatively poor, and they cannot effectively cope with changes in different environments and scenes, which limits their performance in practical applications. In addition, in the fusion methods of images and point clouds, existing research mainly focuses on fusion at the data level or result level. Fusion at the data level is limited by the heterogeneity of multi-sensor data, and often requires data from different sensors to be converted into a unified space or format, which inevitably discards a large amount of useful information in the process; fusion at the result level is usually to synthesize the results from different sources through weighted averaging and other methods after performing pose estimation separately. This method is prone to instability of the final result, especially when there is large noise or error, the fusion result is greatly affected. Summary of the invention

[0003] The present invention takes into full account the shortcomings of existing methods, and aims to provide a vehicle posture estimation method based on multi-layer fusion of image and point cloud features. By constructing a self-supervised learning method, the posture transformation estimation results are optimized synchronously using image and point cloud data without expensive labels; by exchanging feature channels, the fusion of image and point cloud features at multiple levels is achieved to improve the accuracy of posture estimation.

[0004] 1. Technical Principle

[0005] Current vehicle attitude estimation methods usually rely on single sensor data, which cannot overcome the inherent limitations of a single sensor. For example, vision-based attitude estimation models are easily affected by depth information ambiguity, illumination changes, and dynamic objects, while point cloud-based attitude estimation models are limited by point cloud sparsity and high computational overhead. In order to effectively combine the advantages of image and point cloud sensor data and improve the accuracy and robustness of attitude estimation, the present invention proposes a vehicle attitude estimation method based on multi-layer fusion of image and point cloud features. The method adopts a self-supervised learning method to avoid relying on expensive label data and reduce training costs. Although image and point cloud data are complementary, existing fusion methods still face challenges. Existing research mainly focuses on fusion at the data level or result level. Fusion at the data level will lead to the loss of a large amount of useful information, while fusion at the result level is prone to unstable results when the noise is large or the error is high. In order to solve these problems, the present invention designs a multimodal feature extraction and fusion module, by constructing a homogeneous feature fusion encoder and a heterogeneous feature fusion encoder, and adopting a feature channel exchange method in the feature fusion modules of the two encoders, to efficiently fuse image and point cloud features at different levels. This method extracts and fuses features layer by layer, allowing the features of images and point clouds to complement each other at multiple levels, thereby improving the model's overall understanding of the scene.

[0006] 2. According to the above principle, the present invention is implemented by the following scheme:

[0007] A vehicle posture estimation method based on multi-layer fusion of image and point cloud features comprises the following steps:

[0008] (1) Obtaining point cloud and image datasets; collecting images and point cloud datasets collected by the corresponding unmanned vehicle during a continuous driving process as a training set. The dataset contains N frames of image data {I1, I2, ..., I N}, point cloud data {PC1, PC2, ..., PC N} and the corresponding calibration parameter set, the sampling frequency of the image and point cloud data is the same, and I t and PC t ,{t=1,2,…,N} have the same acquisition timestamp, the image data is collected by a single camera, and the point cloud data is collected by a single rotating lidar; the same method is used to collect the image and point cloud data sets of the corresponding unmanned vehicle during a continuous driving process as the test set.

[0009] (2) Constructing a pose estimation model; the model consists of a data preprocessing module, a depth estimation module, a multimodal feature extraction and fusion module, and a feature alignment and pose refinement module. The specific construction process includes the following steps:

[0010] (2-a) Construct a data preprocessing module to calculate the PC of two consecutive point clouds at t-1 and t t-1 and PC t As input, the PC t-1 and PC t Generate point cloud projection V through cylindrical projection and coordinate transformation respectively t-1 and V t , where the coordinates are transformed from the point cloud coordinate system to the camera coordinate system.

[0011] (2-b) Construct a depth estimation module based on two consecutive frames of image I at t-1 and t t-1 and I t As input, resize the image to H×W and get the resized image and Processing using the depth estimation neural network Monodepth2 and The relative depth map D is obtained respectively t-1 and D t ; Where H and W are positive integers, and H∈[1,10000], W∈[1,10000].

[0012] (2-c) Construct a multimodal feature extraction and fusion module, which consists of a homogeneous feature fusion encoder, a heterogeneous feature fusion encoder, a radar encoder, an image encoder, and a regressor; the adjusted image obtained in step (2-b) is and Splice in the channel dimension and convert the relative depth map D obtained in step (2-b) t-1 and D t The splicing is performed in the channel dimension, and the spliced ​​results are input into the homogeneous feature fusion encoder to obtain the first-stage image features E with the same shape. img,1 and the deep one-stage feature E depth,1 ; Project the point cloud V obtained in step (2-a) t-1 and V t The concatenation is performed in the channel dimension, and then the radar encoder with ResNet-18 as the feature extractor is input. The output of the radar encoder is pooled into the same shape as E using an adaptive average pooling operation. depth,1 Consistent, we get the first-stage feature E of the point cloud lidar,1 ; E depth,1 and E lidar,1 Input heterogeneous features to fusion encoder to obtain deep two-stage features E depth,2 And point cloud two-stage feature E lidar,2 ; E img,1 Input the image encoder composed of two residual convolution blocks to obtain the image two-stage feature E img,2 ; Eimg,2 、E depth,2 and E lidar,2 Use the regressor to estimate their corresponding posture estimation results respectively, and perform weighted summation on these results to obtain the posture transformation result The regressor consists of three fully connected layers; This is the output of the pose estimation model.

[0013] The homogeneous feature fusion encoder and heterogeneous feature fusion encoder in this step are constructed as follows:

[0014] (2-c-1) Construct a homogeneous feature fusion encoder; the encoder consists of P feature fusion modules with the same structure, where P is a positive integer and P∈[1,10000]; the spliced ​​image and the spliced ​​relative depth Figure 1 Input P feature fusion modules in sequence to obtain the first-stage image feature E img,1 and the deep one-stage feature E depth,1 , which is the output of the homogeneous feature fusion encoder.

[0015] (2-c-2) Construct a heterogeneous feature fusion encoder; the encoder consists of Q feature fusion modules with the same structure, where Q is a positive integer and Q∈[1,10000], and the deep one-stage feature E depth,1 And the point cloud first stage feature E lidar,1 Input Q feature fusion modules together to obtain the deep two-stage feature E depth,2 And point cloud two-stage feature E lidar,2 , which is the output of the heterogeneous feature fusion encoder.

[0016] The fusion module constructed in this step (2-c) is composed of convolution layer 1, activation layer, convolution layer 2, batch normalization layer 1 and batch normalization layer 2; the two inputs of the module are sequentially passed through convolution layer 1, activation layer and convolution layer 2 to obtain features corresponding to the two inputs respectively, and the two features are processed by batch normalization layers 1 and 2 respectively and then channel exchange is performed to obtain exchanged features 1 and exchanged features 2, which are the outputs of the feature fusion module.

[0017] (3) Constructing the model loss function; the specific construction process includes the following steps:

[0018] (3-a) Construct image photometric reconstruction loss L img :

[0019]

[0020] where ‖·||1 is the L-1 regularization constraint, is the adjusted image of the tth frame, for The pseudo image of the tth frame obtained after photometric reconstruction (see Zhou, T., Brown, M., Snavely, N., & Lowe, DG (2017). Unsupervised learning of depth and ego-motion from video. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1851-1858).).

[0021] (3-b) Construct point cloud reconstruction loss L LiDAR :

[0022]

[0023] where ||·||2 represents L-2 regularization, p t-1,k Represents the point cloud projection image V t-1 3D points in Indicates that the point cloud projection image V t Point p in t,k The 3D point obtained after the distortion operation, N represents p t-1,k and The number of valid matching pairs in .

[0024] (3-c) Construct feature fusion loss L CE :

[0025]

[0026] where γ m,i,l represents the offset of the i-th exchangeable channel in the output of the m-th batch normalization layer in the l-th feature fusion module, where l, m, i are all positive integers and l∈[1,10000], m∈[1,2], i∈[1,10000].

[0027] (3-d) Construct the total loss function L total :

[0028] L total =L img +L LiDAR +L CE

[0029] (4) training the pose estimation model; using the training set obtained in step (1) to train the pose estimation model constructed in step (2); using the loss function L constructed in step (3) totalCalculate the model loss; during the training process, use the Adam algorithm to update the model parameters and use L-2 regularization as a constraint until the loss value no longer decreases or reaches the maximum number of training times to obtain a trained pose estimation model.

[0030] (5) Input the test set data obtained in step (1) into the trained pose estimation model. The output of the model is This is the final result of the posture transformation.

[0031] In step (2-b), H and W are preferably 192 and 640, respectively.

[0032] In step (2-c-1), P is preferably 4.

[0033] In step (2-c-2), Q is preferably 2.

[0034] In step (3-c), l is preferably 6, m is preferably 2, and i is preferably 512.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] (1) The self-supervised image and point cloud fusion pose estimation framework constructed by the present invention can efficiently process data from multiple sensors, effectively avoiding the pose estimation failure problem caused by the limitations of a single sensor, and achieving accurate pose estimation. At the same time, the framework uses a self-supervised learning method, does not require expensive labels, and reduces the learning cost.

[0037] (2) The multimodal feature extraction and fusion module constructed by the present invention uses a channel exchange strategy to adaptively fuse features from different sensors without significantly increasing the complexity of the model, thereby improving the robustness of the method without affecting the computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flow chart of a vehicle posture estimation method based on multi-layer fusion of image and point cloud features according to an embodiment of the present invention;

[0039] Figure 2 A structural diagram of a pose estimation model constructed in an embodiment of the present invention;

[0040] Figure 3 A structural diagram of a homogeneous feature fusion encoder in the overall model of an embodiment of the present invention;

[0041] Figure 4 A structural diagram of a heterogeneous feature fusion encoder in the overall model of an embodiment of the present invention;

[0042] Figure 5 A structural diagram of a feature fusion module in the overall model of an embodiment of the present invention;

[0043] Figure 6 Comparison of the pose estimation method according to the embodiment of the present invention and other methods on sequence 9 of the KITTIOdometry dataset;

[0044] Figure 7 A comparison chart of the pose estimation method according to an embodiment of the present invention and other methods on sequence 10 of the KITTIOdometry dataset. DETAILED DESCRIPTION

[0045] The specific implementation mode of the present invention is described below:

[0046] Example 1

[0047] Figure 1 The flowchart of the vehicle posture estimation method based on multi-layer fusion of image and point cloud features implemented by the present invention is shown as follows:

[0048] Step 1, obtain point cloud and image datasets:

[0049] The image and point cloud dataset collected by the unmanned vehicle during a continuous driving process is collected as a training set. The dataset contains N frames of image data {I1, I2, …, I N}, point cloud data {PC1, PC2, ..., PC N} and the corresponding calibration parameter set, the sampling frequency of the image and point cloud data is the same, and I t and PC t ,{t=1,2,…,N} have the same acquisition timestamp, the image data is collected by a single camera, and the point cloud data is collected by a single rotating lidar; the same method is used to collect the image and point cloud data sets of the corresponding unmanned vehicle during a continuous driving process as the test set.

[0050] Step 2: The model consists of a data preprocessing module, a depth estimation module, a multimodal feature extraction and fusion module, a feature alignment and posture refinement module. The specific construction process includes the following steps:

[0051] Figure 2 The figure shows a structural diagram of a pose estimation model constructed in an embodiment of the present invention, and the specific steps are as follows:

[0052] (2-a) Constructing data preprocessing module

[0053] Take two consecutive frames of point cloud PC at t-1 and t t-1 and PC t As input, the PC t-1 and PC t Generate point cloud projection V through cylindrical projection and coordinate transformation respectivelyt-1 and V t , where the coordinates are transformed from the point cloud coordinate system to the camera coordinate system.

[0054] (2-b) Constructing the depth estimation module

[0055] Take two consecutive frames of image I at t-1 and t t-1 and I t As input, resize the image to H×W and get the resized image and Processing using the depth estimation neural network Monodepth2 and The relative depth map D is obtained respectively t-1 and D t ; where H and W are 192 and 640 respectively.

[0056] (2-c) Constructing multimodal feature extraction and fusion module

[0057] This module consists of a homogeneous feature fusion encoder, a heterogeneous feature fusion encoder, a radar encoder, an image encoder, and a regressor. The adjusted image obtained in step (2-b) is and Splice in the channel dimension and convert the relative depth map D obtained in step (2-b) t-1 and D t The splicing is performed in the channel dimension, and the spliced ​​results are input into the homogeneous feature fusion encoder to obtain the first-stage image features E with the same shape. img,1 and the deep one-stage feature E depth,1 ; Project the point cloud V obtained in step (2-a) t-1 and V t The concatenation is performed in the channel dimension, and then the radar encoder with ResNet-18 as the feature extractor is input. The output of the radar encoder is pooled into the same shape as E using an adaptive average pooling operation. depth,1 Consistent, we get the first-stage feature E of the point cloud lidar,1 ; E depth,1 and E lidar,1 Input heterogeneous features to fusion encoder to obtain deep two-stage features E depth,2 And point cloud two-stage feature E lidar,2 ; E img,1 Input the image encoder composed of two residual convolution blocks to obtain the image two-stage feature E img,2 ; E img,2 、E depth,2 and E lidar,2 Use the regressor to estimate their corresponding posture estimation results respectively, and perform weighted summation on these results to obtain the posture transformation result The regressor consists of three fully connected layers; This is the output of the pose estimation model.

[0058] The homogeneous feature fusion encoder and heterogeneous feature fusion encoder in this step are constructed as follows:

[0059] (2-c-1) Constructing a homogeneous feature fusion encoder

[0060] Figure 3 The figure shows the structure of the homogeneous feature fusion encoder in the pose estimation model according to the embodiment of the present invention; the encoder is composed of four feature fusion modules with the same structure; the spliced ​​image and the spliced ​​relative depth are Figure 1 Then, the four feature fusion modules are input in sequence to obtain the first-stage feature E of the image. img,1 and the deep one-stage feature E depth,1 , which is the output of the homogeneous feature fusion encoder.

[0061] (2-c-2) Constructing a heterogeneous feature fusion encoder

[0062] Figure 4 The structure diagram of the heterogeneous feature fusion encoder in the pose estimation model according to the embodiment of the present invention is shown. The encoder is composed of two feature fusion modules with the same structure. depth,1 And the point cloud first stage feature E lidar,1 Input two feature fusion modules together to obtain the deep two-stage feature E depth,2 And point cloud two-stage feature E lidar,2 , which is the output of the heterogeneous feature fusion encoder.

[0063] The feature fusion module constructed in this step (2-c) consists of convolution layer 1, activation layer, convolution layer 2, batch normalization layer 1 and batch normalization layer 2.

[0064] Figure 5 The figure shows the structure of the feature fusion module in the feature fusion encoder of the embodiment of the present invention; the two inputs of the module are sequentially passed through the convolution layer 1, the activation layer, and the convolution layer 2 to obtain the features corresponding to the two inputs respectively, and the two features are processed by the batch normalization layer 1 and 2 respectively and then the channels are exchanged to obtain the exchanged feature 1 and the exchanged feature 2, which are the outputs of the feature fusion module.

[0065] Step 3: construct the model loss function. The specific construction process includes the following steps:

[0066] (3-a) Construct image photometric reconstruction loss L img :

[0067]

[0068] where ||·‖1 is the L-1 regularization constraint, is the adjusted image of the tth frame, for The t-th frame pseudo image is obtained after photometric reconstruction.

[0069] (3-b) Construct point cloud reconstruction loss L LiDAR :

[0070]

[0071] where ‖·‖2 represents L-2 regularization, p t-1,k Represented in the point cloud projection V t-1 3D points in Indicates that the point cloud projection image V t Point p in t,k The 3D point obtained after the distortion operation, N represents p t-1,k and The number of valid matching pairs in .

[0072] (3-c) Construct feature fusion loss L CE :

[0073]

[0074] where γ m,i,l represents the offset of the i-th exchangeable channel in the output of the m-th batch normalization layer in the l-th feature fusion module, where l=6, m=2, i=512.

[0075] (3-d) Construct the total loss function L total :

[0076] L total =L img +L LiDAR +L CE

[0077] Step 4: Train the pose estimation model:

[0078] Use the training set obtained in step 1 to train the pose estimation model constructed in step 2; use the loss function L constructed in step 3 total Calculate the model loss; during the training process, use the Adam algorithm to update the model parameters and use L-2 regularization as a constraint until the loss value no longer decreases or reaches the maximum number of training times to obtain a trained pose estimation model.

[0079] Step 5: Infer and obtain the pose transformation result:

[0080] The test set data obtained in step 1 is input into the trained pose estimation model. The output of the model is This is the final result of the posture transformation.

[0081] Example 2

[0082] The method in Example 1 was used to conduct a self-supervised multi-sensor fusion mileage estimation method experiment on the public dataset KITTIOdometry. The experimental operating system was Ubuntu 18.04, based on the PyTorch 1.8.0 framework of CUDA 11.3 and cuDNN 8.2.1, and trained and tested on a personal computer equipped with Intel (R) Xeon (R) W-2123 CPU (3.60 GHz) and NVIDIA GeForce RTX 3090 (24 GB) hardware.

[0083] This embodiment uses the translation error t err (%), rotation error r err The two indicators (deg / 100m) are used to compare the six odometer methods, Monodepth2, SC-SfMLearner, FeatDepth, Zhou et al., MLF-VO, and DeLORA, with the method of the present invention on the KITTIOdometry dataset. err (%) and r err (deg / 100m) is calculated in units of 100 meters, which respectively represent the average translation error of a trajectory length of 100 meters and the average rotation error of a subsequence of a trajectory length of 100 meters. The unit is angle, and the calculation formula is as follows:

[0084]

[0085] In the formula Respectively represent the true value of the translation vector and the true value of the rotation matrix of the i-th frame, denote the translation vector and rotation matrix of the i-th frame calculated by the odometer method, respectively. N denotes the number of frames in the 100-meter subsequence trajectory. ||·|| denotes the Euclidean norm. trace(·) denotes the matrix trace operation.

[0086] The comparison results are shown in Table 1. It can be found that compared with other methods, the present invention can obtain accurate odometer results on multiple sequences of the KITTIOdometry dataset. In all two sequences, three evaluation indicators reach the optimal level.

[0087] Figure 6 , Figure 7The figure shows the pose estimation results comparison of the pose estimation method of the embodiment of the present invention and other methods on sequence 09 and sequence 10 of the KITTI Odometry dataset. The results show that the pose prediction results of the model designed by the present invention are closer to the true value results than the image-based odometer method (MLF-VO) and the point cloud data-based odometer method (DeLORA); the present invention has a smaller error than the pose estimation method based on a single sensor in the visualization results of the entire sequence; the present invention can achieve better results in both the sequence with rich scene features (sequence 09) and the sequence with sparse scene features (sequence 10).

[0088] The above-described embodiments are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made based on the structure and principle of the present invention should be included in the protection scope of the present invention.

[0089] Table 1

[0090]

Claims

1. A vehicle posture estimation method based on multi-layer fusion of image and point cloud features, characterized in that: The following steps are involved: (I) Obtain point cloud and image datasets: The image and point cloud dataset collected by the unmanned vehicle during a continuous driving process is collected as a training set. The dataset contains N frames of image data {I1, I2, …, I N }, point cloud data {PC1, PC2, ..., PC N } and the corresponding calibration parameter set, the sampling frequency of the image and point cloud data is the same, and I t and PC t ,{t=1,2,…,N} have the same acquisition timestamp, the image data is collected by a single camera, and the point cloud data is collected by a single rotating laser radar; the image and point cloud data sets collected by the corresponding unmanned vehicle during a continuous driving process are used as the test set in the same way; (II) Constructing a pose estimation model: The model consists of a data preprocessing module, a depth estimation module, and a multimodal feature extraction and fusion module. The specific construction process includes the following steps: (II-1) Constructing data preprocessing module: Take two consecutive frames of point cloud PC at t-1 and t t-1 and PC t As input, the PC t-1 and PC t Generate point cloud projection V through cylindrical projection and coordinate transformation respectively t-1 and V t , where the coordinates are transformed from the point cloud coordinate system to the camera coordinate system; (II-2) Constructing the depth estimation module: Take two consecutive frames of image I at t-1 and t t-1 and I t As input, resize the image to H×W and get the resized image and Processing using the depth estimation neural network Monodepth2 and The relative depth map D is obtained respectively t-1 and D t ; Where H and W are positive integers, and H∈[1,10000], W∈[1,10000]; (II-3) Construct a multimodal feature extraction and fusion module: This module consists of a homogeneous feature fusion encoder, a heterogeneous feature fusion encoder, a radar encoder, an image encoder, and a regressor; the adjusted image obtained in step (II-2) and Splice in the channel dimension and convert the relative depth map D obtained in step (II-2) t-1 and D t The splicing is performed in the channel dimension, and the spliced ​​results are input into the homogeneous feature fusion encoder to obtain the first-stage image features E with the same shape. img,1 and the deep one-stage feature E depth,1 ; Project the point cloud V obtained in step (II-1) t-1 and V t The concatenation is performed in the channel dimension, and then the radar encoder with ResNet-18 as the feature extractor is input. The output of the radar encoder is pooled into the same shape as E using an adaptive average pooling operation. depth,1 Consistent, we get the point cloud first-stage feature E lidar,1 ; E depth,1 and E lidar,1 Input heterogeneous features to fusion encoder to obtain deep two-stage features E depth,2 And point cloud two-stage feature E lidar,2 ; E img,1 Input the image encoder composed of two residual convolution blocks to obtain the image two-stage feature E img,2 ; E img,2 、E depth,2 and E lidar,2 Use the regressor to estimate their corresponding posture estimation results respectively, and perform weighted summation on these results to obtain the posture transformation result The regressor consists of three fully connected layers; This is the output of the pose estimation model; (III) Construct the model loss function. The specific construction process is as follows: (III-1) Constructing image photometric reconstruction loss L img : where ||·||1 is the L-1 regularization constraint, is the adjusted image of the tth frame, for The pseudo image of the tth frame obtained after photometric reconstruction; (III-2) Constructing point cloud reconstruction loss L LiDAR : where ||·||2 represents L-2 regularization, p t-1,k Represents the point cloud projection image V t-1 3D points in Indicates that the point cloud projection image V t Point p in t,k The 3D point obtained after the distortion operation, N represents p t-1,k and The number of valid matching pairs in ; (III-3) Constructing feature fusion loss L CE : where γ m,i,l represents the offset of the i-th exchangeable channel in the output of the m-th batch normalization layer in the l-th feature fusion module, where l, m, i are all positive integers and l∈[1,10000], m∈[1,2], i∈[1,10000]; (III-4) Constructing the total loss function L total : L total =L img +L LiDAR +L CE (IV) Training pose estimation model: Use the training set obtained in step (I) to train the pose estimation model constructed in step (II); use the loss function L constructed in step (III) total Calculate the model loss; during the training process, use the Adam algorithm to update the model parameters and use L-2 regularization as a constraint until the loss value no longer decreases or reaches the maximum number of training times, and then obtain a trained pose estimation model; (V) Reasoning and obtaining posture transformation results: The test set data obtained in step (I) is input into the trained pose estimation model, and the output T of the model is t t-1 This is the final result of the posture transformation.

2. A vehicle posture estimation method based on multi-layer fusion of image and point cloud features as claimed in claim 1, characterized in that: The homogeneous feature fusion encoder in step (II-3) is composed of P feature fusion modules with the same structure, where P is a positive integer and P∈[1,10000]. The spliced ​​image and the spliced ​​relative depth map are sequentially input into the P feature fusion modules to obtain the first-stage feature E of the image. img,1 and the deep one-stage feature E depth,1 , which is the output of the homogeneous feature fusion encoder.

3. A vehicle posture estimation method based on multi-layer fusion of image and point cloud features as claimed in claim 1, characterized in that: The heterogeneous feature fusion encoder in step (II-3) is composed of Q feature fusion modules with the same structure, where Q is a positive integer and Q∈[1,10000], and the deep one-stage feature E depth,1 And the point cloud first stage feature E lidar,1 Input Q feature fusion modules together to obtain the deep two-stage feature E depth,2 And point cloud two-stage feature E lidar,2 , which is the output of the heterogeneous feature fusion encoder.

4. A vehicle posture estimation method based on multi-layer fusion of image and point cloud features as claimed in claim 1, characterized in that: The feature fusion module is composed of convolution layer 1, activation layer, convolution layer 2, batch normalization layer 1 and batch normalization layer 2; the two inputs of the module are sequentially processed by convolution layer 1, activation layer and convolution layer 2 to obtain features corresponding to the two inputs respectively, and the two features are processed by batch normalization layers 1 and 2 respectively and then channel exchange is performed to obtain exchanged features 1 and exchanged features 2, which are the outputs of the feature fusion module.

Images