Vehicle localization method based on multi-view and multi-scale feature fusion

Abstract

The present invention belongs to the technical field of vehicle positioning, and specifically discloses a vehicle positioning method based on multi-view and multi-scale feature fusion. A multi-view multi-scale pyramid fusion neural network is proposed in the method of the present invention. At the input level of the network, a multi-channel range image RIV projected by a lidar point cloud and a BEV representation generated by a normal distribution transformation are used to complement each other at global and local scales, thereby improving the overall understanding of the scene. Multi-view input effectively alleviates the limitations of a single view and improves the robustness and anti-interference of the model. At the network architecture level, a self-attention mechanism using Mamba is proposed. The present invention can not only make full use of the complementary correlations between multiple views, but also encode and integrate contextual information across multi-scale features. The global descriptor finally generated is feature-rich, thereby improving the accuracy of vehicle positioning.

Description

Technical Field

[0001] The invention belongs to the technical field of vehicle positioning, and relates to a vehicle positioning method based on multi-view and multi-scale feature fusion. Background Art

[0002] Place Recognition (PR) plays a vital role in supporting various applications such as autonomous driving, mobile robotics, and multi-agent mapping in large-scale challenging environments where Global Positioning System (GPS) signals are unavailable. PR (also known as loop closure detection) is the process by which an agent recognizes its current location and matches it with previously visited locations in a database. Specifically, PR is essential for autonomous vehicles to relocalize and achieve loop closure in Simultaneous Localization and Mapping (SLAM) systems, thereby mitigating or even eliminating long-term accumulated errors. To achieve this, PR must remain accurate and robust under different conditions, including long-term changes such as different seasons, weather conditions, and time periods, as well as dynamic changes in the environment.

[0003] In recent years, researchers have published a series of image-based location recognition methods and achieved satisfactory results. However, due to the inherent imaging and field of view limitations of camera sensors, these methods are easily affected by changes in lighting and viewpoint.

[0004] In contrast, Lidar sensors can accurately detect the geometric structure of the environment from a 360-degree angle and are naturally robust to changes in light and viewpoint. Therefore, Lidar-based place recognition has attracted more and more attention and has shown good results. An effective point cloud proxy representation not only provides comprehensive scene description information, but also indirectly provides sufficient space for subsequent steps to reduce model complexity. At present, in terms of the point cloud representation of the lowest point representation (Lidar Place Recognition, LPR) method, it mainly includes three-dimensional point clouds, voxels, normal distribution transformation units, bird's-eye views (Bird's-Eye-View, BEV), and range view images (Range Image View, RIV). However, few people have tried to combine the above different representation methods to explore the complementary advantages of multi-scale features from different perspectives. It is worth noting that there is a lack of research on exploring the geometric and intensity information of point clouds from multiple perspectives at the same time, which provides a potential way for the development of the LPR field.

[0005] Although point-based methods can effectively preserve the geometric information of environmental elements, the large number of points leads to a long inference time. Voxel-based methods face the problem of computational redundancy because the density distribution of point clouds is uneven, resulting in many empty voxels. In addition, the efficiency of 3D convolution decreases as the complexity of point clouds increases. Projection-based methods use top-down or spherical projection techniques to convert point cloud data into BEV and RIV. BEV projection can capture targets at different positions and is particularly suitable for processing targets with large horizontal coverage. However, this method will cause the loss of geometric and height information, affecting the recognition of targets with more height information. In contrast, RIV projection can preserve the distance information of the original point cloud and capture more detailed data. However, due to the near-large and far-small characteristics of point cloud scanning, distant targets are often occluded in RIV.

[0006] To solve these problems, the current mainstream method usually uses multi-projection fusion strategies. These methods combine projection data from different angles to compensate for the information loss caused by a single projection at the input or output stage.

[0007] However, these methods do not completely or fundamentally solve the inherent information loss problem in the projection process, resulting in serious information loss. In addition, the existing methods are also deficient in feature extraction, resulting in the global descriptor generated for vehicle positioning being unable to fully describe the scene information, thus leading to errors in vehicle positioning position recognition. Summary of the invention

[0008] The purpose of the present invention is to propose a vehicle positioning method based on multi-view and multi-scale feature fusion, which represents the original point cloud collected by the lidar sensor on the vehicle as a multi-view input, and finally facilitates the accurate positioning of the vehicle through the proposed vehicle repositioning neural network based on the multi-view multi-scale pyramid fusion neural network structure.

[0009] In order to achieve the above object, the present invention adopts the following technical scheme:

[0010] A vehicle positioning method based on multi-view and multi-scale feature fusion includes the following steps:

[0011] Step 1. First, the original point cloud collected by the lidar sensor on the vehicle is converted into a range view image, a range view intensity image, a bird's-eye view entropy value image, and a bird's-eye view score image through a multi-view input representation, and a dataset is constructed;

[0012] Step 2. Build a vehicle relocalization neural network based on a multi-view multi-scale pyramid fusion neural network structure, which includes a multi-scale feature encoder, a multi-view and multi-scale feature fusion module, and a feature aggregation module;

[0013] The multi-scale feature encoder includes a RIV encoding branch and a BEV encoding branch. The two branches have the same structure and both use a lightweight ResNet-34 network to process input features. The processing of each ResNet-34 network is divided into four stages.

[0014] The output feature maps of the RIV encoding branch and the BEV encoding branch at the corresponding stage are input into the multi-view and multi-scale feature fusion module for feature fusion. The fused feature maps and the original feature maps are connected by residual connection and then input into the next stage.

[0015] Multi-view and multi-scale feature fusion module, including the reshaping connection module, Mamba module, and view decomposition and reshaping module;

[0016] First, the output features from the corresponding stages of the RIV encoding branch and the BEV encoding branch are input into the reshaping connection module for shape reshaping, and the reshaped features are spliced ​​and input into the Mamba module for deep feature fusion;

[0017] The fused features are input into the view decomposition and reshaping module for view separation, separated into RIV features and BEV features, and then reshaped to the same size as their respective inputs, and returned to the RIV encoding branch and the BEV encoding branch accordingly;

[0018] Step 3. Use the data set in step 1 to train the vehicle relocalization neural network built in step 2;

[0019] The input of the RIV encoding branch is the distance view image and the distance view intensity image obtained in step 1, and the input of the BEV encoding branch is the bird's-eye view entropy image and the bird's-eye view score image obtained in step 1;

[0020] Step 4. Input the point cloud data of the current vehicle collected by the lidar sensor and preprocess it using the method in step 1. Then input the preprocessed result into the trained vehicle relocation neural network to obtain the vehicle positioning result.

[0021] The present invention has the following advantages:

[0022] As described above, the present invention relates to a vehicle positioning method based on multi-view and multi-scale feature fusion, in which a novel multi-view multi-scale pyramid fusion neural network MPTF-Net is proposed to realize the positioning of intelligent vehicles. At the input level of MPTF-Net, the present invention uses the multi-channel distance image RIV projected by the lidar point cloud, and the BEV representation generated by the Normal Distribution Transform (NDT) (the BEV representation captures the local statistical distribution characteristics of the point cloud and provides robustness to local noise interference). RIV provides rich depth and boundary features, while BEV (bird's eye view) provides local structural complexity (entropy value) and target confidence (score) through NDT, which can complement each other at the global and local scales, thereby improving the overall understanding of the scene. Multi-view input can effectively alleviate the limitations of a single view and improve the robustness and anti-interference of the model. In complex scenes or when there are perceptual anomalies (such as occlusion, dynamic objects, and sparse areas of point clouds), the network can flexibly utilize information from different perspectives and modalities to avoid decision-making errors caused by single-perspective input, thereby achieving more accurate vehicle positioning. In addition, at the network architecture level, the MPTF-Net proposed in the present invention proposes a self-attention mechanism using Mamba for deep feature fusion. The present invention can not only make full use of the complementary correlations between multiple views, but also encode and integrate contextual information across multi-scale features, thereby enriching the features of the global descriptor finally generated for vehicle positioning and improving the accuracy of vehicle positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a structural diagram of a multi-view multi-scale pyramid fusion neural network MPTF-Net in an embodiment of the present invention;

[0024] Figure 2 This is a network structure diagram of the first convolutional block of the ResNet-34 network in an embodiment of the present invention;

[0025] Figure 3 : is a network structure diagram of the second convolutional block of the ResNet-34 network in an embodiment of the present invention;

[0026] Figure 4 : is a network structure diagram of the third convolutional block of the ResNet-34 network in an embodiment of the present invention;

[0027] Figure 5 : is a network structure diagram of the fourth convolutional block of the ResNet-34 network in an embodiment of the present invention;

[0028] Figure 6: is a network structure diagram of a multi-view and multi-scale feature fusion module in an embodiment of the present invention;

[0029] Figure 7 A network structure diagram of a vertical compression module in an embodiment of the present invention;

[0030] Figure 8 4 is a network structure diagram of the Mamba module in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0032] Example 1

[0033] This embodiment 1 describes a vehicle positioning method based on multi-view and multi-scale feature fusion, in which a multi-view multi-scale pyramid fusion neural network, called MPTF-Net, is proposed, which aims to effectively integrate RIV and BEV representations containing point cloud geometry and intensity cues to generate a unique global descriptor. Specifically, in terms of the input representation of the network, the present invention mainly uses normal distribution transformation to explore the geometry and intensity cues of the point cloud to enhance the resistance to the influence of local environmental changes. In terms of feature encoding, the present invention introduces a multi-scale pyramid fusion MPTF module to fuse the multi-scale local features encoded by the RIV and BEV convolutional neural network branches, and learns the potential interactive correlations between them to enhance the richness and descriptiveness of the features. For feature aggregation, the present invention introduces context-gated assisted NextVLAD to aggregate and generate a global descriptor with discriminative and viewpoint-invariant properties.

[0034] like Figure 1 As shown, the vehicle positioning method based on multi-view and multi-scale feature fusion includes the following steps:

[0035] Step 1. First, the original point cloud collected by the lidar sensor on the vehicle is converted into range view image, range view intensity image, bird's-eye view entropy value image and bird's-eye view score image through multi-view input representation, and a dataset is constructed.

[0036] Since most existing LPR methods rely on single-view input of point clouds, such as RIV generated by spherical projection or BEV generated by top-down projection, these methods are more or less constrained by the data loss caused by projection.

[0037] Therefore, different from the conventional RIV representation, the RIV representation adopted by the present invention not only consists of a dual-channel representation consisting of distance and intensity, but also densely reconstructs them to alleviate the impact of sparsity or noise existing in RIV.

[0038] Specifically, the process of obtaining the distance view image and the distance view intensity image is as follows:

[0039] First, the original point cloud (3D point cloud) collected by the lidar sensor on the vehicle is converted into a range image through spherical projection, and the spatial point in each point cloud is converted into image coordinates to obtain a range view image.

[0040] The formula for converting each spatial point in the point cloud into image coordinates is as follows:

[0041] ( u v ) = ( 1 2 [ 1 − a r c t a n ( y , x ) π − 1 ] ω [ 1 − ( a r c s i n ( z r − 1 ) + f u p ) f − 1 ] h ) ;

[0042] Among them, (u, v) represents the coordinates of the point in the distance view image, (x, y, z) represents the spatial position of the point in the point cloud, and r represents the distance from the origin in the original point cloud to the midpoint of the point cloud. represents the upper viewing angle, and f represents the full viewing angle.

[0043] In urban driving scenes, most objects are located on the ground plane. Therefore, the most straightforward approach is to generate interpolated points based on vertically adjacent pixels to supplement the range image. However, due to the inevitable occlusion between background and foreground objects, linear interpolation may result in interpolated pixels that are far away in the actual source point cloud and cannot accurately represent the object surface.

[0044] To address this challenge, the present invention additionally retrieves the corresponding intensity values ​​of two neighboring pixels in the range view image.

[0045] In the range view image, the intensity values ​​of two adjacent pixels are retrieved in turn, and the difference between them is calculated; if the intensity difference between adjacent pixels is less than the preset threshold range, the pixels are considered to belong to the same foreground object, thereby avoiding occlusion between the background and foreground objects and minimizing the information loss caused by occlusion.

[0046] Based on this judgment result, the pixels of the foreground object are marked as high intensity values, while the background pixels are marked as low intensity values; in this way, a distance view intensity image is generated, thereby effectively distinguishing the foreground object and the background area.

[0047] When projecting point clouds onto bird's-eye views, there will be serious loss of the height and geometric structure information of the object. Most existing methods use a combination of multi-layer perceptrons and pooling operations to extract preliminary feature information from bird's-eye views. This method cannot fully exploit and utilize bird's-eye views.

[0048] Taking into account the variation of point cloud density with distance, the present invention firstly divides the point cloud evenly into sector polar units, each unit is respectively composed of a ring and a sector along the radial and azimuthal directions, and a plurality of sector grid units are divided.

[0049] Traditional methods usually characterize LiDAR scans through coarse statistics, such as maximum height, intensity, and number of points per grid cell. However, over-reliance on statistical local extremes to represent point clouds limits their effectiveness in dealing with local environmental variations and noise. Unlike traditional statistics, the present invention uses normal distribution to characterize the local distribution of geometric and intensity information within a sector grid cell.

[0050] Specifically, the present invention uses two metrics, probability density score and entropy, to describe all points in the fan-shaped grid unit, and draws a bird's-eye view entropy value image and a bird's-eye view score image according to the probability density score and the entropy value.

[0051] The entropy value represents the complexity of the structure of the point cloud in the grid. The higher the entropy value, the more complex the structure. The score represents the density of the point cloud. A high score means a more concentrated point cloud. The probability density score calculation formula is as follows:

[0052] ;

[0053] In the formula, represents the probability density score, is the number of points in the fan-shaped grid unit, is a point in the point cloud, , are the mean and covariance of the midpoints of the fan-shaped grid cells, respectively.

[0054] The calculation formula of entropy is as follows:

[0055] ;

[0056] in, Represents the entropy value, represents the mean of all points, represents the covariance, and d represents the specified dimension.

[0057] The two images of RIV are concatenated together in the channel dimension as the input of the RIV encoding branch described below. At the same time, the two images of BEV are concatenated together in the channel dimension as the input of the BEV encoding branch described below to construct the dataset.

[0058] Step 2. Build a vehicle relocalization neural network based on a multi-view multi-scale pyramid fusion neural network (MPTF-Net for short). The network structure is as follows: Figure 1 As shown in the figure, the MPTF-Net network includes a multi-scale feature encoder, a multi-view and multi-scale feature fusion module MPTF, and a feature aggregation module.

[0059] In large-scale image classification tasks, ResNet performs well, and its unique residual structure helps learn more effective and discriminative image features. The introduction of residual connections alleviates the gradient vanishing problem, making deep networks easier to train and converge faster. Based on this consensus, the present invention adopts the residual block of ResNet as the encoding backbone of the RIV and BEV encoding branches.

[0060] The multi-scale feature encoder includes a RIV encoding branch and a BEV encoding branch. The two branches have the same structure and both use a lightweight ResNet-34 network to process input features.

[0061] The input of the RIV encoding branch is the distance view image and the distance view intensity image obtained in step 1, and the input of the BEV encoding branch is the bird's-eye view entropy image and the bird's-eye view score image obtained in step 1.

[0062] Specifically, the distance view image and the distance view intensity image are concatenated and used as the input of the RIV coding branch, and the bird's-eye view entropy image and the bird's-eye view score image are concatenated and used as the input of the BEV coding branch.

[0063] The processing of each ResNet-34 network is divided into four stages, defined as the first, second, third, and fourth stages. Each stage includes several convolution operations and activation functions, which are described in detail below.

[0064] The output feature maps of the corresponding stages of the RIV encoding branch and the BEV encoding branch are input into the multi-view and multi-scale feature fusion module for feature fusion. The fused feature maps are connected to the original feature maps through residual connection and then input into the next stage.

[0065] Take the output feature map of the first stage of the RIV encoding branch and the BEV encoding branch as an example:

[0066] The RIV features output by the first stage of the RIV encoding branch and the BEV features output by the first stage of the BEV encoding branch are input into the multi-view and multi-scale feature fusion module for feature fusion. The specific fusion process will be described in detail below.

[0067] The fused feature map includes the fused RIV features and the fused BEV features.

[0068] The fused RIV features and the output feature map of the first stage of the RIV encoding branch are input into the second stage of the RIV encoding branch after residual connection; similarly, the fused BEV features and the output feature map of the first stage of the BEV encoding branch are input into the second stage of the BEV encoding branch after residual connection.

[0069] The following is combined with Figure 2 To Attachment Figure 4 , describe in detail the structure diagram and processing flow of the ResNet-34 network.

[0070] The ResNet-34 network consists of one convolutional layer, one pooling layer, and four convolutional blocks.

[0071] Four convolution blocks are defined as the first, second, third, and fourth convolution blocks, which correspond to one of the four stages of ResNet-34 network processing respectively.

[0072] Specifically, the first convolution block corresponds to the first stage of ResNet-34 network processing, the second convolution block corresponds to the second stage of ResNet-34 network processing, the third convolution block corresponds to the third stage of ResNet-34 network processing, and the fourth convolution block corresponds to the fourth stage of ResNet-34 network processing.

[0073] The convolution layer uses 64 3×3 convolution kernels with an input dimension of The feature map of is convolved with a step size of 1 and a dilation rate of 1. After BN and ReLU activation, the dimension is feature map.

[0074] in, represents the height of the feature, Indicates the width of the feature map.

[0075] The pooling layer uses a 4×4 pooling kernel to perform maximum pooling on the feature map output by the convolutional layer, with a step size of 4, and the resulting dimension is feature map.

[0076] like Figure 2 As shown in the figure, the first convolution block uses 6 layers of 64 3×3 convolution kernels to convolve with the feature map output by the pooling layer, and connects them through the residual structure to obtain a dimension of feature map.

[0077] The six convolutional layers of the first convolutional block are defined as convolutional layers 1-1, 1-2, 1-3, 1-4, 1-5, and 1-6.

[0078] Convolutional layer 1-1 uses 64 3×3 convolution kernels to perform convolution with the feature map output by the pooling layer, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is Feature map.

[0079] Convolutional layer 1-2 uses 64 3×3 convolution kernels to convolve with the feature map output by convolutional layer 1-1, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0080] The feature map output by convolution layer 1-2 is first connected to the input feature map of convolution layer 1-1 through a residual structure, and then used as the input feature map of convolution layer 1-3.

[0081] Convolutional layers 1-3 use 64 3×3 convolution kernels to convolve with the input feature maps of convolutional layers 1-3, with a step size of 1 and a dilation rate of 1. After ReLU activation, the resulting dimension is feature map.

[0082] Convolutional layers 1-4 use 64 3×3 convolution kernels to convolve with the feature maps output by convolutional layers 1-3, with a step size of 1 and a dilation rate of 1. After ReLU activation, the resulting dimension is feature map.

[0083] The feature map output by convolutional layer 1-4 is first connected to the input feature map of convolutional layer 1-3 through a residual structure, and then used as the input feature map of convolutional layer 1-5.

[0084] Convolutional layers 1-5 use 64 3×3 convolution kernels to perform convolutions with the input feature maps of convolutional layers 1-5, with a step size of 1 and a dilation rate of 1. After ReLU activation, the resulting dimension is feature map.

[0085] Convolutional layers 1-6 use 64 3×3 convolution kernels to convolve with the feature maps output by convolutional layers 1-5, with a step size of 1 and a dilation rate of 1. After ReLU activation, the resulting dimension is feature map.

[0086] The feature map output by convolutional layers 1-6 is first connected to the input feature map of convolutional layers 1-5 through a residual structure, and then used as the feature map output by the first convolutional block.

[0087] like Figure 3 As shown, the second convolutional block uses 8 layers of 128 3×3 convolutional kernels to convolve with the feature map output by the first convolutional block, and connects them through the residual structure to obtain a dimension of feature map.

[0088] The 8 convolutional layers of the second convolutional block are defined as convolutional layers 2-1, 2-2, 2-3, 2-4, 2-5, 2-6, 2-7, and 2-8.

[0089] Convolutional layer 2-1 uses 128 3×3 convolution kernels to convolve with the feature map output by the first convolutional block, with a step size of 2 and a dilation rate of 0.5. After ReLU activation, the dimension is The feature map of

[0090] Convolutional layer 2-2 uses 128 3×3 convolution kernels to perform convolution with the feature map output by convolutional layer 2-1, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0091] The feature map output by the convolution layer 2-2 is first connected to the input feature map of the convolution layer 2-1 through a residual structure, and then used as the input feature map of the convolution layer 2-3.

[0092] Convolutional layer 2-3 uses 128 3×3 convolution kernels to convolve with the input feature map of convolutional layer 2-3, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0093] Convolutional layer 2-4 uses 128 3×3 convolution kernels to convolve with the feature map output by convolutional layer 2-3, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0094] The feature map output by the convolutional layer 2-4 is first connected to the input feature map of the convolutional layer 2-3 through a residual structure, and then used as the input feature map of the convolutional layer 2-5.

[0095] Convolutional layers 2-5 use 128 3×3 convolution kernels to perform convolutions with the input feature maps of convolutional layers 2-5, with a step size of 1 and a dilation rate of 1. After ReLU activation, the resulting dimension is feature map.

[0096] Convolutional layers 2-6 use 128 3×3 convolution kernels to convolve with the feature map output by convolutional layers 2-5, with a step size of 1 and a dilation rate of 1. After ReLU activation, the resulting dimension is feature map.

[0097] The feature map output by the convolutional layer 2-6 is first connected to the input feature map of the convolutional layer 2-5 through a residual structure, and then used as the input feature map of the convolutional layer 2-7.

[0098] Convolutional layers 2-7 use 128 3×3 convolution kernels to perform convolution with the input feature map of convolutional layers 2-7, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0099] Convolutional layer 2-8 uses 128 3×3 convolution kernels to convolve with the feature map output by convolutional layer 2-7, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0100] The feature map output by convolution layer 2-8 is first connected to the input feature map of convolution layer 2-7 through a residual structure, and then used as the feature map output by the second convolution block.

[0101] like Figure 4 As shown, the third convolution block uses 12 layers of 256 3×3 convolution kernels to convolve with the feature map output by the second convolution block, and connects them through the residual structure to obtain a dimension of feature map.

[0102] The 12 convolutional layers of the third convolutional block are defined as convolutional layers 3-1, 3-2, 3-3, 3-4, 3-5, 3-6, 3-7, 3-8, 3-9, 3-10, 3-11, and 3-12.

[0103] Convolutional layer 3-1 uses 256 3×3 convolution kernels to convolve with the feature map output by the second convolution block, with a step size of 2 and a dilation rate of 0.5. After ReLU activation, the dimension is feature map.

[0104] Convolutional layer 3-2 uses 256 3×3 convolution kernels to perform convolution with the feature map output by convolutional layer 3-1, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0105] The feature map output by the convolution layer 3-2 is first connected to the input feature map of the convolution layer 3-1 through a residual structure, and then used as the input feature map of the convolution layer 3-3.

[0106] Convolutional layer 3-3 uses 256 3×3 convolution kernels to perform convolution with the input feature map of convolutional layer 3-3, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0107] Convolutional layer 3-4 uses 256 3×3 convolution kernels to convolve with the feature map output by convolutional layer 3-3, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0108] The feature map output by the convolution layer 3-4 is first connected to the input feature map of the convolution layer 3-3 through a residual structure, and then used as the input feature map of the convolution layer 3-5.

[0109] Convolutional layers 3-5 use 256 3×3 convolution kernels to perform convolution with the input feature map of convolutional layers 3-5, with a step size of 2 and a dilation rate of 0.5. After ReLU activation, the dimension is feature map.

[0110] Convolutional layers 3-6 use 256 3×3 convolution kernels to convolve with the feature map output by convolutional layer 3-5, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0111] The feature map output by the convolution layer 3-6 is first connected to the input feature map of the convolution layer 3-5 through a residual structure, and then used as the input feature map of the convolution layer 3-7.

[0112] Convolutional layers 3-7 use 256 3×3 convolution kernels to perform convolution with the input feature map of convolutional layers 3-7, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0113] Convolutional layer 3-8 uses 256 3×3 convolution kernels to convolve with the feature map output by convolutional layer 3-7, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0114] The feature map output by the convolutional layer 3-8 is first connected to the input feature map of the convolutional layer 3-7 through a residual structure, and then used as the input feature map of the convolutional layer 3-9.

[0115] Convolutional layers 3-9 use 256 3×3 convolution kernels to perform convolution with the input feature map of convolutional layers 3-9, with a step size of 2 and a dilation rate of 0.5. After ReLU activation, the dimension is feature map.

[0116] Convolutional layer 3-10 uses 256 3×3 convolution kernels to convolve with the feature map output by convolutional layer 3-9, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0117] The feature map output by the convolution layer 3-10 is first connected to the input feature map of the convolution layer 3-9 through a residual structure, and then used as the input feature map of the convolution layer 3-11.

[0118] Convolutional layer 3-11 uses 256 3×3 convolution kernels to perform convolution with the input feature map of convolutional layer 3-11, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0119] Convolutional layer 3-12 uses 256 3×3 convolution kernels to convolve with the feature map output by convolutional layer 3-11, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0120] The feature map output by the convolution layer 3-12 is first connected to the input feature map of the convolution layer 3-11 through a residual structure, and then used as the feature map output by the third convolution block.

[0121] like Figure 5 As shown in the figure, the fourth convolutional block uses 6 layers of 512 3×3 convolutional kernels to convolve with the feature map output by the third convolutional block, and connects them through the residual structure to obtain a dimension of feature map.

[0122] The six convolutional layers of the fourth convolutional block are defined as convolutional layers 4-1, 4-2, 4-3, 4-4, 4-5, and 4-6.

[0123] Convolutional layer 4-1 uses 512 3×3 convolution kernels to perform convolution with the feature map output by the third convolution block, with a step size of 2 and a dilation rate of 0.5. After ReLU activation, the dimension is feature map.

[0124] Convolutional layer 4-2 uses 512 3×3 convolution kernels to convolve with the feature map output by convolutional layer 4-1, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0125] The feature map output by the convolution layer 4-2 is first connected to the input feature map of the convolution layer 4-1 through a residual structure, and then used as the input feature map of the convolution layer 4-3.

[0126] Convolutional layer 4-3 uses 512 3×3 convolution kernels to perform convolution with the input feature map of convolutional layer 4-3, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0127] Convolutional layer 4-4 uses 512 3×3 convolution kernels to convolve with the feature map output by convolutional layer 4-3, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0128] The feature map output by the convolution layer 4-4 is first connected to the input feature map of the convolution layer 4-3 through a residual structure, and then used as the input feature map of the convolution layer 4-5.

[0129] Convolutional layer 4-5 uses 512 3×3 convolution kernels to convolve with the input feature map of convolutional layer 4-5, with a step size of 1 and a dilation rate of 1. After ReLU activation, the dimension is feature map.

[0130] Convolutional layers 4-6 use 512 3×3 convolution kernels to convolve with the feature map output by convolutional layers 4-5, with a step size of 1 and a dilation rate of 1. After ReLU activation, the resulting dimension is feature map.

[0131] The feature map output by the convolutional layer 4-6 is first connected to the input feature map of the convolutional layer 4-5 through a residual structure, and then used as the feature map output by the fourth convolutional block.

[0132] Considering that shallow network features are better at capturing rich texture information, while deep features are better at accurately transmitting semantic information, the present invention proposes a multi-view and multi-scale feature fusion module MPTF.

[0133] This module effectively integrates the features of RIV and BEV branches and achieves comprehensive feature fusion.

[0134] By interactively integrating multi-view features at different scales, the MPTF module can better integrate complementary low-level and high-level features from different perspectives, thereby extracting richer and more descriptive representations to comprehensively understand and describe the environment.

[0135] like Figure 6 As shown in the figure, the multi-view and multi-scale feature fusion module MPTF includes a reshaping connection module, a Mamba module, and a view decomposition and reshaping module. The general processing flow of MPTF is as follows:

[0136] MPTF first reshapes the features from different perspectives (BEV and RIV), then concatenates the reshaped features and inputs them into the Mamba result for further deep feature fusion. The fused features are view-separated and then reshaped to the same size as when they were input into MPTF for output.

[0137] like Figure 1 As shown in the figure, taking the output features of the first stage of the RIV encoding branch and the BEV encoding branch as an example, the output features from the first stage of the RIV encoding branch and the BEV encoding branch are first input into the reshaping connection module for shape reshaping; the reshaped features are spliced ​​and input into the Mamba module for deep feature fusion, and the fused features are input into the view decomposition and reshaping module for view separation, separated into RIV features and BEV features, and then reshaped to the same size as their respective inputs (the RIV features after view separation are reshaped to the same size as the input RIV features, and the BEV features after view separation are reshaped to the same size as the input BEV features), and returned to the RIV encoding branch and the BEV encoding branch respectively.

[0138] The RIV features after MPTF fusion are residually connected with the features output by the first stage of the RIV encoding branch, and serve as the input feature map of the second stage of the RIV encoding branch. Similarly, the BEV features after fusion are residually connected with the features output by the first stage of the BEV encoding branch, and serve as the input feature map of the second stage of the BEV encoding branch.

[0139] The Reshape Connection module can directly adjust the shape of tensors through operations such as reshape, flatten, squeeze, etc.

[0140] The view decomposition and reshaping module maps the fused feature map into two perspective features through a linear layer, then reshapes it to the same size as the input, and finally outputs it after activation by the ReLU activation function.

[0141] like Figure 8 As shown, the specific processing flow of the Mamba module in this embodiment is as follows:

[0142] First, the input feature map of the Mamba module is passed through a linear layer to obtain A and D respectively.

[0143] Among them, A and D are the feature representations of the input feature map of the Mamba module after different linear transformations. They play the role of initial feature decomposition and benchmark features in the entire module.

[0144] Then feature A undergoes a one-dimensional convolution operation and After the activation function, features X, B, and C are obtained; among them, X, B, and C are the features obtained by further processing A, representing the feature information of a specific pattern or dimension.

[0145] Then, features A, X, B, and C are simultaneously input into the state space layer for fusion, and the fused feature representation Y is obtained, which represents the result after dynamic relationship capture and interaction of multiple features.

[0146] After weighting Y The features D activated by the activation function are combined and then passed through a linear layer to obtain the fused output.

[0147] like Figure 1 As shown, the feature aggregation module includes a vertical compression module, a global descriptor generation module and an ECA module.

[0148] The features after RIV encoding, i.e., the output features of the RIV encoding branch, and the features after BEV encoding, i.e., the output features of the BEV encoding branch, are respectively subjected to a vertical compression operation by a vertical compression module to further extract features and reduce redundant information.

[0149] For features with a visual input dimension of N×C×h×w, splicing is performed along the width dimension, and the feature quantity with a spliced ​​dimension of C×h×Nw is input into the vertical compression module for feature compression operation to obtain compressed features.

[0150] like Figure 7 As shown, the vertical compression module consists of six convolutional layers and one pooling layer.

[0151] The six convolutional layers are defined as the first, second, third, fourth, fifth and sixth convolutional layers.

[0152] The first convolutional layer uses 512 1×1 convolution kernels and the input dimension is The feature map of is convolved with a step size of 1 and a dilation rate of 1. After Sigmoid activation, the dimension is feature map.

[0153] The second convolutional layer uses 512 1×1 convolution kernels and the input dimension is The feature map of is convolved with a step size of 1 and a dilation rate of 1. After Sigmoid activation, the dimension is feature map.

[0154] The pooling layer performs maximum pooling on the feature map output by the second convolutional layer along the height dimension, and the dimension is feature map.

[0155] The third convolutional layer uses 256 3×3 convolution kernels to convolve with the feature map output by the pooling layer, with a step size of 1 and a dilation rate of 1. After BN and ReLU activation, the dimension is feature map.

[0156] The fourth convolutional layer uses 256 3×3 convolution kernels to convolve with the feature map output by the third convolutional layer, with a step size of 1 and a dilation rate of 1. After BN and ReLU activation, the dimension is feature map.

[0157] The fifth convolutional layer uses 256 3×3 convolution kernels to perform convolution with the feature map output by the fourth convolutional layer, with a step size of 1 and a dilation rate of 1. After BN and ReLU activation, the dimension is feature map.

[0158] The sixth convolutional layer uses 256 3×3 convolution kernels to perform convolution with the feature map output by the fifth convolutional layer, with a step size of 1 and a dilation rate of 1. After BN and ReLU activation, the dimension is feature map.

[0159] In the global descriptor generation module, the feature maps output by each vertical compression module are input into a NextVLAD respectively, and pooling is learned in an end-to-end manner to obtain a one-dimensional global descriptor.

[0160] The NextVLAD model structure is an existing technology, specifically derived from the following references:

[0161] Lin R, Xiao J, Fan J. NextVLAD: An efficient neural network to aggregate frame-level features for large-scale video classification[C] / / Proceedings of the European Conference on Computer Vision (ECCV) Workshops.2018: 0-0.

[0162] The one-dimensional global descriptors obtained by RIV after NextVLAD and BEV after NextVLAD are concatenated and sent to ECA attention for further enhancement to obtain the final compact and recognizable feature vector, which is the global descriptor used to realize vehicle positioning. The global descriptor is feature-rich and helps to improve the accuracy of vehicle positioning.

[0163] The processing flow of the ECA module is as follows:

[0164] First, the input feature map of the ECA module is globally averaged pooled to obtain the global feature representation of each channel; then, the globally pooled features are processed through one-dimensional convolution to generate channel attention weights; where this convolution operation uses a convolution layer with a convolution kernel size of K to achieve local cross-channel information interaction between each channel and its K adjacent channels while avoiding information loss caused by dimensionality reduction; then, the ReLU activation function is applied to the convolution weights to adjust the weights; finally, the adjusted weights are multiplied element-by-element with the input feature map of the ECA module to generate a weighted feature map.

[0165] The calculation formula of the convolution kernel size K is as follows:

[0166] ;in express The nearest odd number, C represents the number of channels of the input feature map of the ECA module, and is the coefficient used to calculate the size of the convolution kernel K.

[0167] Step 3. Use the dataset in step 1 to train the vehicle relocalization neural network built in step 2. Train the deep learning network and iterate and optimize the network parameters through stochastic gradient descent.

[0168] During the training process of the vehicle relocation neural network, the following parameters are set:

[0169] Set the size of the LiDAR scan RIV input of the vehicle's lidar sensor to ,For the point cloud BEV size of the dataset, it is uniformly set to ;

[0170] For the RIV encoding branch and the BEV encoding branch, the pre-trained weights from ResNet-34 are used;

[0171] For the multi-view and multi-scale feature fusion modules, the embedding dimensions are set to dmodel = 128, 256, and 512, the number of heads is set to nhead = 4, and the intermediate dimensions of the feed-forward layer are set to dffn = 256, 512, and 1024;

[0172] For NextVLAD, the intermediate feature dimension of NextVLAD is set to 1024, the number of clusters is set to 64, and the output feature dimension is set to 256. Therefore, the size of the global descriptor generated by the method of the present invention is 1×512.

[0173] Step 4. Input the point cloud data of the current vehicle and preprocess it using the processing method of step 1. Then input the preprocessed result into the trained vehicle relocation neural network to obtain the vehicle positioning result.

[0174] Specifically, the global descriptor can compress complex input data into a fixed-length representation. By comparing the global descriptors of the current frame with those of the frames stored in the database, it is possible to identify whether the vehicle's location or scene matches. If the distance between the global descriptors is small enough (less than a preset threshold), the two frames are considered to represent the same location, thereby achieving accurate vehicle positioning.

[0175] In addition, in order to verify the effectiveness of the method proposed in the present invention, the following experiments were carried out.

[0176] To build a point cloud dataset, this example uses the public dataset nuScenes for training and verification, and divides it according to the official dataset division method. The nuScenes dataset covers 1,000 driving scenarios of 20 seconds each, and is equipped with a complete set of autonomous driving sensors, including six cameras, a lidar, and five radars.

[0177] The data was collected in four different areas in Boston and Singapore: Boston Seaport (BS), SG-OneNorth (SON), SG-Queenstown (SQ), and SG-HollandVillage (SHV). The BS subset is the largest, with 550 scenes, while the others contain 215, 135, and 100 scenes respectively.

[0178] In order to train and evaluate the robustness of the location recognition network, the present invention uses the BS dataset for training and tests the generalization performance of the network on the SON dataset. The data collected within 120 days on the BS dataset are divided into a training query set, a verification query set, a test query set, and a database set. Specifically, the data of the first 105 days are used as a training query set, and samples with an interval of 1 meter are selected to construct a database set; then the remaining 15 days of data are divided into a test query set and a verification query set at a ratio of 1:4; finally, those query samples for which there is no corresponding index in the database within a range of 5 meters are removed. After this series of processing, a database set containing 9686 samples, a training query set of 8407 samples, a verification query set of 1002 samples, and a test query set of 3008 samples are finally obtained on the BS dataset. On the SON dataset, there are a database set of 4796 samples and a test query set of 3308 samples, as shown in Table 1 below.

[0179] Table 1 Dataset division

[0180]

[0181] In order to test the network model proposed in the present invention and verify its positioning effect, two representative visual position recognition algorithms NetVLAD and Patch-NetVLAD, two lidar position recognition algorithms PointNetVLAD and OverlapTransformer, and a millimeter-wave radar position recognition algorithm AutoPlace are introduced for comparison.

[0182] The output global descriptor sizes of NetVLAD, Patch-NetVLAD, PointNetVLAD, OverlapTransformer, AutoPlace and the present invention are 9216×1, 256×1, 256×1, 256×1, 9216×1, and 512×1, respectively.

[0183] Recall@1, Recall@5, and Recall@10 are used as evaluation indicators. The comparison results are shown in Table 2.

[0184] Table 2 Comparison results of different methods

[0185]

[0186] It can be seen from Table 2 that the method of the present invention far exceeds other comparison methods in the most important evaluation index Recall@1, and reaches 97.69. This is mainly due to the multi-view input representation and NDT extraction of the present invention, which reduces information loss and extracts more sufficient feature information, thereby improving positioning accuracy.

[0187] Comparison results show that compared with representative single-view and multi-view baseline methods, the global descriptor generated by the present invention exhibits stronger distinguishability on different datasets, while also meeting the requirements of practical online deployment applications.

[0188] The method of the present invention achieves the best accuracy in the robust location recognition task, outperforming all the comparative methods. The unique attempt of the present invention in multi-view and multi-scale feature fusion provides a new idea for further research on location recognition.

[0189] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with the field under the guidance of this specification fall within the essential scope of this specification and should be protected by the present invention.

Claims

1. A vehicle positioning method based on multi-view and multi-scale feature fusion, characterized in that: The steps include: Step 1. First, the original point cloud collected by the lidar sensor on the vehicle is converted into a range view image, a range view intensity image, a bird's-eye view entropy value image, and a bird's-eye view score image through a multi-view input representation, and a dataset is constructed; Step 2. Build a vehicle relocalization neural network based on a multi-view multi-scale pyramid fusion neural network structure, which includes a multi-scale feature encoder, a multi-view and multi-scale feature fusion module, and a feature aggregation module; The multi-scale feature encoder includes a RIV encoding branch and a BEV encoding branch. The two branches have the same structure and both use a lightweight ResNet-34 network to process input features. The processing of each ResNet-34 network is divided into four stages. The output feature maps of the RIV encoding branch and the BEV encoding branch at the corresponding stage are input into the multi-view and multi-scale feature fusion module for feature fusion. The fused feature maps and the original feature maps are connected by residual connection and then input into the next stage. Multi-view and multi-scale feature fusion module, including the reshaping connection module, Mamba module, and view decomposition and reshaping module; First, the output features from the corresponding stages of the RIV encoding branch and the BEV encoding branch are input into the reshaping connection module for shape reshaping, and the reshaped features are spliced ​​and input into the Mamba module for deep feature fusion; The fused features are input into the view decomposition and reshaping module for view separation, separated into RIV features and BEV features, and then reshaped to the same size as their respective inputs, and returned to the RIV encoding branch and the BEV encoding branch accordingly; Step 3. Use the data set in step 1 to train the vehicle relocalization neural network built in step 2; The input of the RIV encoding branch is the distance view image and the distance view intensity image obtained in step 1, and the input of the BEV encoding branch is the bird's-eye view entropy image and the bird's-eye view score image obtained in step 1; Step 4. Input the point cloud data of the current vehicle collected by the lidar sensor and preprocess it using the method in step 1. Then input the preprocessed result into the trained vehicle relocation neural network to obtain the vehicle positioning result.

2. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 1 is characterized in that: In step 1, the process of acquiring the distance view image and the distance view intensity image is as follows: First, the original point cloud collected by the LiDAR sensor on the vehicle is converted into a range image through spherical projection, and the spatial point in each point cloud is converted into image coordinates to obtain a range view image. The formula for converting each spatial point in the point cloud into image coordinates is as follows: ； Among them, (u, v) represents the coordinates of the point in the distance view image, (x, y, z) represents the spatial position of the point in the point cloud, and r represents the distance from the origin in the original point cloud to the midpoint of the point cloud. represents the upper perspective, and f represents the full perspective; In the range view image, the intensity values ​​of two adjacent pixels are retrieved in sequence and the difference between them is calculated; if the intensity difference between adjacent pixels is less than a preset threshold range, the pixels are considered to belong to the same foreground object; Based on this judgment result, the pixels of the foreground object are marked as high intensity values, while the background pixels are marked as low intensity values; in this way, a distance view intensity image is generated, thereby effectively distinguishing the foreground object and the background area.

3. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 1, characterized in that: In step 1, the process of obtaining the bird's-eye view entropy value image and the bird's-eye view score image is as follows: Firstly, the original point cloud collected by the laser radar sensor on the vehicle is evenly divided into sector polar units; each sector polar unit is composed of a ring and a sector along the radial and azimuth directions, respectively, and a plurality of sector grid units are divided; Normal distribution is used to characterize the local distribution of geometric and intensity information within the fan-shaped grid cells; Specifically, the probability density score and entropy are used to describe all the points in the fan-shaped grid unit, and the bird's-eye view entropy value image and the bird's-eye view score image are drawn according to the probability density score and the entropy value. The probability density score calculation formula is as follows: ； In the formula, represents the probability density score, is the number of points in the fan-shaped grid unit, is a point in the point cloud, , are the mean and covariance of the midpoints of the fan-shaped grid cells, respectively; The calculation formula of entropy is as follows: ； in, Represents the entropy value, represents the mean of all points, represents the covariance, and d represents the specified dimension.

4. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 1, characterized in that: The ResNet-34 network includes a convolutional layer, a pooling layer, and four convolutional blocks; Define four convolution blocks as the first, second, third, and fourth convolution blocks, which correspond to one of the four stages of ResNet-34 network processing respectively; The convolution layer uses 64 3×3 convolution kernels with an input dimension of The feature map of is convolved with a step size of 1 and a dilation rate of 1. After the BN normalization layer and ReLU activation, the dimension is The feature map of in, represents the height of the feature, Indicates the width of the feature map; The pooling layer uses a 4×4 pooling kernel to perform maximum pooling on the feature map output by the convolutional layer, with a step size of 4, and the resulting dimension is The feature map of The first convolution block uses 6 layers of 64 3×3 convolution kernels to convolve with the feature map output by the pooling layer, and connects them through a residual structure to obtain a dimension of The feature map of The second convolutional block uses 8 layers of 128 3×3 convolutional kernels to convolve with the feature map output by the first convolutional block, and connects them through a residual structure to obtain a dimension of The feature map of The third convolutional block uses 12 layers of 256 3×3 convolutional kernels to convolve with the feature map output by the second convolutional block, and connects them through a residual structure to obtain a dimension of The feature map of The fourth convolutional block uses 6 layers of 512 3×3 convolutional kernels to convolve with the feature map output by the third convolutional block, and obtains a dimension of feature map.

5. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 1, characterized in that: In step 2, the specific processing flow of the Mamba module is as follows: First, the input feature map of the Mamba module is passed through a linear layer to obtain A and D, where A and D are the feature representations of the input feature map of the Mamba module after different linear transformations; Then feature A undergoes a one-dimensional convolution operation and After the activation function, we get features X, B, and C; Then, features A, X, B, and C are simultaneously input into the state space layer for fusion to obtain the fused feature representation Y. Y is weighted and combined with the The features D activated by the activation function are combined and then passed through a linear layer to obtain the fused output.

6. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 1, characterized in that: The feature aggregation module includes a vertical compression module, a global descriptor generation module and an ECA module; The features after RIV encoding, i.e., the output features of the RIV encoding branch, and the features after BEV encoding, i.e., the output features of the BEV encoding branch, are respectively subjected to a vertical compression operation by a vertical compression module; In the global descriptor generation module, the feature maps output by each vertical compression module are input into a NextVLAD respectively, and the pooling is learned in an end-to-end manner to obtain a one-dimensional global descriptor; The one-dimensional global descriptors obtained by RIV after NextVLAD and BEV after NextVLAD are concatenated and sent to ECA attention for further enhancement to obtain the final global descriptor for vehicle positioning.

7. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 6, characterized in that: The vertical compression module includes six convolutional layers and one pooling layer; Define six convolutional layers as the first, second, third, fourth, fifth and sixth convolutional layers; The first convolutional layer uses 512 1×1 convolution kernels and the input dimension is The feature map of is convolved with a step size of 1 and a dilation rate of 1. After Sigmoid activation, the dimension is The feature map of The second convolutional layer uses 512 1×1 convolution kernels and the input dimension is The feature map of is convolved with a step size of 1 and a dilation rate of 1. After Sigmoid activation, the dimension is The feature map of The pooling layer performs maximum pooling on the feature map output by the second convolutional layer along the height dimension, and the dimension is The feature map of The third convolutional layer uses 256 3×3 convolution kernels to convolve with the feature map output by the pooling layer, with a step size of 1 and a dilation rate of 1. After the BN normalization layer and ReLU activation, the dimension is The feature map of The fourth convolutional layer uses 256 3×3 convolution kernels to convolve with the feature map output by the third convolutional layer, with a step size of 1 and a dilation rate of 1. After BN and ReLU activation, the dimension is The feature map of The fifth convolutional layer uses 256 3×3 convolution kernels to perform convolution with the feature map output by the fourth convolutional layer, with a step size of 1 and a dilation rate of 1. After BN and ReLU activation, the dimension is The feature map of The sixth convolutional layer uses 256 3×3 convolution kernels to perform convolution with the feature map output by the fifth convolutional layer, with a step size of 1 and a dilation rate of 1. After BN and ReLU activation, the dimension is feature map.

8. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 6, characterized in that: The processing flow of the ECA module is as follows: First, the input feature map of the ECA module is globally averaged pooled to obtain the global feature representation of each channel; Next, the globally pooled features are processed through one-dimensional convolution to generate channel attention weights; Among them, this convolution operation uses a convolution layer with a convolution kernel size of K, which avoids information loss caused by dimensionality reduction while realizing local cross-channel information interaction between each channel and its K adjacent channels; Then, the ReLU activation function is applied to the convolution weights to adjust the weights; Finally, the adjusted weights are multiplied element-wise with the input feature map of the ECA module to generate a weighted feature map.

9. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 8, characterized in that: The calculation formula of the convolution kernel size K is as follows: ； in express The nearest odd number, C represents the number of channels of the input feature map of the ECA module, and is the coefficient used to calculate the size of the convolution kernel K.

10. The vehicle positioning method based on multi-view and multi-scale feature fusion according to claim 6, characterized in that: In step 3, the vehicle relocation neural network performs the following parameter settings during the training process: First, the size of the LiDAR scan RIV input on the vehicle is set to ,For the point cloud BEV size of the dataset, it is uniformly set to ; For the RIV encoding branch and the BEV encoding branch, the pre-trained weights from ResNet-34 are used; For the multi-view and multi-scale feature fusion modules, the embedding dimensions are set to dmodel = 128, 256, and 512, the number of heads is set to nhead = 4, and the intermediate dimensions of the feed-forward layer are set to dffn = 256, 512, and 1024; For NextVLAD, the intermediate feature dimension of NextVLAD is set to 1024, the number of clusters of NextVLAD is set to 64, the output feature dimension of NextVLAD is set to 256, and the size of the global descriptor generated by NextVLAD is 1×512.

Images