Dynamic adaptive BEV perception multi-scale feature fusion method

Abstract

The invention relates to the field of intelligent driving based on BEV, in particular to a multi-scale feature fusion method for dynamic adaptive BEV perception. According to the scheme, the method comprises the following steps: data acquisition: synchronously acquiring 360-degree multi-view image data around a vehicle through a plurality of wide-angle cameras deployed around the vehicle body; feature extraction is carried out, and a multi-scale feature map F is extracted from the acquired image data through ResNet or Swin Transform; generating a dynamic adaptive BEV query; pyramid multi-scale feature fusion; performing space-time fusion and BEV representation generation; and decoding the final BEV feature map after time-space fusion into a specific perception result, and outputting the specific perception result. The method is suitable for intelligent driving based on the BEV.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving based on BEV (Bird's-Eye-View), specifically to a multi-scale feature fusion method for dynamic adaptive BEV perception. Background Technology

[0002] Intelligent driving is a core development direction for the automotive industry. Its key lies in accurately perceiving the surrounding environment through sensors to provide a basis for vehicle decision-making, planning, and control. Environmental perception technology integrates data from multiple sensors, such as LiDAR and cameras, to build a three-dimensional understanding of the driving scenario.

[0003] In recent years, BEV perception has become a key technological paradigm in the field of environmental perception. The core of this technology lies in converting independent perspective images captured by multiple onboard cameras into a unified bird's-eye view centered on the vehicle. However, existing BEV-based static perception technologies still have the following shortcomings and deficiencies in practical applications, especially when dealing with complex and changing road scenarios:

[0004] Insufficient scene adaptability and unreasonable allocation of computing resources: Most existing mainstream BEV perception models (such as BEVFormer) use fixed-resolution BEV query grids to represent physical space (e.g., using a 200x200 grid to cover a specific area). This static and uniform grid division method lacks adaptability to scene complexity. In scenarios with simple road structures (such as straight urban roads), overly dense query grids can cause a lot of computational redundancy; while in scenarios with complex road topology (such as interchange ramp weaving areas and construction zones), fixed query granularity may not be fine enough to accurately capture key geometric details such as lane curvature and guide line boundaries, leading to a decrease in perception accuracy.

[0005] In addition, there are problems such as low coupling efficiency of multi-granularity features, easy loss of scene details, and poor perception robustness under complex and extreme conditions. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multi-scale feature fusion method for dynamic adaptive BEV perception, which achieves high-precision and high-robust output of BEV static perception results.

[0007] The present invention achieves the above objectives by adopting the following technical solution: the present invention provides a multi-scale feature fusion method for dynamic adaptive BEV perception, comprising:

[0008] S1, Data Acquisition;

[0009] Synchronously collecting multi-view image data of 360° around the vehicle through multiple wide-angle cameras deployed around the vehicle body;

[0010] S2, feature extraction;

[0011] extracting a multi-scale feature map F from the collected image data through ResNet or Swin Transformer;

[0012] S3, generating a dynamic adaptive BEV query;

[0013] S4, pyramid multi-scale feature fusion;

[0014] S5, spatio-temporal fusion and BEV representation generation;

[0015] S6, decoding the final BEV feature map after spatio-temporal fusion into specific perception results for output.

[0016] Further, step S3 specifically includes:

[0017] Taking the multi-scale feature map F and an initial query Q as input, outputting a query coordinate dynamically adjusted in resolution according to the complexity of the current scene, and the specific process is as follows:

[0018] S301, scene complexity analysis;

[0019] First, a lightweight semantic segmentation network is used to analyze the multi-scale feature map F, extract lane line curvature variance and obstacle distribution entropy features, and linearly combine the lane line curvature variance and obstacle distribution entropy features to generate a scene complexity heat map H corresponding to the BEV plane;

[0020] S302, resolution dynamic adjustment;

[0021] A lightweight gating network composed of an MLP is designed to take the heat map H and the initial query Q as input, and calculate a resolution scaling factor for each grid position, as follows:

[0022] ;

[0023] wherein, the resolution scaling factor is denoted by, the feature concatenation operation is denoted by, the feature value or feature vector corresponding to the heat map H at the grid (i, j) position is denoted by, the initial query Q at the grid (i, j) position is denoted by, the bias term is denoted by, the weight matrix of the last linear transformation layer in the gating network is denoted by;

[0024] S303、Dynamic coordinate mapping;

[0025] According to the calculated scaling factor, the original uniform grid coordinates are nonlinearly transformed by bilinear interpolation to generate adaptive query coordinates, in the following way:

[0026] ;

[0027] In the formula, , respectively represent the preset maximum offset in the x and y directions, which defines the maximum distance that each grid point can move, , respectively represent the horizontal and vertical coordinates of the original uniform grid, , respectively represent the horizontal and vertical coordinates of the query coordinates.

[0028] Further, step S3 specifically further includes:

[0029] S304, semantic constraint K-means++ clustering;

[0030] The multi-view image features are divided into K semantic clusters , and the objective function is optimized to not only consider geometric distance, but also introduce semantic consistency as a constraint term through KL divergence , in the following way:

[0031] ;

[0032] In the formula, represents a set of K cluster centers, represents the center vector of the th cluster, represents the assignment of N data points, represents the index of the cluster to which the nth data point is assigned, represents the feature vector of the nth multi-view image, represents the center point of the cluster assigned to the feature , represents a weight coefficient, represents a function for obtaining the semantic probability distribution of a certain feature vector.

[0033] S305, correlation weight and local attention;

[0034] Each BEV query first calculates the correlation weight with each cluster center Thus, the cluster with high correlation is located in the following manner:

[0035] ;

[0036] wherein, is a temperature coefficient, denotes a center vector of the jth cluster, denotes the ith BEV query vector, and T denotes a transpose.

[0037] Further, the step S4 specifically comprises:

[0038] Taking the multi-scale feature map F as input, a feature map F_fused after multi-granularity information fusion is output, and the specific process is as follows:

[0039] S401, multi-scale features are extracted in parallel;

[0040] The input multi-scale feature map F is divided into three groups along the channel dimension, and is processed in parallel through heterogeneous convolution branches with different receptive fields;

[0041] The high-resolution branch adopts a 3*3 small convolution kernel to capture sub-pixel level lane edge geometric details;

[0042] The mesoscopic branch adopts a 5*5 hollow convolution to model the spatial continuity of the road along and the isolation belt;

[0043] The low-resolution branch adopts a 7*7 large convolution kernel to analyze the macro-topological relationship of the intersection diversion belt.

[0044] S402, dynamic channel attention weighting;

[0045] The multi-scale features processed in parallel are spliced to obtain spliced features, and the spliced features are sent to an SE module. The SE module learns a dynamic weight for each feature channel through global average pooling and two fully connected networks in the following manner:

[0046] ;

[0047] wherein, denotes a dynamic weight, and sigma and delta respectively denote Sigmoid and ReLU activation functions, denotes spliced features, , respectively denote weight matrices of two fully connected layers in the SE module.

[0048] Further, the step S4 specifically further comprises introducing a time sequence continuity constraint based on a hidden Markov model, and specifically comprises:

[0049] BEV feature sequence The temporal evolution model is a state transition process, which takes into account the vehicle's own speed. The method is as follows:

[0050] ;

[0051] In the formula, This represents the bird's-eye view features or state at the current time point t. This represents the bird's-eye view features or state at the previous time point t-1. Indicates a Gaussian distribution. This represents speed multiplied by time, used to estimate the spatial displacement of the BEV feature due to the vehicle's own motion during this time period. This represents the covariance matrix of the normal distribution.

[0052] By using Bayesian theory, the observation information of the current frame is fused. and time-series priors based on historical information The query results for the current frame The optimization can be performed as follows:

[0053] ;

[0054] In the formula, This represents the observation information at the current time point t, that is, the multi-view image data collected by the vehicle-mounted camera at time t. It is a conditional probability, representing the probability under given observation conditions. The next state is The probability of.

[0055] Furthermore, step S5 specifically includes:

[0056] Through a spatial cross-attention mechanism, the dynamic query grid is used as the Query and interacts with the fused feature map F_fused as the Key and Value to aggregate spatial information and generate the BEV feature BEV_t at the current time. Then, the BEV feature BEV_t-1 at the previous time is read, the coordinates are aligned according to the vehicle's own motion, and then fused with BEV_t through a temporal self-attention mechanism.

[0057] Furthermore, step S1 specifically includes:

[0058] Multiple wide-angle cameras deployed around the vehicle body simultaneously collect 360° multi-view image data of the vehicle's surroundings.

[0059] Furthermore, step S2 specifically includes:

[0060] extracting a multi-scale feature map F from the collected image data through ResNet or Swin Transformer.

[0061] Advantages of the present application:

[0062] The present application innovatively integrates a BEV query (Queries) elastic resolution mechanism driven by semantic density and a heterogeneous feature fusion architecture based on pyramid multi-scale attention. Data is acquired through a vehicle-mounted multi-view camera, a non-uniform BEV query grid matching the scene complexity is generated through a dynamic query module, a multi-scale fusion module is used to construct a feature representation that takes into account both the global and the details, and finally the efficient interaction and fusion of the two are realized within a spatio-temporal attention framework, outputting a high-precision and high-robustness BEV static perception result. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 is a dynamic adaptive BEV perception multi-scale feature fusion method flowchart provided by an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application.

[0065] The present application provides a dynamic adaptive BEV perception multi-scale feature fusion method, as shown in Figure 1 specifically includes:

[0066] S1, data acquisition;

[0067] Synchronously collecting multi-view image data of 360° around the vehicle through a plurality of wide-angle cameras deployed around the vehicle body.

[0068] S2, feature extraction;

[0069] extracting a multi-scale feature map F from the collected image data through ResNet or Swin Transformer.

[0070] S3, generating a dynamic adaptive BEV query;

[0071] Taking the multi-scale feature map F and a set of initial queries Q as input, outputting a query coordinate dynamically adjusted in resolution according to the current scene complexity, the specific process being as follows:

[0072] S301, scene complexity analysis;

[0073] First, a lightweight semantic segmentation network is used to analyze multi-scale feature maps F, extract lane line curvature variance and obstacle distribution entropy features, and linearly combine the lane line curvature variance and obstacle distribution entropy features to generate a scene complexity heat map H corresponding to the BEV plane;

[0074] S302, resolution dynamic adjustment;

[0075] A lightweight gating network composed of an MLP is designed to take the heat map H and the initial query Q as input, and calculate a resolution scaling factor for each grid position as follows:

[0076] ;

[0077] In the formula, resolution scaling factor, feature concatenation operation, the feature value or feature vector corresponding to the heat map H at the grid (i, j) position, the initial query Q at the grid (i, j) position, bias term, which provides additional learnable parameters for the model, increases the fitting ability of the model, so that the output is not completely dependent on the weighted sum of the input, and the overall translation can be performed, the weight matrix of the last linear transformation layer in the gating network, which performs weighted summation on the high-dimensional features extracted by the MLP, and maps (or transforms) them into a single numerical value. This value is then sent to the Sigmoid function to generate a resolution scaling factor between 0 and 1;

[0078] S303, dynamic coordinate mapping;

[0079] According to the calculated scaling factor, the original uniform grid coordinates are nonlinearly transformed by bilinear interpolation to generate adaptive query coordinates, achieving the effect of densification in complex areas and sparsification in simple areas, as follows:

[0080] ;

[0081] In the formula, , respectively represent the maximum offset in the x and y directions, which defines the maximum distance each grid point can move, , respectively represent the horizontal and vertical coordinates of the original uniform grid, , respectively represent the horizontal and vertical coordinates of the query coordinates.

[0082] To further improve the query efficiency and the generalization ability to long-tail scenarios, the application introduces a sparse attention mechanism guided by feature clustering.

[0083] Semantic constrained K-means++ clustering: divide multi-view image features into K semantic clusters . The optimization objective function not only considers geometric distance, but also introduces semantic consistency as a constraint term through KL divergence

[0084] ;

[0085] In the formula, , represents the set of K cluster centers, , represents the center vector of the i-th cluster, , represents the assignment of N data points, , represents the index of the cluster to which the n-th data point is assigned, , represents the feature vector of the n-th multi-view image, , represents the center point of the cluster assigned to the feature , represents a weight coefficient, , represents a function for obtaining the semantic probability distribution of a certain feature vector, , represents the KL divergence. Correlation weight and local attention: each BEV query First, the correlation weight

[0086] with each cluster center is calculated, so as to locate a few (for example, M=3) clusters with high relevance.

[0087] ;

[0088] In the formula, , is the temperature coefficient, , represents the center vector of the j-th cluster. Subsequent Top-K feature screening and attention calculation are only performed within the M high weight clusters, which greatly reduces the computational complexity and improves the detection efficiency and accuracy of the model in unconventional scenarios such as construction areas, , represents the i-th BEV query vector, each corresponds to a specific query point in the bird's eye view space, and its task is to actively aggregate information related to this spatial position in the image features, and T represents transposition.

[0089] S4, pyramid multi-scale feature fusion; ​​​

[0090] With the multi-scale feature map F as the input, a feature map F_fused fused through multi-granularity information is output, and the specific process is as follows:

[0091] S401, multi-scale features are extracted in parallel;

[0092] The input multi-scale feature map F is divided into three groups along the channel dimension, and is processed in parallel through heterogeneous convolution branches with different receptive fields;

[0093] The high-resolution branch adopts a 3*3 small convolution kernel to capture sub-pixel level lane edge geometric details;

[0094] The mesoscopic branch adopts a 5*5 hollow convolution to model the spatial continuity of the road along and the isolation belt;

[0095] The low-resolution branch adopts a 7*7 large convolution kernel to analyze the macro-topological relationship of the intersection diversion belt.

[0096] S402, dynamic channel attention weighting;

[0097] The multi-scale features processed in parallel are spliced to obtain spliced features, and the spliced features are sent to an SE module; the SE module learns a dynamic weight for each feature channel through global average pooling and two fully connected networks, in the following manner:

[0098] ;

[0099] In the formula, The dynamic weight is represented by σ and δ, respectively Sigmoid and ReLU activation functions, The spliced features are represented by σ and δ, respectively Sigmoid and ReLU activation functions, 、 The weight matrices of the two fully connected layers in the SE (Squeeze-and-Excitation) module are represented by σ and δ, respectively.

[0100] In order to enhance the perception robustness under dynamic changes and occlusion, the application introduces a time series continuity constraint based on a hidden Markov model.

[0101] State transition model: BEV feature sequence The time series evolution of the BEV feature sequence is modeled as a state transition process, which takes into account the vehicle's own motion speed .

[0102] ;

[0103] In the formula, The bird's eye view feature or state at the current time point t is represented by σ and δ, respectively Sigmoid and ReLU activation functions, represents the aerial view feature or state at the last time point t-1, represents a Gaussian distribution, represents the velocity multiplied by time, used to estimate the displacement of the BEV feature in space due to the vehicle's own movement in this period of time, represents the covariance matrix of the normal distribution.

[0104] Maximum a posteriori estimation: through Bayesian theory, the observation information of the current frame is fused and the time series prior based on historical information , the query result of the current frame is optimized.

[0105] ;

[0106] wherein, represents the observation information at the current time point t, i.e. the multi-view image data collected by the vehicle-mounted camera at t, is a conditional probability, representing the probability of the state being given the observation condition .

[0107] S5, spatio-temporal fusion and BEV representation generation;

[0108] Through the spatial cross-attention mechanism, the dynamic query grid is taken as Query, and the fused feature map F_fused is taken as Key and Value to interact, aggregate spatial information, generate the BEV feature BEV_t at the current time, then read the BEV feature BEV_t-1 at the last time, coordinate alignment according to the vehicle's own movement, and then through the time series self-attention mechanism and BEV_t fusion, to enhance the time continuity and eliminate ambiguity.

[0109] S6, the final BEV feature map after spatio-temporal fusion is decoded into specific perception results for output.

[0110] Compared with the prior art, the present application has the following remarkable beneficial effects through the two core innovations of dynamic adaptive BEV query mechanism and pyramid multi-scale attention (PSA) fusion architecture:

[0111] 1. The present application solves the problems of insufficient scene adaptability and unreasonable allocation of computing resources, and realizes the coordinated improvement of precision and efficiency.

[0112] Specific improvement means: The present application discards the fixed and uniform BEV query grid in the prior art. By introducing a set of elastic resolution mechanism driven by real-time semantic density, the density of the query grid can be automatically adjusted according to the dynamic changes of the road topology (such as the interweaving area of the interchange or the straight road). In the complex structure area, the system automatically improves the resolution to sub-meter level (such as 0.25m), to accurately fit the arc-shaped lane boundary (curvature error <5cm); and in the simple structure area, the resolution is reduced, thereby greatly reducing the redundant calculation.

[0113] Beneficial effects brought: The design enables the "on-demand allocation" of computing resources, intelligently focusing on the areas most in need of fine perception. This not only achieves centimeter-level full-element resolution of static elements in complex scenarios, significantly improving perception accuracy, but also reduces computational load in simple scenarios. Compared with the fixed grid method, the inference speed is significantly improved while maintaining high accuracy, providing strong support for the dual demands of real-time performance and accuracy for L4 level autonomous driving.

[0114] 2. The present application solves the problem of low efficiency of multi-granularity feature coupling, significantly improving the integrity of scene details and the accuracy of perception.

[0115] Specific improvement means: In view of the defect that the prior art performs feature interaction on a single scale, resulting in the disconnection of macrostructure and microgeometry, the present application designs a pyramid multi-scale attention (PSA) fusion architecture. This architecture extracts fine-grained features (such as zebra crossing texture), mesoscopic continuity features (such as road edge), and macro-topological features (such as road intersection guide belt direction) on different scales through group-parallel heterogeneous convolution. Then, through the SE module, the channel weights are dynamically learned and allocated, and finally the weighted multi-scale features are deeply fused.

[0116] Beneficial effects brought: This multi-scale feature symbiotic system effectively solves the detail-structure imbalance problem. It ensures that the model can "see" both the sub-pixel edges of the lane line and the overall layout of the intersection, avoiding the loss of key information. For example, in the interchange spiral ramp scene, this architecture can accurately fuse the virtual-actual line gradient details and the global curvature features of the ramp, significantly improving the completion accuracy of the lane line interruption area.

[0117] 3. The present application solves the problem of poor perception robustness in complex and extreme working conditions, improving the all-weather and all-scenario adaptability of the autonomous driving system.

[0118] Specific improvement means: The application not only improves the performance in the conventional scene through the above two innovations, but also enhances the generalization ability of the system through the feature clustering guided open domain adaptive mechanism and the timing continuity constraint. The feature clustering mechanism enables the model to improve the element recognition rate of long-tail scenes such as temporary construction area and special-shaped intersection without additional labeled data. At the same time, the introduced HMM timing model uses historical information to correct and complete the perception results of the current frame.

[0119] Beneficial effects brought: The comprehensive application of these designs makes the perception system of the application have very high robustness. In the low-visibility scene such as heavy rain and backlight, the detection recall rate of static elements can still remain at a high level. The introduction of the timing model effectively ensures the continuity and stability of the perception of the vehicle in the tunnel scene, avoiding the perception interruption caused by signal loss or temporary occlusion. Finally, the application provides an automatic driving system with a core perception ability that can operate stably and reliably in various complex or even extreme conditions.

[0120] The above is only the preferred embodiment of the application, and it should be understood that the application is not limited to the form disclosed herein, and should not be considered as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein by the above-mentioned teaching or related technical or knowledge. The modification and change made by the person skilled in the art without departing from the spirit and scope of the application shall be within the protection scope of the claims attached to the application.

Claims

1. A multi-scale feature fusion method for dynamic adaptive BEV perception, characterized in that, include: S1, Data Acquisition; S2, Feature Extraction; S3. Generate dynamic adaptive BEV query; S4, Multi-scale feature fusion of pyramids; S5, Spatiotemporal Fusion and BEV Representation Generation; S6. Decode the final BEV feature map after spatiotemporal fusion into specific perception results and output them.

2. The multi-scale feature fusion method for dynamic adaptive BEV perception according to claim 1, characterized in that, Step S3 specifically includes: Taking a multi-scale feature map F and a set of initial queries Q as input, the output is a query coordinate that is dynamically adjusted in resolution according to the current scene complexity. The specific process is as follows: S301, Scene Complexity Analysis; First, a lightweight semantic segmentation network is used to analyze the multi-scale feature map F, extract the lane line curvature variance and obstacle distribution entropy features, and then linearly combine the lane line curvature variance and obstacle distribution entropy features to generate a scene complexity heatmap H corresponding to the BEV plane. S302, Dynamic resolution adjustment; Design a lightweight gated network based on an MLP, taking a heatmap H and an initial query Q as input, and calculate a resolution scaling factor for each grid location as follows: ； In the formula, This represents the resolution scaling factor. This indicates a feature concatenation operation. This represents the eigenvalue or eigenvector corresponding to the heatmap H at grid position (i, j). Let Q represent the initial query at grid position (i, j). Indicates the bias term. This represents the weight matrix of the last linear transformation layer in the gated network. S303, Dynamic coordinate mapping; Based on the calculated scaling factor, the original uniform grid coordinates are nonlinearly transformed using bilinear interpolation to generate adaptive query coordinates, as follows: ； In the formula, , These represent the preset maximum offsets in the x and y directions, respectively, defining the maximum distance each grid point can move. , These represent the x and y coordinates of the original uniform grid, respectively. , These represent the x and y coordinates of the query coordinates, respectively.

3. The multi-scale feature fusion method for dynamic adaptive BEV perception according to claim 2, characterized in that, Step S3 specifically also includes: S304, Semantic Constraints of K-means++ Clustering; Multi-view image features Divided into K semantic clusters The optimized objective function considers not only geometric distance but also KL divergence. Semantic consistency is introduced as a constraint, as follows: ； In the formula, Denotes the set of K cluster centers. Indicates the first The center vector of each cluster This indicates the allocation of N data points. This represents the index of the cluster to which the nth data point is assigned. This represents the feature vector of the nth multi-view image. Indicates assignment to feature The center point of that cluster, This represents a weighting coefficient. This represents a function used to obtain the semantic probability distribution of a given feature vector; S305, Relevance Weights and Local Attention; Each BEV query First, calculations are performed with each cluster center. Association weight This allows us to locate highly correlated clusters, as follows: ； In the formula, For temperature coefficient, Let the vector represent the center vector of the j-th cluster. Let represent the i-th BEV query vector, and T represent the transpose.

4. The multi-scale feature fusion method for dynamic adaptive BEV perception according to claim 2, characterized in that, Step S4 specifically includes: Taking a multi-scale feature map F as input, the output is a feature map F_fused after fusing multi-granularity information. The specific process is as follows: S401, Parallel extraction of multi-scale features; The input multi-scale feature map F is divided into three groups along the channel dimension and processed in parallel through heterogeneous convolutional branches with different receptive fields. High-resolution branches, using 3 × 3 small convolution kernels, capture subpixel-level lane line edge geometric details; In the mesoscopic branch, a 5 × 5 dilated convolution is used to model the spatial continuity of roadside curbs and medians; The low-resolution branch uses a 7 × 7 large convolution kernel to analyze the macroscopic topological relationship of the traffic flow divider at the intersection; S402, Dynamic Channel Attention Weighting; The multi-scale features processed in parallel are concatenated to obtain concatenated features, which are then fed into the SE module. The SE module learns a dynamic weight for each feature channel through global average pooling and a two-layer fully connected network, as follows: ； In the formula, The dynamic weights are represented by σ and δ, which represent the Sigmoid and ReLU activation functions, respectively. Indicates splicing characteristics, , These represent the weight matrices of the two fully connected layers in the SE module.

5. The multi-scale feature fusion method for dynamic adaptive BEV perception according to claim 4, characterized in that, Step S4 further includes introducing temporal continuity constraints based on hidden Markov models, specifically including: BEV feature sequence The temporal evolution model is a state transition process, taking into account the vehicle's own speed. The method is as follows: ； In the formula, This represents the bird's-eye view features or state at the current time point t. This represents the bird's-eye view features or state at the previous time point t-1. Indicates a Gaussian distribution. This represents speed multiplied by time, used to estimate the spatial displacement of the BEV feature due to the vehicle's own motion during this time period. Represents the covariance matrix of a normal distribution; By using Bayesian theory, the observation information of the current frame is fused. and time-series priors based on historical information The query results for the current frame The optimization can be performed as follows: ； In the formula, This represents the observation information at the current time point t, that is, the multi-view image data collected by the vehicle-mounted camera at time t. It is a conditional probability, representing the probability under given observation conditions. The next state is The probability of.

6. The multi-scale feature fusion method for dynamic adaptive BEV perception according to claim 4, characterized in that, Step S5 specifically includes: Through a spatial cross-attention mechanism, the dynamic query grid is used as the Query and interacts with the fused feature map F_fused as the Key and Value to aggregate spatial information and generate the BEV feature BEV_t at the current time. Then, the BEV feature BEV_t-1 at the previous time is read, the coordinates are aligned according to the vehicle's own motion, and then fused with BEV_t through a temporal self-attention mechanism.

7. The multi-scale feature fusion method for dynamic adaptive BEV perception according to claim 1, characterized in that, Step S1 specifically includes: Multiple wide-angle cameras deployed around the vehicle body simultaneously collect 360° multi-view image data of the vehicle's surroundings.

8. The multi-scale feature fusion method for dynamic adaptive BEV perception according to claim 1, characterized in that, Step S2 specifically includes: Multiscale feature maps F are extracted from the acquired image data using ResNet or Swing Transformer.

Images