Unmanned multi-mode perception fusion method based on cross-vehicle cooperative attention mechanism

Abstract

The embodiment of the invention provides an unmanned multi-mode perception fusion method based on a cross-vehicle cooperative attention mechanism, and belongs to the technical field of data processing, and the method specifically comprises the steps: collecting environment data, and carrying out the time-space alignment processing of the environment data, and obtaining multi-source perception data; performing feature extraction on the multi-source sensing data of each vehicle based on a multi-scale expansion convolutional network, and performing time alignment and standardization processing on the extracted features to obtain standardized semantic features corresponding to each vehicle; the standardized semantic features of all the vehicles are mapped into query vectors, key vectors and value vectors, the attention weight between every two vehicles is calculated through a cross-vehicle cooperative attention mechanism according to the query vectors, the key vectors and the value vectors, weighted fusion is conducted on the value vectors of all the vehicles according to the attention weights, and global fusion features corresponding to all the vehicles are obtained; and performing target detection and recognition based on the global fusion features, and outputting a sensing result. Through the scheme disclosed by the invention, the sensing accuracy, the real-time performance and the reliability are improved.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to an autonomous driving multimodal perception fusion method based on a cross-vehicle collaborative attention mechanism. Background Technology

[0002] Currently, with the rapid development of intelligent connected vehicles and autonomous driving technology, perception systems have evolved from single-vehicle intelligence to multi-vehicle collaboration and vehicle-road collaboration. Traditional autonomous driving systems rely on onboard sensors to complete environmental perception, but their perception range is limited by their own field of view and sensor performance.

[0003] Existing collaborative perception technologies face four main shortcomings when fusing information from multiple vehicles: First, they lack an adaptive weight allocation mechanism, resulting in rigid handling of redundancy and quality differences in the perception features of different vehicles, leading to insufficient fusion efficiency and robustness. Second, the spatiotemporal alignment methods are simple and cannot overcome data mismatch caused by vehicle dynamics and communication delays, affecting fusion accuracy. Third, they do not fully consider the volatility and finiteness of V2X communication bandwidth and lack a data selection mechanism based on information value, resulting in low communication efficiency. Fourth, the fusion models such as Transformer or graph neural networks used are complex in structure and computationally burdensome, making it difficult to achieve real-time deployment and application on in-vehicle terminals.

[0004] It is evident that there is an urgent need for a multimodal perception fusion method for autonomous driving based on a cross-vehicle collaborative attention mechanism that offers high accuracy, real-time performance, and reliability. Summary of the Invention

[0005] In view of this, the present disclosure provides an autonomous driving multimodal perception fusion method based on a cross-vehicle collaborative attention mechanism, which at least partially solves the problems of poor perception accuracy, real-time performance and reliability in the prior art.

[0006] This disclosure provides an embodiment of an autonomous driving multimodal perception fusion method based on a cross-vehicle cooperative attention mechanism, including:

[0007] Step 1: Collect environmental data by using multimodal sensors mounted on each vehicle, and perform spatiotemporal alignment processing on the environmental data to obtain multi-source perception data under a unified spatiotemporal reference.

[0008] Step 2: Based on a multi-scale dilated convolutional network, feature extraction is performed on the multi-source perception data of each vehicle, and the extracted features are time-aligned and standardized to obtain the standardized semantic features corresponding to each vehicle.

[0009] Step 3: Map the standardized semantic features of each vehicle into query vector, key vector and value vector, and calculate the attention weight between each pair of vehicles through a cross-vehicle collaborative attention mechanism. Based on the attention weight, perform weighted fusion of the value vectors of each vehicle to obtain the global fusion features corresponding to each vehicle.

[0010] Step 4: Perform target detection and recognition based on global fusion features, and output the perception results.

[0011] According to a specific implementation of an embodiment of this disclosure, the step of performing spatiotemporal alignment processing on environmental data includes:

[0012] The pose transformation matrix is ​​used to uniformly map the data collected by each sensor to the global coordinate system;

[0013] Data from different sensors is synchronized using a unified timestamp.

[0014] According to one specific implementation of this disclosure, the multi-scale dilated convolutional network uses convolutional layers with different dilation rates for feature extraction to simultaneously capture local details and distant target information.

[0015] According to one specific implementation of this disclosure, the convolutional layers with different dilation rates are combined in parallel or in series, and the set of dilation rates includes one or more of 1, 2, 4, and 8.

[0016] According to a specific implementation of an embodiment of this disclosure, step 3 specifically includes:

[0017] Step 3.1: Input the standardized semantic features of each vehicle into the linear mapping layer to obtain the query vector, key vector, and value vector;

[0018] Step 3.2: Calculate the semantic similarity matrix between vehicles based on their query vectors, key vectors, and value vectors.

[0019] Step 3.3: Based on the semantic similarity matrix, spatial distance, communication delay, and confidence correction terms are introduced to calculate the attention weights between each pair of vehicles;

[0020] Step 3.4: The value vectors of each vehicle are weighted and fused according to the attention weights. A multi-head attention structure is used for parallel computation. The input and output are connected through residual connections to form short-circuit paths, thereby obtaining the global fused features corresponding to each vehicle.

[0021] According to a specific implementation of this disclosure, the expression for the attention weight is:

[0022]

[0023] in, Indicates vehicle For vehicles attention weights of features Indicates the spatial distance between vehicles. This indicates a delay in communication between vehicles. Indicates the confidence level of the feature. Let be the distance decay function. This represents the distance attenuation coefficient, used to measure the spatial distance between vehicles. The intensity of the penalty for attention weights, For the delay penalty function, This represents the delay penalty coefficient, used to measure the communication delay between vehicles. The intensity of the penalty for attention weights, This represents the adjustment coefficient.

[0024] According to a specific implementation of this disclosure, the expression for the global fusion feature is:

[0025]

[0026] in, Indicates the first The value vector feature representation of each vehicle.

[0027] According to a specific implementation of an embodiment of this disclosure, step 4 specifically includes:

[0028] Step 4.1: Input the global fusion features into the lightweight detection network, and generate preliminary target detection boxes and category probabilities through the convolutional detection head of the lightweight detection network containing the channel attention mechanism;

[0029] Step 4.2: Perform non-maximum suppression on the preliminary target detection boxes to eliminate redundant boxes and retain the detection result with the highest confidence as the final perception result and output it.

[0030] According to a specific implementation of an embodiment of this disclosure, after step 5, the method further includes:

[0031] Based on the confidence level of the perception results, the weight coefficients of the feature confidence correlation terms used to calculate attention weights are dynamically adjusted to achieve online adaptive optimization of the fusion parameters.

[0032] The autonomous driving multimodal perception fusion scheme based on cross-vehicle collaborative attention mechanism in this embodiment includes: Step 1, collecting environmental data through multimodal sensors mounted on each vehicle, and performing spatiotemporal alignment processing on the environmental data to obtain multi-source perception data under a unified spatiotemporal reference; Step 2, extracting features from the multi-source perception data of each vehicle based on a multi-scale dilated convolutional network, and performing temporal alignment and standardization processing on the extracted features to obtain standardized semantic features corresponding to each vehicle; Step 3, mapping the standardized semantic features of each vehicle into query vectors, key vectors, and value vectors, and calculating the attention weights between each pair of vehicles based on the cross-vehicle collaborative attention mechanism, and performing weighted fusion of the value vectors of each vehicle according to the attention weights to obtain the global fusion features corresponding to each vehicle; Step 4, performing target detection and recognition based on the global fusion features, and outputting the perception results.

[0033] The beneficial effects of this disclosure are as follows: By constructing a cross-vehicle feature association modeling framework, the semantic hierarchical fusion and dynamic weight allocation of perceived information between different vehicles are achieved, thereby effectively improving the integrity and robustness of environmental perception under limited communication conditions. By introducing a collaborative attention mechanism, this invention can adaptively calculate feature importance based on the relative position, confidence level, and communication latency between vehicles, achieving efficient information selection and optimal fusion, reducing redundant data transmission, and improving recognition accuracy. Furthermore, this invention aims to reduce the computational complexity of the algorithm through a lightweight feature extraction and fusion structure, enabling the method to run in real-time on an in-vehicle embedded platform, meeting the requirements of autonomous driving systems for high precision, high real-time performance, and high reliability, thus providing an efficient and scalable solution for multi-vehicle collaborative perception and intelligent decision-making in autonomous vehicles. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating an autonomous driving multimodal perception fusion method based on a cross-vehicle collaborative attention mechanism, provided as an embodiment of this disclosure. Detailed Implementation

[0036] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0037] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0038] It should be noted that various aspects of the embodiments described below are within the scope of the appended claims. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0039] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The illustrations only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0040] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0041] This disclosure provides a method for multimodal perception fusion in autonomous driving based on a cross-vehicle collaborative attention mechanism, which can be applied to the monitoring process of autonomous vehicles in traffic control scenarios.

[0042] See Figure 1 This is a flowchart illustrating an autonomous driving multimodal perception fusion method based on a cross-vehicle collaborative attention mechanism, provided in an embodiment of this disclosure. Figure 1 As shown, the method mainly includes the following steps:

[0043] Step 1: Collect environmental data by using multimodal sensors mounted on each vehicle, and perform spatiotemporal alignment processing on the environmental data to obtain multi-source perception data under a unified spatiotemporal reference.

[0044] In practice, each driverless vehicle is equipped with sensors such as cameras, LiDAR, millimeter-wave radar, and inertial measurement units (IMU) to form a multimodal information source.

[0045] Each sensor outputs the following:

[0046] (1)

[0047] in, Represents a sequence of visual images. Represents 3D point cloud data. It has millimeter-wave radar reflection characteristics. This includes the vehicle's own state information (speed, acceleration, attitude angle, etc.).

[0048] To ensure spatiotemporal consistency, this module establishes a time synchronizer and a coordinate unifier on the vehicle side. All data is timestamped using a unified timestamp. Alignment, spatial mapping to the global coordinate system :

[0049] (2)

[0050] in For the first The vehicle's pose transformation matrix. These are the local coordinates of the sensor.

[0051] The sampling frequency can be adaptively adjusted according to the complexity of the scene. For example, when vehicles are traveling at high speeds or during periods of heavy traffic, the system automatically increases the sampling frequency of the LiDAR and millimeter-wave radar to increase environmental density information; when the vehicle is moving at low speeds or stationary, the system reduces the sampling frequency to save computing power and bandwidth resources.

[0052] Step 2: Based on a multi-scale dilated convolutional network, feature extraction is performed on the multi-source perception data of each vehicle, and the extracted features are time-aligned and standardized to obtain the standardized semantic features corresponding to each vehicle.

[0053] In practical implementation, converting raw multimodal sensor data into a unified semantic feature representation is a crucial foundation for achieving cross-vehicle collaborative fusion. Its core tasks include: multimodal data input preprocessing, feature extraction based on dilated convolution, feature temporal alignment and normalization, and channel compression and embedding mapping. Through this module's processing, multi-source data from different vehicles are transformed into a unified high-dimensional semantic feature tensor, providing standardized input for the subsequent attention fusion module.

[0054] (1) Multimodal data input and preprocessing

[0055] Each autonomous vehicle synchronously collects environmental information through devices such as visual sensors (cameras), lidar, and millimeter-wave radar. The data set is represented as follows:

[0056] (3)

[0057] in: : No. RGB image sequence acquired from the vehicle; LiDAR point cloud data; Millimeter-wave radar reflection characteristics.

[0058] The acquired data first undergoes coordinate alignment and noise filtering. Point cloud data is density-reduced using voxel grid filtering, while image sequences are noise-removed and edge details are preserved through bilateral filtering. Subsequently, the system synchronizes time based on the timestamps of each sensor to ensure that multimodal data are accessed simultaneously. Alignment.

[0059] (2) Feature extraction network based on dilated convolution

[0060] Traditional convolutional neural networks (CNNs) suffer from a limited receptive field when extracting spatial features, making it difficult to capture long-range semantic information while maintaining resolution. To address this, this invention employs a multi-scale dilated convolution network (MSDCN) structure to expand the receptive field while keeping computational costs stable, thereby enhancing the network's sensitivity to distant targets and complex environments.

[0061] The feature extraction process can be represented as:

[0062] (4)

[0063] in: : No. Layer output features; : Convolutional kernel weights; : indicates that the expansion rate is Convolution operation; : Non-linear activation functions (such as ReLU); : Bias term; It represents the multiscale expansion rate.

[0064] By setting multiple expansion rates The network can simultaneously perceive local edge information and global structural information without increasing the number of parameters, thereby obtaining a more comprehensive environmental representation in complex traffic scenarios (such as distant obstacles, building obstructions, or areas with multiple vehicles).

[0065] The main advantage of this structure compared to traditional convolutional networks is:

[0066] Expand the receptive field to improve the ability to capture large targets and long-range information;

[0067] To avoid spatial resolution loss caused by pooling layers and preserve detailed information;

[0068] The reduced convolutional layer stacking depth requirement makes the model easier to deploy on automotive embedded platforms.

[0069] (3) Feature time alignment and standardization processing

[0070] To ensure temporal consistency of different vehicle features during fusion, this module introduces a time alignment mechanism. Let the vehicle... The data sampling time is The unified reference time is The feature alignment process is as follows:

[0071] (5)

[0072] in The time offset is used to fill in missing frames through interpolation or backtracking mechanisms.

[0073] Subsequently, all features were standardized to eliminate device differences and feature offsets:

[0074] (6)

[0075] in The first Vehicle characteristic mean and standard deviation To prevent division by zero from becoming a stable term, this process ensures that the features of multiple vehicles are statistically comparable, which contributes to the stability of subsequent collaborative attention calculations.

[0076] (4) Feature compression and embedding mapping

[0077] To reduce communication and computational overhead, this invention performs channel compression and embedding mapping after standardizing feature output. Linear dimensionality reduction is achieved using 1×1 convolution.

[0078] (7)

[0079] Obtain compact feature tensor ,in .

[0080] Next, the dimensionality-reduced features are projected onto the collaborative feature space:

[0081] (8)

[0082] in This is the feature embedding matrix, used to map vehicle features to the cross-vehicle semantic space, providing input for the cross-vehicle collaborative attention calculation in the next module.

[0083] Step 3: Map the standardized semantic features of each vehicle into query vector, key vector and value vector, and calculate the attention weight between each pair of vehicles through a cross-vehicle collaborative attention mechanism. Based on the attention weight, perform weighted fusion of the value vectors of each vehicle to obtain the global fusion features corresponding to each vehicle.

[0084] In practical implementation, the cross-vehicle collaborative attention mechanism is used to achieve feature-level semantic collaboration and dynamic fusion among multiple autonomous vehicles. Its main goal is to capture the dependencies between the perception results of different vehicles at the feature level through the attention mechanism, and adaptively adjust the feature fusion weights based on multi-dimensional information such as spatial location, communication latency, and feature confidence, thereby improving the global perception accuracy and robustness while ensuring real-time performance.

[0085] (1) Functional Overview

[0086] In autonomous driving cooperative scenarios, the perception results of different vehicles often have the following problems: information redundancy in the overlapping areas of vision; spatiotemporal offset caused by communication delay; and fusion inconsistency caused by differences in feature confidence.

[0087] This module introduces a cross-vehicle cooperative attention (CVCA) mechanism to establish a feature-level semantic dependency network among multiple vehicles. This allows each vehicle to not only utilize its own perception information but also to "selectively" absorb supplementary features from neighboring vehicles, thereby forming a unified global semantic feature representation.

[0088] (2) Algorithm principle

[0089] Assume there are a total of vehicles participating in the collaboration. For each vehicle, after feature extraction and standardization in the second module, a feature tensor is obtained:

[0090]

[0091] First, each vehicle feature is mapped to a query (Q), key (K), and value (V) to enable multi-head attention computation:

[0092] (9)

[0093] in, For a trainable parameter matrix, For the embedded dimension.

[0094] In traditional single-vehicle attention mechanisms, attention calculation is performed only within a single node, lacking cross-node feature dependency modeling. The cross-vehicle collaborative attention calculation formula proposed in this invention is as follows:

[0095] (10)

[0096] in: :vehicle For vehicles Attention weights for features; Spatial distance between vehicles; Inter-vehicle communication delay; Feature confidence; This is a distance decay function used to suppress excessively distant nodes; For delay penalty function; Three adjustment coefficients.

[0097] Using the above formula, the system considers feature semantic similarity, spatial constraints, and time delay simultaneously during the attention allocation process, thereby achieving physical consistency modeling of multi-source features.

[0098] The fused global features are represented as follows:

[0099] (11)

[0100] This indicates a vehicle. Global perception features obtained after fusing information from other vehicles.

[0101] (3) Collaborative attention calculation process

[0102] The algorithm structure of this module is implemented in a step-by-step manner, and the specific steps are as follows:

[0103] Step 1: Feature Embedding and Dimensionality Reduction: The standardized features output from the second module are processed... Input is passed to the linear mapping layer to obtain the query. ,key ,value Vectors. This process is used to unify feature dimensions and semantic space; the relevant calculation formulas are as described above.

[0104] Step 2: Cross-vehicle feature correlation calculation: Based on the feature representations of multiple vehicles, the system calculates the semantic similarity matrix between nodes. This process is used to measure vehicles. and The semantic complementarity between them provides a basis for subsequent weight allocation.

[0105] Step 3: Attention Weight Correction and Normalization: Based on the similarity matrix, the system introduces spatial distance, communication delay, and confidence correction terms to obtain a weighted attention matrix. To avoid formula repetition, the specific calculation relationship can be found in the aforementioned attention weight definition. After Softmax normalization, the sum of the weights for each vehicle equals 1, thus ensuring the balance and physical consistency of feature fusion.

[0106] Step 4: Global Feature Fusion and Weighted Aggregation: The system is based on the weight matrix. Multi-vehicle characteristics We perform a weighted summation to obtain the fused global semantic features. This result includes perception information from all cooperating vehicles, achieving the transformation from local features to global perception.

[0107] Step 5: Multi-head attention parallelism and residual update: To enhance the model's expressive power and convergence stability, the system employs a multi-head attention structure for parallel computation. Each sub-head focuses on different semantic regions, and the output results are connected and linearly mapped to form the final fused features. Simultaneously, residual connections are used to create short-circuit paths between the input and output to avoid feature degradation caused by deep computation.

[0108] Step 6: Output and caching of fusion results: The final generated fusion features The data is cached in a local shared storage area to provide input for subsequent detection and decision-making modules. The system also records the attention matrix and confidence distribution for online parameter adaptive adjustment in the next cycle.

[0109] Step 4: Perform target detection and recognition based on global fusion features, and output the perception results.

[0110] In practice, after obtaining the global fusion features, the global fusion features can be... This is then converted into concrete perception results. To ensure structural simplicity and computational efficiency, and with target detection as the main output, the system aims to identify and locate obstacles, pedestrians, or vehicles in the environment.

[0111] (1) Functional Overview

[0112] The module input is multi-vehicle fusion features. The output consists of object detection boxes and category labels. The system uses a lightweight detection network to map high-dimensional semantic features into structured detection results, achieving efficient conversion from global features to physical targets.

[0113] (2) Detection network structure and implementation

[0114] This module adopts a lightweight convolutional detection head structure, mainly including:

[0115] Feature compression layer: using Convolutional dimensionality reduction reduces the number of feature channels and lowers computational cost;

[0116] Candidate box generation layer: Generates multi-scale anchor boxes at each feature location to adapt to targets of different sizes;

[0117] Detection and prediction layer: Outputs the class probability corresponding to each anchor box. With bounding box parameters .

[0118] The set of detection results can be represented as:

[0119] (12)

[0120] To enhance the representation of salient features, this module introduces a channel attention mechanism to weight high-response channels:

[0121] (13)

[0122] This design can automatically enhance the feature response of salient regions, thereby improving detection accuracy.

[0123] (3) Post-processing and result output

[0124] After the detection phase, the system performs non-maximum suppression (NMS) on the prediction results to remove redundant detection boxes and retain the results with the highest confidence.

[0125] (14)

[0126] in This is the IoU threshold. The final output includes: target category label; bounding box coordinates; and prediction confidence score.

[0127] All test results are cached in chronological order in a local shared storage area to guide path planning and decision-making.

[0128] (4) Confidence feedback and adaptive optimization module

[0129] To improve the model's stability and adaptability in complex scenarios, this module implements an online confidence feedback mechanism. The system calculates the average confidence score of all vehicle detection results within each detection cycle. The fusion parameter weights for cross-vehicle collaborative attention fusion are dynamically adjusted accordingly. The specific update formula is as follows:

[0130] (15)

[0131] in, For the current attention fusion parameters, This is the learning rate coefficient. For the first The system assesses the detection confidence of each vehicle. When the detection confidence of a vehicle is below the average, the system automatically increases the attention weight of features of its neighboring vehicles to improve overall recognition performance and fusion robustness.

[0132] This feedback mechanism enables online adaptive adjustment of feature fusion parameters, thereby maintaining dynamic stability and global consistency of model performance in multi-vehicle collaborative perception scenarios.

[0133] This embodiment provides a multimodal perception fusion method for autonomous driving based on a cross-vehicle collaborative attention mechanism. By constructing a cross-vehicle feature association modeling framework, it achieves semantic-level fusion and dynamic weight allocation of perception information between different vehicles, thereby effectively improving the completeness and robustness of environmental perception under limited communication conditions. By introducing a collaborative attention mechanism, this invention can adaptively calculate feature importance based on the relative position, confidence level, and communication latency between vehicles, achieving efficient information selection and optimal fusion, reducing redundant data transmission, and improving recognition accuracy. Furthermore, this invention aims to reduce the computational complexity of the algorithm through a lightweight feature extraction and fusion structure, enabling the method to run in real-time on an in-vehicle embedded platform. This meets the requirements of autonomous driving systems for high precision, high real-time performance, and high reliability, thus providing an efficient and scalable solution for multi-vehicle collaborative perception and intelligent decision-making in autonomous vehicles.

[0134] It should be understood that the various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof.

[0135] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An unmanned multi-modal perception fusion method based on cross-vehicle collaborative attention mechanism, characterized in that, The method comprises the following steps: Step 1, collecting environmental data by multi-modal sensors mounted on each vehicle, and performing spatio-temporal alignment processing on the environmental data to obtain multi-source perception data under a unified spatio-temporal reference; Step 2, performing feature extraction on the multi-source perception data of each vehicle based on a multi-scale dilated convolution network, and performing time alignment and standardization processing on the extracted features to obtain standardized semantic features corresponding to each vehicle; Step 3, mapping the standardized semantic features of each vehicle into query vectors, key vectors and value vectors, and calculating the attention weights between each pair of vehicles through a cross-vehicle collaborative attention mechanism according to the query vectors, key vectors and value vectors, and weighting and fusing the value vectors of each vehicle according to the attention weights to obtain global fusion features corresponding to each vehicle; Step 4, performing target detection and identification based on the global fusion features, and outputting the perception results.

2. The method of claim 1, wherein, The step of performing spatio-temporal alignment processing on the environmental data comprises: Mapping the data collected by each sensor to a global coordinate system through a pose transformation matrix; Synchronizing the data from different sensors in time according to a unified timestamp.

3. The method of claim 1, wherein, The multi-scale dilated convolution network uses convolution layers with different dilation rates to extract features to simultaneously capture local details and long-distance target information.

4. The method of claim 3, wherein, The convolution layers with different dilation rates are combined in parallel or in series, and the set of their dilation rates includes one or more of 1, 2, 4 and 8.

5. The method of claim 1, wherein, Step 3 specifically comprises: Step 3.1, inputting the standardized semantic features of each vehicle into a linear mapping layer to obtain query vectors, key vectors and value vectors; Step 3.2, calculating the semantic similarity matrix between each pair of vehicles according to the query vectors, key vectors and value vectors of each vehicle; Step 3.3, calculating the attention weights between each pair of vehicles based on the semantic similarity matrix after introducing spatial distance, communication delay and confidence correction terms; Step 3.4, weighting and fusing the value vectors of each vehicle according to the attention weights, calculating in parallel using a multi-head attention structure, forming a short circuit path between the input and the output through a residual connection, and obtaining global fusion features corresponding to each vehicle.

6. The method of claim 5, wherein, The expression of the attention weight is: ； wherein, denotes a vehicle to a vehicle a feature attention weight, denotes a spatial distance between vehicles, denotes a communication delay between vehicles, denotes a feature confidence, is a distance decay function, denotes a distance decay coefficient for measuring a spatial distance between vehicles a penalty strength on the attention weight, is a latency penalty function, denotes a latency penalty coefficient for measuring a communication delay between vehicles a penalty strength on the attention weight, denotes an adjustment coefficient.

7. The method of claim 6, wherein, The expression of the global fusion feature is: ； wherein, represents a value vector feature representation of the th vehicle.

8. The method of claim 7, wherein, Step 4 specifically comprises: Step 4.1, inputting the global fusion feature into a lightweight detection network, and generating preliminary target detection boxes and class probability through the convolution detection head containing the channel attention mechanism in the lightweight detection network; Step 4.2, performing non-maximum suppression processing on the preliminary target detection boxes to eliminate redundant boxes and retain the detection result with the highest confidence as the final perception result and output.

9. The method of claim 8, wherein, After step 5, the method further comprises: According to the confidence of the perception result, dynamically adjusting the weight coefficient of the feature confidence correlation term used to calculate the attention weight to realize online adaptive optimization of the fusion parameters.

Images