An internet of vehicles multi-agent cooperative perception method and system

Abstract

The application relates to a vehicle networking multi-agent cooperative perception method and system. The method comprises the following steps: constructing a cooperative perception network, wherein the cooperative perception network comprises one ego agent and multiple cooperative agents; acquiring and processing perception data through the cooperative perception network; performing feature extraction to obtain intermediate features; performing adaptive sparsification to obtain sparse features, and compressing and transmitting the sparse features to the ego agent; performing time sequence feature enhancement on the features of all agents at the ego agent end; fusing the features to obtain fused features; constructing a vehicle networking perception model, and realizing perception according to the fused features; and detecting the model. Through the adaptive sparsification mechanism, the calculation complexity of the traditional global attention is reduced from the square level to the linear level, the calculation cost is significantly reduced while the multi-agent feature interaction accuracy is maintained, and the method is more suitable for real-time operation on the resource-limited vehicle edge device.

Description

Technical Field

[0001] This invention relates to the field of vehicle networking, and more specifically, to a multi-agent collaborative perception method and system for vehicle networking. Background Technology

[0002] Environmental perception capability is the fundamental prerequisite for autonomous driving systems to achieve autonomous decision-making and planning control. However, the performance limit of perception systems relying on a single intelligent agent is constrained by the physical characteristics of onboard sensors: sensors such as LiDAR and cameras have limitations in effective ranging range and field of view, and in complex urban road environments, occlusion effects caused by buildings and large vehicles are unavoidable, making long-tail problems such as "ghost pedestrians" and sudden blind spot changes difficult to solve. These individual, inherent perception limitations constitute one of the core bottlenecks for the practical application of high-level autonomous driving.

[0003] To overcome this bottleneck, vehicle-to-everything (V2X) cooperative perception technology has emerged, considered a key enabling technology for improving the safety redundancy and reliability of autonomous driving systems. This technology, based on highly reliable, low-latency communication between vehicles and between vehicles and infrastructure, allows multiple agents to share their raw data, feature maps, or object detection results, theoretically enabling the construction of a global environmental cognition model that surpasses the vision of any single agent. Among these, the cooperative perception paradigm based on intermediate features has become the mainstream research in both academia and industry due to its optimal balance between communication overhead and information density. Despite its promising prospects, existing cooperative perception methods still face significant challenges in three dimensions—communication, computation, and dynamic scene modeling—in their path to large-scale practical deployment. Regarding communication efficiency, although feature-level collaboration has become the mainstream solution for balancing perception performance and bandwidth overhead, most existing methods lack intelligent sparsity filtering of transmitted features. For example, the method proposed in CN120612562A, "A Multi-Vehicle Cooperative Fusion Perception Method Based on Spatiotemporal Context," filters important features through spatial confidence maps, but its sparsity strategy is relatively fixed and fails to adaptively adjust the compression rate according to the dynamic content of the scene. This results in transmission redundancy still existing in simple scenarios, or loss of key target information due to over-compression in complex scenarios. In terms of utilizing temporal information, how to deeply mine historical frame information to enhance the robustness of current perception is key to improving system performance. Existing solutions typically focus on fusing historical features to provide contextual information. For example, CN118378660A, "A Multi-Agent Cooperative Perception Method Robust to Spatiotemporal Errors," proposes a spatiotemporal iterative enhancement framework, whose temporal enhancement module fuses historical features through a deformable attention mechanism. However, such methods typically involve directly concatenating or weighting historical features with current features during the fusion process, failing to explicitly mine and utilize the feature differences between adjacent frames caused by the relative motion of the vehicle and the target. These "differences" are a direct reflection of the scene's dynamic characteristics, containing rich motion information. Ignoring dedicated modeling of this leads to the model's insensitivity to feature changes in rapidly moving targets and difficulty in effectively filtering outdated static background interference from historical frames. At the multi-agent feature fusion level, an efficient and accurate feature interaction mechanism is crucial for achieving collaborative benefits. Traditional methods based on global attention or graph neural networks, while capable of modeling complex cross-agent interactions, suffer from computational complexity that increases quadratically with the number of agents, making it difficult to meet the stringent real-time requirements of edge devices in reality. A more fundamental issue is that not all spatial locations and agent features contribute equally to the perception of the agent. In typical collaborative perception scenarios, the feature information provided by numerous background areas and distant or severely occluded collaborators is highly redundant. Performing indiscriminate, intensive attention calculations on these features is not only inefficient but may also introduce excessive noise, compromising the purity of the fused features. Therefore, there is an urgent need for a fusion mechanism that can adaptively focus on key regions and key vehicles. An ideal solution should be able to dynamically identify and prioritize features rich in semantic and complementary information, thereby significantly reducing unnecessary computational overhead while maintaining fusion accuracy. However, existing methods still have significant shortcomings in achieving this "sparse yet efficient" interaction and seamlessly integrating it with the overall spatiotemporal fusion framework.

[0004] In summary, current multi-agent collaborative perception technologies still face common problems in practical deployments, such as limited communication bandwidth, scarce computing resources, and inaccurate dynamic scene modeling, which in turn affect the perception effect. Summary of the Invention

[0005] Therefore, it is necessary to provide a multi-agent collaborative perception method and system for vehicle networking with better perception performance to address the aforementioned technical problems.

[0006] One type of multi-agent cooperative perception method for vehicle-to-everything (V2X) includes: S1: Construct a collaborative sensing network, which includes: a self-aware agent and multiple collaborative agents; S2: Acquire and process sensing data through a collaborative sensing network to obtain processed sensing data; S3: The collaborative sensing network extracts features from the processed sensing data to obtain intermediate features; S4: Multiple collaborative agents adaptively sparsify the features to obtain sparse features, compress the sparse features to obtain a compressed feature representation, and transmit the compressed feature representation to the self-agent agent; S5: At the self-agent end, the received features of the cooperative agents are first decompressed and reconstructed, and then the features of all agents are enhanced in a temporal sequence to obtain the enhanced features. S6: Fuse the enhanced features of all agents to obtain the fused features; S7: Construct a vehicle-to-everything (V2X) perception model, and the detection model achieves perception based on the fused features.

[0007] Further, in step S2, specifically: in each processing cycle, the self-intelligent agent and multiple cooperative intelligent agents synchronously collect local sensor data; among them, the cooperative intelligent agents also need to receive metadata including pose broadcast by the self-intelligent agent; subsequently, each cooperative intelligent agent, according to the coordinate transformation relationship, uniformly transforms the raw perception data it has collected to the vehicle coordinate system, completes the spatial alignment of multi-source data, and obtains the processed perception data.

[0008] Furthermore, in step S3, specifically: The processed perception data includes point cloud data. The self-agent and collaborative agents use a shared-weight feature encoding network to extract features from the point cloud data. The feature encoding network uses a PointPillars backbone network to convert the point cloud data into structured cylindrical feature tensors, and further extracts them into high-order feature maps in a two-dimensional bird's-eye view format. Each agent ultimately extracts a feature map with a dimension of... intermediate features ,in The number of feature channels, and This represents the spatial dimensions of the feature map.

[0009] Further, in step S4, specifically: S4.1: Construct an adaptive sparse generator, through which each cooperative agent processes its local intermediate features. To generate a sparse mask corresponding to its intermediate features. In its implementation, the adaptive sparse generator first generates a confidence map from the intermediate input features through a lightweight network consisting of depthwise separable convolutions. The confidence map reflects the importance of the current intermediate features at each spatial location and in each channel; then, a spatially sparse map is obtained through nonlinear activation and spatial smoothing. The calculation process is as follows:

[0010] in, For the Sigmoid function, It is a Gaussian smoothing function; Simultaneously, a dynamic sparsity rate predictor is introduced, which adaptively predicts a dynamic sparsity rate based on intermediate features and confidence information through a multilayer perceptron. Its value reflects the proportion of information that needs to be retained in the current scene; finally, a Top-Sparse Map is performed on the spatial sparse graph based on this sparsity rate. K Filter and generate a binary sparse mask. The total number of spatial locations that were selected ; S4.2: Each cooperative agent performs sparsification processing on the intermediate features based on the generated sparse mask, and then compresses and transmits the sparse features and the mask; in specific implementation, the cooperative agents first utilize the sparse mask... For intermediate features Sparsity processing is performed to obtain sparse features. In the sparse feature model, only feature values ​​at important spatial locations are retained to adapt to the limited communication bandwidth in real-world scenarios. Subsequently, based on the characteristics of the feature value distribution after sparsification, the sparse features are quantized and lossless entropy encoded to obtain a compressed representation. ; and sparse masking Binarization and bit packing are performed to obtain Next, compress the data packet. Transmitted to the self-intelligent agent via wireless communication.

[0011] Furthermore, in step S5, specifically: Historical feature caching and update mechanism: The self-agent agent includes a feature alignment module and a difference-aware temporal fusion module; the self-agent agent needs to maintain a lightweight circular cache to store the intermediate features of all agents, including the self-agent agent, at the previous time step. The cache uses a first-in, first-out (FIFO) strategy. After completing the temporal feature enhancement processing for the current moment, it automatically overwrites the corresponding historical features with the features of the current moment. Specifically, when the agent is at moment... Received and decompressed features from each cooperative agent With mask Then, the self-regulating agent reads the information from the cache of each agent. Historical characteristics of a moment and the current characteristics of each agent Corresponding historical characteristics The input is fed into the feature alignment module and the difference-aware temporal fusion module. After temporal enhancement is completed, the historical features of each agent in the buffer are updated to the current features.

[0012] Furthermore, the feature alignment module specifically comprises: Based on the feature alignment module of multi-scale optical flow, historical features are... Precisely align to the current feature The feature space; this module constructs a multi-scale pyramid from coarse to fine, at each scale Above, a lightweight optical flow estimation network is used to compute the optical flow field from historical features to current features. Using the upsampled optical flow from the previous scale as initialization, and calculating the residual optical flow to distort historical features, this process is expressed by the following formula: = + (Concat[ , ( , )]) in, For the first Layer optical flow estimation network, Concat is a feature warping operation based on bilinear sampling, and Concat is a feature concatenation operation. First, use bilinear interpolation to extract historical features. and current features Downsampling to a specified scale to construct Feature pyramids at various scales; used Optical flow estimation networks with different input scales are used to estimate optical flow from historical features to current features in ascending order of scale; for the first layer of the pyramid, the initial optical flow field is... Set to zero; at each scale Above, the optical flow estimated at the previous scale. Upsample to the current scale, and use this upsampled optical flow through a bilinear sampling distortion operation. Preliminary alignment of historical features yields distorted historical features at this scale. ( , ); to the current scale features It is then stitched together with the distorted historical features and input into the optical flow network at the current scale. The residual optical flow is calculated in the middle, and this scale The final optical flow field The optical flow is obtained by adding the residual optical flow to the upsampled initial optical flow. This process is progressive, using coarse-scale optical flow to capture large-scale motion and fine-tuning with fine-scale optical flow to gradually improve alignment accuracy. Finally, after optical flow estimation is completed at the highest resolution scale, the final optical flow field is obtained. ; Utilizing the historical characteristics of the original resolution from this optical flow field The distortion is performed, and the final output is related to the current feature. Enhanced historical features of spatial alignment .

[0013] Furthermore, the difference-aware temporal fusion module specifically comprises: First, calculate the current features and the aligned enhanced historical features. The original difference tensor between them:

[0014] This difference tensor It directly encodes the dynamic changes in the scene caused by the relative motion between the vehicle and the target; A bottleneck structure is adopted, firstly compressing the number of channels through dimensionality reduction convolution, then applying ReLU nonlinear activation and batch normalization, and finally constraining the feature values ​​to the [-1,1] interval through dimensionality increase convolution and Tanh activation function, highlighting significant positive and negative variation patterns; the process is expressed as follows: ReLU(BN( )) in and The convolution weights are for dimensionality reduction and dimensionality increase, respectively. The Tanh activation function ensures that the output can represent both positive and negative changes. Constructing a lightweight saliency detection network A significance map of change is generated based on the absolute value of the difference. This is used to quantify the dynamics of each spatial location:

[0015] in The saliency plot is the Sigmoid function. The spatial region where significant movement occurred was identified; The current features and enhanced differential features are processed independently to improve their ability to perceive dynamic changes; for the current feature stream, spatial modulation is performed using a change saliency map to enhance the feature response in regions of change.

[0016] For the differential feature stream, the same saliency plot weighting is used to highlight the contribution of the regions of change:

[0017] in, and These are modulation networks for the current features and for enhancing differential features, respectively; Modulated current features Differences after modulation and aligned historical features The concatenation is performed along the channel dimension using a gated network based on global average pooling. Generate adaptive fusion weights: ( (GAP[ , , ])) in, It consists of two layers of dimensionality-reduced convolution and a ReLU nonlinear activation function; the weights It can be broken down into features for modulating the current characteristics. Differences after modulation The two feature branches are weighted separately. and Then, the two weighted features are concatenated and fed into a temporal fusion network. To obtain preliminary fusion results and achieve temporal enhancement of the current features; this process is represented as follows: ]) Finally, the temporal feature fusion results are... Compared with the original current features By combining residual connections, the residual strength is controlled by the average significance; the final output is the current feature after difference-aware temporal enhancement. The process is represented by the following formula:

[0018] in, Represents the addition operator; For adaptive residual gating, It is a saliency plot exist The operation of averaging.

[0019] Further, in step S6, specifically: A sparsely guided multi-agent fusion module is constructed, whose network structure includes a sparse perception relative position encoding submodule, a cross-agent sparse cross attention submodule, and an adaptive multi-source feature fusion submodule. Sparse-aware relative position encoding submodule: Enhanced spatial modeling capabilities for key regions are achieved through directional difference sensing fields and sparse mask-based weight modulation; two independent learnable embedding tables are maintained. and ,in , The maximum relative distance in each dimension. For attention head dimension; For position pairs in the feature map and relative offset , After pruning, the index corresponds to the embedding. and ; Sparse mask Generating direction-specific modulation factors using a lightweight convolution modulator:

[0020] in For the Sigmoid function, and They represent respectively by and Convolutional blocks are formed by convolutions; obtained by averaging along the width and height dimensions. Independent modulation of the position embedding in the height and width dimensions:

[0021] in A learnable scaling factor; Adding attention to query and key features before attention computation enhances the spatial reasoning ability of subsequent cross-agent sparse cross attention for key regions. Cross-agent sparse cross-attention submodule: The self-aware agent generates query features through convolutional blocks. Each collaborative intelligent agent Key-value pairs are generated using two convolutional blocks. ;Will and Projected onto having In a multi-headed representation space, so as to... Channel dimensions are divided into Multiple subspaces are used to capture different contextual information; positional encoding is added to query and key features; traditional dot product attention is decomposed into horizontal and vertical paths, thereby reducing computational complexity from Reduce to In the horizontal path, Containing lines The query vector for all positions, and Corresponding sparse mask Selected Individual lines The key-value vector at the key position; the same applies to vertical paths; the calculation for cross-agent sparse cross attention is as follows:

[0022]

[0023] in, and They represent and The dimension; reducing computational complexity from Further down to ,in ,in Indicates the total number of important positions; when At this point, the aforementioned cross-agent sparse cross-attention computation degenerates into self-attention computation on the intermediate features of the self-agent; the outputs from all attention heads are concatenated and linearly projected to form interaction features on each path; finally, the interaction features on the horizontal and vertical paths are summed and connected by weighted sums and residuals to obtain the final representation, which is represented by the following equation:

[0024] in For learnable scaling factor, The final interaction features are then fed into the adaptive multi-source feature fusion module, which then feeds the interaction features of the self-agent and the cooperative agents into the subsequent interaction features. Adaptive multi-source feature fusion: For each interaction feature, a weight calculation network is used. Generate spatial importance graph Then, along The dimension is normalized using Softmax, and this process can be represented as:

[0025] in, Represents a set of collaborative intelligent agents. Indexing the interaction pairs of self-aware agents and collaborative agents. It is an exponential function, whose function is to ensure that all weights are positive and to reinforce the relative differences between weights; the final fused features are obtained by weighted summation: .

[0026] Further, in step S7, specifically: The vehicle-to-everything (V2X) perception model is a lightweight detection head network. This detection head employs a dual-branch parallel structure, responsible for object classification and bounding box regression tasks respectively. The classification branch uses a convolutional kernel with a size of [missing information]. The convolutional layer implementation is used to predict the confidence that each anchor box at each spatial location contains a specific category of object; the regression branch also uses... Convolutional layers are responsible for regressing the precise geometric parameters of each object bounding box, including the center coordinates in 3D space. ,size and yaw angle The final fused global context features Input the above-mentioned detection head network and output the collaborative sensing results; The vehicle-to-everything (V2X) perception model is trained using a multi-task loss function, which consists of two parts: bounding box regression loss and classification loss. The overall optimization objective is formed by weighted summation. First, the bounding box regression loss employs a weighted smooth L1 loss function, which smoothly transitions between L1 and L2 losses and exhibits better robustness to outliers; its mathematical expression is:

[0027] in, It is the index of the anchor box. To predict bounding box parameters, To correspond to the actual annotation box parameters, For indicator functions, The number of positive samples; the SmoothL1 function is defined as:

[0028] Here This is a hyperparameter that controls the transition range between L1 and L2 losses; Secondly, the classification loss adopts the focus loss mechanism; the focus loss expression is:

[0029] in, The target confidence level predicted by the model. To balance the weights for each category, This is a focusing parameter used to adjust the weight ratio between easy and difficult samples; Ultimately, the overall loss function is the weighted sum of the two losses mentioned above:

[0030] in, and These are the weight coefficients for regression loss and classification loss, used to balance the contributions of the two tasks to model training.

[0031] Furthermore, the present invention also provides a vehicle-to-everything (V2X) multi-agent collaborative perception system, comprising: Network building module: Constructs a collaborative perception network, which includes: a self-aware agent and multiple collaborative agents; Data acquisition module: Acquires and processes sensing data through a collaborative sensing network to obtain processed sensing data; Intermediate Feature Module: The collaborative sensing network extracts features from the processed sensing data to obtain intermediate features; Collaborative agent processing module: Multiple collaborative agents adaptively sparsify the features to obtain sparse features, compress the sparse features to obtain a compressed feature representation, and transmit the compressed feature representation to the self-agent agent; Enhancement module: At the self-agent end, the received features of the cooperative agents are first decompressed and reconstructed, and then the features of all agents are enhanced in a temporal manner based on the corresponding historical features to obtain the enhanced features; Fusion module: Fuses the enhanced features of all agents to obtain fused features; Perception module: Constructs a vehicle-to-everything (V2X) perception model, and the detection model realizes perception based on the fused features.

[0032] This invention enhances the ability to characterize dynamic targets by improving temporal features, while effectively suppressing static background interference, thus improving the system's detection sensitivity and tracking stability for moving targets. Through adaptive sparsity optimization, the computational complexity of traditional global attention is reduced from quadratic to linear, significantly reducing computational overhead while maintaining the accuracy of multi-agent feature interaction. This makes the method more suitable for real-time operation on resource-constrained automotive edge devices. Attached Figure Description

[0033] Figure 1 Here is a flowchart of a multi-agent collaborative perception method for vehicle networking in one embodiment; Figure 2 This is a framework diagram of a multi-agent collaborative perception method for vehicle networking in one embodiment; Figure 3 This is a network structure diagram for difference-aware temporal feature enhancement in one embodiment; Figure 4 This is a network structure diagram of multi-agent feature fusion based on sparse guidance in one embodiment; Figure 5 This is a performance comparison result from one embodiment; Figure 6 This is a block diagram of a vehicle-to-everything (V2X) multi-agent collaborative perception system in one embodiment; Detailed Implementation To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] Example 1: This embodiment provides, as follows: Figure 1 The method for multi-agent cooperative perception in vehicle networking, as shown, includes: S1: Construct a collaborative sensing network, which includes: a self-aware agent and multiple collaborative agents; S2: Acquire and process sensing data through a collaborative sensing network to obtain processed sensing data; S3: The collaborative sensing network extracts features from the processed sensing data to obtain intermediate features; S4: Multiple collaborative agents adaptively sparsify the features to obtain sparse features, compress the sparse features to obtain a compressed feature representation, and transmit the compressed feature representation to the self-agent agent; S5: At the self-agent end, the received features of the cooperative agents are first decompressed and reconstructed, and then the features of all agents are enhanced in a temporal sequence to obtain the enhanced features. S6: Fuse the enhanced features of all agents to obtain the fused features; S7: Construct a vehicle-to-everything (V2X) perception model, and the detection model achieves perception based on the fused features.

[0035] This embodiment enhances the ability to characterize dynamic targets by augmenting temporal features, while effectively suppressing static background interference, thus improving the system's detection sensitivity and tracking stability for moving targets. Through adaptive sparsity reduction, the computational complexity of traditional global attention is reduced from quadratic to linear, significantly lowering computational overhead while maintaining the accuracy of multi-agent feature interaction. This makes the method more suitable for real-time operation on resource-constrained in-vehicle edge devices.

[0036] Example 2: This embodiment further discloses information based on Embodiment 1: Framework such as Figure 2 As shown Specifically, step S2 includes the following: In this embodiment, step S2 aims to complete the collection and spatial normalization of distributed sensing data. Specifically, the self-agent (i.e., the agent that executes the final sensing decision) first establishes a dynamic cooperative network with multiple neighboring cooperative agents based on the communication link status. At the beginning of each synchronization cycle (determined by the sensor sampling cycle of the self-agent), all agents within the network, including the self-agent and each cooperative agent, acquire raw point cloud data from their respective perspectives using the vehicle-mounted LiDAR sensor. Simultaneously, the self-agent broadcasts its own pose and other information via wireless communication technology. Upon receiving this information, the cooperative agents, combining their own high-precision positioning information, calculate a rigid transformation matrix from the local sensor coordinate system to the self-agent coordinate system. Subsequently, this transformation matrix is ​​used to reproject the agent's raw point cloud data, mapping all data to a unified spatial reference system centered on the self-agent. This coordinate alignment process eliminates spatial inconsistencies caused by differences in relative pose between vehicles, ensuring that sensing data from different perspectives are strictly aligned in space, facilitating subsequent collaborative feature extraction and fusion by subsequent modules.

[0037] In step S3, all agents use a shared-weight feature encoding network to process the point cloud data in their respective perceptual data. The main structure of this feature encoding network is based on the backbone network of PointPillars. Specifically, the network first converts the input point cloud data into a structured cylindrical feature tensor to facilitate subsequent convolutional network processing. Then, through feature transformation and aggregation within the network, the cylindrical feature tensor is further encoded into a two-dimensional high-order feature map with rich semantic information. For any agent... At any moment The intermediate features of the final output can be represented as This feature map contains One channel, and has Spatial resolution. The self-aware agent and all collaborating agents independently generate intermediate features within their respective perceptual ranges through this same process; these features will serve as input for subsequent fusion modules.

[0038] In step S4, adaptive sparse mask generation and sparse feature compression and transmission need to be performed sequentially. Further, step S4 includes the following detailed steps: S4.1 Each cooperative agent needs to use the adaptive sparse generator designed in this embodiment to process its local intermediate features. To generate a sparse mask corresponding to its intermediate features. In its implementation, this adaptive sparse generator first generates a confidence map from the intermediate input features through a lightweight network consisting of depthwise separable convolutions. This confidence map reflects the importance of the current intermediate features at each spatial location and in each channel. Then, a spatially sparse map is obtained through nonlinear activation and spatial smoothing. The calculation process is as follows:

[0039] in, For the Sigmoid function, The function is Gaussian smoothing. However, given the significant differences in feature sparsity requirements across different real-world scenarios (such as traffic scenarios on highways and in densely populated urban areas), a fixed sparsity rate can lead to transmission redundancy in simple scenarios or loss of key targets in complex scenarios. To address this issue, this embodiment designs a dynamic sparsity rate predictor. This predictor uses the confidence map... With intermediate features After concatenation, a dynamic sparsity rate is regressed using global average pooling and a two-layer multilayer perceptron. Its value reflects the proportion of information that needs to be retained in the current scene, and its specific calculation process is as follows:

[0040] The sparsity is constrained by... Within a certain range, to ensure the sparsity is reasonable. Ultimately, according to Determine the number of feature locations to retain. K And select a spatially sparse graph Center front K Generate a binary sparse mask at the position of the maximum value. .

[0041] S4.2 Each collaborative agent performs sparsification processing on the intermediate features based on the generated sparse mask, and then compresses and transmits the sparse features and the mask. In specific implementation, to optimize communication bandwidth while ensuring the integrity of key perception information, this embodiment adopts a two-stage strategy of "sparserification first, then compression." First, the original intermediate features are filtered using the sparse mask, retaining only the feature values ​​of key regions, greatly reducing the amount of data to be transmitted. Specifically, sparse features are obtained through the following sparsification process: : .

[0042] in This represents the element-wise multiplication operator. Subsequently, regarding sparse characteristics... The sparse mask will also be used in subsequent feature fusion stages. Compression is performed separately. For sparse feature compression, it is first converted into a low-bit integer representation through dynamic range quantization, the process of which can be expressed as:

[0043] Quantization bits As a key adjustable parameter, by reducing This approach can significantly improve the overall compression ratio at the cost of a small amount of precision, thus flexibly adapting to different channel conditions. Subsequently, lossless entropy coding (such as Zlib) is applied to the quantized data. This coding method adaptively utilizes the concentrated distribution and low information entropy of the sparsely quantized data to allocate short codewords to high-frequency values, thereby dynamically achieving extremely high compression efficiency while ensuring the lossless transmission of key feature information. For sparse mask compression, bit packing is used to encode the binary sparse mask... It can be converted into a byte stream, and can also be further compressed using entropy encoding.

[0044] Finally, the collaborative agent compresses the data packets. The data is transmitted to the autonomous agent via a wireless link. After receiving the data packet, the vehicle sequentially performs entropy decoding, bit unpacking, and dequantization operations to reconstruct the sparse mask without loss. It then recovers an approximate representation of the intermediate features. For this approximate representation of the intermediate features, a cascaded convolutional network is used to reconstruct them into high-quality intermediate features. This provides input for the subsequent collaborative feature fusion module.

[0045] In step S5, the self-agent enhances the received multi-agent features (including the self-agent's own features) through a difference-aware temporal fusion module. Further, step S5 includes the following details: S5.1 Historical Feature Caching and Update Mechanism. The self-agent needs to maintain a lightweight circular cache to store the intermediate features of all agents, including the self-agent, at the previous time step. This cache employs a first-in, first-out (FIFO) strategy. After completing the temporal feature enhancement processing for the current moment, it automatically overwrites the corresponding historical features with the features from the current moment, ensuring constant storage consumption. Specifically, when the agent is at moment... Received and decompressed features from each cooperative agent With mask Then, the self-agent reads the data from the cache of each agent (including the self-agent) in... Historical characteristics of a moment and the current characteristics of each agent Corresponding historical characteristics The input is fed into the difference-aware temporal fusion module. After temporal enhancement is completed, the historical features of each agent in the buffer are updated to the current features, i.e., execution is performed. The replacement operation ensures that when performing temporal feature enhancement, both the features of the current frame and the historical features of the previous frame can be utilized simultaneously. This preserves the necessary temporal context information for enhancing the current perception, while avoiding unlimited growth of storage space through a cyclic update mechanism.

[0046] S5.2 Multi-scale feature alignment and difference-aware fusion like Figure 3 As shown, in order to fully utilize temporal information and accurately capture dynamic changes in the scene, this embodiment employs a two-level processing differential-aware temporal fusion module, the core process of which is as follows: S5.2.1 Feature Alignment Based on Multi-Scale Optical Flow In tasks involving agents processing and perceiving continuous video, pixel-level displacements occur between consecutive frames due to the motion of objects or sensors. If misaligned historical features are directly applied... With current features When fusing features, these displacements can cause spatial mismatches, introducing artifacts such as motion blur, ghosting, or texture misalignment into the fusion result. To effectively utilize historical frame information and preemptively eliminate the effects of inter-frame motion, a feature alignment module based on multi-scale optical flow was first designed to align historical features... Precisely align to the current feature The feature space. This module constructs a multi-scale pyramid from coarse to fine, at each scale... Above, a lightweight optical flow estimation network is used to compute the optical flow field from historical features to current features. The network is initialized with the optical flow upsampled at the previous scale, and the residual optical flow is calculated to distort the historical features. This process can be expressed by the following formula: = + (Concat[ , ( , )]) in, For the first The optical flow estimation network is a multi-layer convolutional network consisting of multiple convolutional layers and ReLU functions. For bilinear sampling, there is a feature warping operation, and for bilinear sampling, there is a feature concatenation operation.

[0047] Specifically, bilinear interpolation is first used to extract historical features. and current features Downsampling to a specified scale is used to construct a feature pyramid with three scales, with resolutions of [H / 4, W / 4], [H / 2, W / 2], and [H, W] respectively. Then, a three-layer optical flow estimation network with input scales of [H / 4, W / 4], [H / 2, W / 2], and [H, W] is used to estimate optical flow from historical features to current features in ascending order of scale. For the first layer of the pyramid (… Initial optical flow field Set to zero. At each scale Above, the optical flow estimated at the previous scale. Upsample to the current scale, and use this upsampled optical flow through a bilinear sampling distortion operation. Preliminary alignment of historical features yields distorted historical features at this scale. ( , Subsequently, the current scale features will be... It is then stitched together with the distorted historical features and input into the optical flow network at the current scale. The residual optical flow is calculated in the middle, and this scale The final optical flow field The optical flow is obtained by adding the residual optical flow to the upsampled initial optical flow. This process is progressive, using coarse-scale (low-resolution) optical flow to capture large-scale motion and fine-tuning with fine-scale (high-resolution) optical flow to gradually improve alignment accuracy. Finally, after optical flow estimation is completed at the highest resolution scale, the final optical flow field is obtained. (Specific settings) ). Utilizing the historical characteristics of this optical flow field on the original resolution. The distortion is performed, and the final output is related to the current feature. Enhanced historical features of spatial alignment Based on the aligned historical features, the subsequent difference-aware temporal fusion module can focus more on identifying and processing real scene content changes (such as newly appearing objects and changes in lighting) rather than being disturbed by a large number of motion artifacts.

[0048] S5.2.2 Enhancement of Temporal Features in Difference Perception Building upon multi-scale feature alignment, the difference-aware temporal fusion module enhances the dynamic perception capability of current features by explicitly modeling and utilizing the changes between aligned historical features and current features. The core innovation of this module lies in treating the original feature differences as an independent information source for specialized processing, rather than simply fusing aligned historical features with current features. The specific implementation includes the following key technical aspects: (1) Original feature difference quantization. First, calculate the original difference tensor between the current feature and the aligned historical features:

[0049] This difference tensor It directly encodes the dynamic changes in the scene caused by the relative motion between the vehicle and the target.

[0050] (2) Differential Feature Enhancement Path. Considering that the original differential tensor may contain noise and has limited feature representation ability, a dedicated differential processing network is designed to enhance it. This network adopts a bottleneck structure, first compressing the number of channels through dimensionality reduction convolution, then performing ReLU nonlinear activation and batch normalization, and finally constraining the feature values ​​to the [-1,1] interval through dimensionality increase convolution and Tanh activation function to highlight significant positive and negative change patterns. This process can be expressed as follows: ReLU(BN( )) in and The weights are for dimensionality reduction and dimensionality increase, respectively. The Tanh activation function ensures that the output can represent both positive and negative changes.

[0051] (3) Significance detection of changes. Simultaneously, a lightweight significance detection network is used to generate a significance map of changes based on the absolute value of the differences. This is used to quantify the dynamics of each spatial location:

[0052] in This is a lightweight saliency detection network, which consists of two layers of dimension-reduced convolutions and a ReLU nonlinear activation function. The saliency plot is the Sigmoid function. It identifies the spatial regions where significant movement occurs.

[0053] (4) Dual-stream feature modulation. The current feature and enhanced differential features need to be processed independently to enhance their ability to perceive dynamic changes. For the current feature stream, spatial modulation is performed using a change saliency map to enhance the feature response in the changing regions:

[0054] For the differential feature stream, the same saliency plot weighting is used to highlight the contribution of the regions of change:

[0055] in, and The modulation networks are for the current features and the enhanced differential features, respectively. Both networks employ... It is constructed using convolutional layers, batch normalization layers, and ReLU nonlinear activation.

[0056] (5) Adaptive temporal feature fusion. The modulated current features are then fused together. Differences after modulation and aligned historical features The concatenation is performed along the channel dimension using a gated network based on global average pooling. Generate adaptive fusion weights: ( (GAP[ , , ])) in, It consists of two layers of dimensionality-reduced convolutions and a ReLU nonlinear activation function. The weights... It can be broken down into features for modulating the current characteristics. Differences after modulation The two feature branches are weighted separately. and Then, the two weighted features are concatenated and fed into a temporal fusion network. This process yields preliminary fusion results, enabling temporal enhancement of the current features. It can be represented as follows: ]) Among them, time-series fusion network use It is constructed using dimensionality-reducing convolutions, batch normalization layers, and ReLU nonlinear activation.

[0057] (6) Residual connection and output. Finally, the temporal feature fusion results are... Compared with the original current features By combining residual connections, the residual strength is controlled by the average significance. The final output yields the current features after difference-aware temporal enhancement. This process can be represented by the following formula:

[0058] in, Represents the addition operator; For adaptive residual gating, It is a saliency plot exist The process involves averaging upwards. This series of difference-driven designs effectively enhances the feature representation of dynamic targets while suppressing static background interference, significantly improving the sensitivity to moving targets.

[0059] Specifically, step S6 includes the following: To address the shortcomings of traditional feature fusion methods in the collaborative perception feature fusion stage, such as weak spatial modeling, high computational redundancy, and insufficient robustness to complex scenes, this embodiment proposes a lightweight solution. This solution leverages sparse priors as a breakthrough point, mining cross-vehicle spatial associations through sparse-guided cross-attention, significantly reducing computational load. Simultaneously, it utilizes attention to dynamically calibrate multi-source features, effectively improving representation stability and fusion accuracy under extreme conditions such as occlusion and truncation. In this embodiment, through the sparse-guided attention fusion module, the agent needs to process all intermediate features and sparse masks of each agent after differential perception temporal enhancement to achieve multi-agent feature fusion. The network structure of the sparse-guided multi-agent fusion module is as follows: Figure 4 As shown, it specifically includes three core modules: sparse perception relative position encoding, cross-agent sparse cross attention, and adaptive multi-source feature fusion.

[0060] S6.1 Sparse Perception Relative Position Coding To overcome the limitations of traditional relative position encoding in collaborative sensing scenarios, this embodiment proposes a sparse sensing relative position encoding mechanism. Traditional relative position encoding methods assign a uniform and isotropic positional bias to the entire feature map, which cannot adapt to the non-uniform distribution of key target regions in traffic scenarios, resulting in insufficient accuracy in modeling the spatial relationships of semantically important regions.

[0061] This embodiment achieves accurate spatial relationship modeling by combining directional difference sensing fields with dynamic sparse modulation, which can specifically enhance the positional awareness of key areas while suppressing interference from background areas. First, two independent and learnable positional bias tables are constructed, corresponding to the height and width dimensions respectively: Indicates a positional offset in the height direction. This indicates the positional offset in the width direction, where , The maximum relative distance in each dimension. This dual-dashboard design allows the model to employ differentiated perception ranges in the height and width directions, effectively adapting to the geometric characteristics of driving scenarios.

[0062] For any two positions in the feature map and Calculate the relative position offset:

[0063] After cropping the offset to the preset range, the corresponding offset value is indexed from the position offset table:

[0064] in, This represents the clipping operation. Based on this, a dynamic modulation mechanism based on sparse masks is introduced. Sparse mask A spatial modulation map is generated by processing the data through a lightweight convolutional network.

[0065] in and These represent the kernel size as follows: and The convolution operation is performed. By performing average pooling along the spatial dimension of the modulation map, the modulation factors in the height and width directions are obtained:

[0066] Applying a direction-specific modulation factor to the position bias generates the final sparse-aware position code:

[0067] in This is a learnable scaling parameter. The positional encoding is added to the query and key features before attention computation, providing a semantically rich spatial prior for subsequent cross-agent sparse cross-attention mechanisms.

[0068] S6.2, Cross-agent sparse cross-attention To overcome the problems of high computational complexity and ineffective utilization of semantic sparsity in traditional attention mechanisms during collaborative perception, this embodiment proposes a cross-agent sparse cross-attention mechanism. This mechanism aims to achieve targeted feature interaction between the self-agent and the cooperative agent, avoiding indiscriminate global interaction with all cooperative vehicles, thereby significantly reducing the computational burden while maintaining perception accuracy.

[0069] Specifically, three independent feature projection pathways are first constructed: the self-agent generates query features through the query projection network. Each collaborative intelligent agent Key features are generated using key projection networks and value projection networks, respectively. Sum value characteristics Then, the projected features are decomposed into multi-head representations, that is, the channel dimensions. Divided into Each subspace has a dimension of [number] heads. This allows for the capture of diverse contextual information. At this point, the positional encoding obtained in "S6.1, Sparse-aware relative positional encoding" will be added to the query and key features. Based on this, a sparse-guided axial attention calculation method is then employed to decompose the traditional global dot product attention into two axial paths: horizontal and vertical.

[0070] In the horizontal path, for each row Self-agent query vector Includes all of this line A query representation of a location. Collaborative intelligent agent. The key-value pairs are only used in sparse masks. The key positions of the marker are calculated, that is... and ,in Indicates the first The number of key positions selected in a row. Horizontal attention calculation is represented as:

[0071] Vertical paths use a similar principle, for each column , vehicle query vector Includes all of this column A query at a given location represents a key-value pair of the collaborative agent. and Calculated only at critical locations. Vertical attention is calculated as follows:

[0072] This axial decomposition design reduces computational complexity from that of traditional attention. Reduce to The introduction of sparse masks further reduces the complexity to ,in This indicates the total number of key positions. Specifically, when... In this case, the aforementioned cross-attention calculation degenerates into the self-attention of the vehicle agent, ensuring that the vehicle features can also be adequately modeled in context.

[0073] Next, the outputs from all attention heads are concatenated along the channel dimension and multi-head information is integrated through linear projection. Finally, the interaction features of the horizontal and vertical paths are fused with weighted summation and residual connections:

[0074] in For learnable scaling factor, The original value features are used as residual connections to preserve the original information. Subsequently, the interaction features between the self-agent and the cooperative agents are analyzed. It will be fed into the subsequent adaptive multi-source feature fusion module.

[0075] S6.3 Adaptive Multi-Source Feature Fusion To effectively integrate the perceptual features of multiple cooperative agents and dynamically balance their contributions, this embodiment employs an interactive perception-based adaptive feature fusion mechanism. This mechanism generates spatial importance weights and normalizes these weights to achieve adaptive weighted fusion of features, thereby highlighting information-rich regions and reducing noise or redundancy. Specifically, it first addresses the interaction features between each pair of cooperative agents and their self-agent. A spatial importance graph is generated using a weight calculation network consisting of multiple dimensionality-reducing convolutions, ReLU functions, and regularization layers. This diagram reflects the importance of the features of each agent at different spatial locations; then, along... The dimension (i.e., the number of agents) is normalized using Softmax to ensure that the sum of all weights at each spatial location is 1, thus forming normalized weights. This normalization process can be represented as:

[0076] in, Represents a set of collaborative intelligent agents. Indexing the interaction pairs of self-aware agents and collaborative agents. The normalized weights, It is an exponential function, whose function is to ensure that all weights are positive and to reinforce the relative differences between weights. Finally, the interaction features of all agents are fused through a weighted summation to obtain the fused features:

[0077] This mechanism can dynamically adjust the spatial weights of the features of each agent, ensuring that key perceptual information is fully preserved during the fusion process, thereby enhancing the robustness and accuracy of the overall system.

[0078] Specifically, step S7 includes the following: Based on the output of the aforementioned feature fusion module, this embodiment generates the final 3D object detection result through a lightweight detection head network. This detection head employs a dual-branch parallel structure, responsible for object classification and bounding box regression tasks respectively. Specifically, the classification branch is implemented using a 1×1 convolutional layer, with its input channels equal to the fused feature dimension and its output channels equal to the product of the preset number of anchor boxes and the number of categories. This branch is used to predict the confidence that each anchor box at each spatial location contains a specific category of object. The regression branch also uses a 1×1 convolutional layer, with its output channels equal to the product of the number of anchor boxes and the number of bounding box parameters. This branch is responsible for regressing the precise geometric parameters of each object bounding box, including the center coordinates in 3D space. ,size and yaw angle Furthermore, the final fused global context features are obtained from the feature fusion module in step S5. Then, it is input into the aforementioned detection head network to output the collaborative sensing results. The entire process (steps S1-S6) significantly reduces communication and computing overhead while ensuring accuracy, making it suitable for edge device deployment.

[0079] To effectively train the overall network in the invention, this embodiment designs a multi-task loss function, which consists of two parts: bounding box regression loss and classification loss. The overall optimization objective is formed by weighted summation.

[0080] First, the bounding box regression loss employs a weighted smooth L1 loss function, which smoothly transitions between L1 and L2 losses and exhibits better robustness to outliers. Its mathematical expression is:

[0081] in, It is the index of the anchor box. To predict bounding box parameters, To correspond to the actual annotation box parameters, This is an indicator function (1 for positive samples and 0 for negative samples). The number of positive samples. The SmoothL1 function is defined as:

[0082] Here This is a hyperparameter that controls the transition range between L1 and L2 losses.

[0083] Secondly, the classification loss employs a focal loss mechanism, specifically designed to address the severe imbalance between foreground and background samples in 3D object detection. This loss function introduces an adjustable focusing parameter to reduce the weight of easily classified samples, causing the model to focus more on difficult-to-classify samples. The focal loss expression is as follows:

[0084] in, The target confidence level predicted by the model. To balance the weights for each category, This is a focusing parameter used to adjust the weight ratio between easy and difficult samples.

[0085] Ultimately, the overall loss function is the weighted sum of the two losses mentioned above:

[0086] in, and These are the weight coefficients for regression loss and classification loss, used to balance the contributions of the two tasks to model training. By allocating appropriate weights, it is possible to ensure that classification and regression tasks are optimized synergistically, jointly improving overall detection accuracy.

[0087] Specifically, during training, the parameters related to the loss function are set as follows: , , , .

[0088] In the specific implementation of the multi-agent cooperative perception method for vehicle-to-everything (V2X) based on enhanced spatiotemporal interaction and sparse mechanisms, labeled point cloud data and their labels are used for training, and the labeled point clouds are cropped in range. Taking cooperative perception in traffic scenarios as an example, the independent point cloud range of each agent is generally cropped as follows: a hexahedral region centered on itself, extending 140m in front and behind, 40m to the left and right, and 4m above the ground, is used as the effective input, filtering out redundant points while retaining the complete road target. At the same time, cooperative perception only includes neighboring cooperative agents within 100m of the self-agent agent. This threshold strikes a balance between communication bandwidth and field of view differences—if it is too far, the reliability of the communication link decreases, the parallax increases significantly, and the introduced low-correlation point clouds dilute the features, thus affecting the accuracy of cooperative perception.

[0089] During model training, the Adam optimizer is used for the entire model network, with an initial loss rate of 0.001, a weight decay of 0.0001, and a stepwise learning rate reduction strategy, where the learning rate is reduced to 10% of its original value every 10 rounds, and the batch size is 4.

[0090] To verify and compare the model performance of this embodiment, experiments were conducted on publicly available cooperative perception datasets—the OPV2V dataset and the V2XSet dataset. OPV2V is a simulated real-world vehicle-to-vehicle cooperative perception dataset, containing multi-vehicle cooperative perception scenarios with 2-7 vehicles; while V2XSet is a simulated real-world vehicle-to-infrastructure cooperative perception dataset, containing cooperative perception scenarios with 2-5 vehicles and one roadside sensing facility. Evaluation metrics used were AP@0.3, AP@0.5, and AP@0.7, which are average precision (AP) values ​​at intersection-to-union (IoU) thresholds of 0.3, 0.5, and 0.7, respectively, effectively measuring 3D object detection performance. IoU is the ratio of the overlap area between the predicted bounding box and the ground truth bounding box to their union area, used to measure object detection localization accuracy. To reduce randomness, the average inference result was used as the experimental result. Specifically, under the same key parameter settings and the same data preprocessing method, the method of this embodiment was used to conduct experiments and comparisons with the No fusion (no multi-agent intermediate feature fusion, only single-agent perception) method and existing advanced methods such as Where2comm, V2X-Vit, and V2VNet. The final collaborative perception performance was compared as follows: Figure 5 As shown. According to Figure 5 The perceptual performance of each method on the OPV2V and V2XSet datasets shows that the method in this embodiment has significant perceptual performance in vehicle-to-vehicle and vehicle-to-infrastructure scenarios. Compared with other methods, this embodiment has higher robustness and 3D target detection accuracy in complex scenarios.

[0091] Communication efficiency is significantly improved. Intelligent data filtering based on scene content perception is achieved through an adaptive sparse generator and a dynamic sparsity predictor. A lossless entropy coding dynamic compression transmission method is used, which greatly reduces the amount of data transmitted for features while ensuring the integrity of key perception information. This effectively alleviates the communication bandwidth pressure in the vehicle-to-everything (V2X) environment. Moreover, the above adaptive transmission method has good scene generalization ability and practical deployment feasibility.

[0092] The accuracy of temporal perception is significantly enhanced. By adopting a difference-aware temporal fusion mechanism, the feature representation capability of dynamic targets is strengthened by explicitly modeling the feature differences between the current frame and the aligned historical frames. At the same time, static background interference is effectively suppressed, which improves the system's detection sensitivity and tracking stability for moving targets.

[0093] The computational resource utilization is effectively optimized. Through a sparse-guided cross-attention mechanism, the computational complexity of traditional global attention is reduced from quadratic to linear. While maintaining the accuracy of multi-agent feature interaction, the computational overhead is significantly reduced, making the method more suitable for real-time operation on resource-constrained in-vehicle edge devices.

[0094] Compared to other methods, this embodiment offers a comprehensive improvement in perception performance. The integrated design of this embodiment coordinates and optimizes the three dimensions of communication, computing, and perception. In complex urban road scenarios, especially in challenging environments such as occlusion and blind spots, it can achieve more accurate and robust 3D target detection performance, providing a more reliable environmental perception guarantee for advanced autonomous driving.

[0095] Example 3: This embodiment provides, as follows: Figure 6 The vehicle-to-everything (V2X) multi-agent cooperative perception system shown includes: Network building module: Constructs a collaborative perception network, which includes: a self-aware agent and multiple collaborative agents; Data acquisition module: Acquires and processes sensing data through a collaborative sensing network to obtain processed sensing data; Intermediate Feature Module: The collaborative sensing network extracts features from the processed sensing data to obtain intermediate features; Collaborative agent processing module: Multiple collaborative agents adaptively sparsify the features to obtain sparse features, compress the sparse features to obtain a compressed feature representation, and transmit the compressed feature representation to the self-agent agent; Enhancement module: At the self-agent end, the received features of the cooperative agents are first decompressed and reconstructed, and then the features of all agents are enhanced in a temporal manner based on the corresponding historical features to obtain the enhanced features; Fusion module: Fuses the enhanced features of all agents to obtain fused features; Perception module: Constructs a vehicle-to-everything (V2X) perception model, and the detection model realizes perception based on the fused features.

[0096] This embodiment enhances the ability to characterize dynamic targets by augmenting temporal features, while effectively suppressing static background interference, thus improving the system's detection sensitivity and tracking stability for moving targets. Through adaptive sparsity reduction, the computational complexity of traditional global attention is reduced from quadratic to linear, significantly lowering computational overhead while maintaining the accuracy of multi-agent feature interaction. This makes the method more suitable for real-time operation on resource-constrained in-vehicle edge devices.

Claims

1. A multi-agent cooperative perception method for vehicle-to-everything (V2X) networks, characterized in that, include: S1: Construct a collaborative sensing network, which includes: a self-aware agent and multiple collaborative agents; S2: Acquire and process sensing data through a collaborative sensing network to obtain processed sensing data; S3: The collaborative sensing network extracts features from the processed sensing data to obtain intermediate features; specifically: The processed perception data includes point cloud data. The self-agent and collaborative agents use a shared-weight feature encoding network to extract features from the point cloud data. The feature encoding network uses a PointPillars backbone network to convert the point cloud data into structured cylindrical feature tensors, and further extracts them into high-order feature maps in a two-dimensional bird's-eye view format. Each agent ultimately extracts a feature map with a dimension of... intermediate features ,in The number of feature channels, and Spatial dimensions of the feature map S4: Multiple collaborative agents adaptively sparsify the features to obtain sparse features, compress the sparse features to obtain a compressed feature representation, and transmit the compressed feature representation to the self-agent agents; specifically: S4.1: Construct an adaptive sparse generator, through which each cooperative agent processes its local intermediate features. To generate a sparse mask corresponding to its intermediate features. In its implementation, the adaptive sparse generator first generates a confidence map from the intermediate input features through a lightweight network consisting of depthwise separable convolutions. The confidence map reflects the importance of the current intermediate features at each spatial location and in each channel; then, a spatially sparse map is obtained through nonlinear activation and spatial smoothing. The calculation process is as follows: in, For the Sigmoid function, It is a Gaussian smoothing function; Simultaneously, a dynamic sparsity rate predictor is introduced, which adaptively predicts a dynamic sparsity rate based on intermediate features and confidence information through a multilayer perceptron. Its value reflects the proportion of information that needs to be retained in the current scene; finally, a Top-Sparse Map is performed on the spatial sparse graph based on this sparsity rate. K Filter and generate a binary sparse mask. The total number of spatial locations that were selected ; S4.2: Each cooperative agent performs sparsification processing on the intermediate features based on the generated sparse mask, and then compresses and transmits the sparse features and the mask; in specific implementation, the cooperative agents first utilize the sparse mask... For intermediate features Sparsity processing is performed to obtain sparse features. In the sparse feature model, only feature values ​​at important spatial locations are retained to adapt to the limited communication bandwidth in real-world scenarios. Subsequently, based on the characteristics of the feature value distribution after sparsification, the sparse features are quantized and lossless entropy encoded to obtain a compressed representation. ; and sparse masking Binarization and bit packing are performed to obtain Next, compress the data packet. Transmitted to the self-intelligent agent via wireless communication; S5: At the self-agent end, the received collaborative agent features are first decompressed and reconstructed, and then the features in all agents are enhanced in a temporal sequence to obtain the enhanced features. S6: Fuse the enhanced features of all agents to obtain the fused features; S7: Construct a vehicle-to-everything (V2X) perception model, and the detection model achieves perception based on the fused features.

2. The multi-agent cooperative perception method for vehicle networking according to claim 1, characterized in that, In step S2, specifically: in each processing cycle, the self-intelligent agent and multiple cooperative intelligent agents synchronously collect local sensor data; among them, the cooperative intelligent agents also need to receive metadata including pose broadcast by the self-intelligent agent; subsequently, each cooperative intelligent agent transforms its own collected raw perception data into the vehicle coordinate system according to the coordinate transformation relationship, completes the spatial alignment of multi-source data, and obtains the processed perception data.

3. The multi-agent cooperative perception method for vehicle networking according to claim 1, characterized in that, In step S5, specifically: Historical feature caching and update mechanism: The self-agent agent includes a feature alignment module and a difference-aware temporal fusion module; the self-agent agent needs to maintain a lightweight circular cache to store the intermediate features of all agents, including the self-agent agent, at the previous time step. The cache uses a first-in, first-out (FIFO) strategy. After completing the temporal feature enhancement processing for the current moment, it automatically overwrites the corresponding historical features with the features of the current moment. Specifically, when the agent is at moment... Received and decompressed features from each cooperative agent With mask Then, the self-regulating agent reads the information from the cache of each agent. Historical characteristics of a moment and the current characteristics of each agent Corresponding historical characteristics The input is fed into the feature alignment module and the difference-aware temporal fusion module. After temporal enhancement is completed, the historical features of each agent in the buffer are updated to the current features.

4. The multi-agent cooperative perception method for vehicle networking according to claim 3, characterized in that, The feature alignment module is as follows: Based on the feature alignment module of multi-scale optical flow, historical features are... Precisely align to the current feature The feature space; this module constructs a multi-scale pyramid from coarse to fine, at each scale Above, a lightweight optical flow estimation network is used to compute the optical flow field from historical features to current features. Using the upsampled optical flow from the previous scale as initialization, and calculating the residual optical flow to distort historical features, this process is expressed by the following formula: = + (Concat[ , ( , )]) in, For the first Layer optical flow estimation network, Concat is a feature warping operation based on bilinear sampling, and Concat is a feature concatenation operation. First, use bilinear interpolation to extract historical features. and current features Downsampling to a specified scale to construct Feature pyramids at various scales; used Optical flow estimation networks with different input scales are used to estimate optical flow from historical features to current features in ascending order of scale; for the first layer of the pyramid, the initial optical flow field is... Set to zero; at each scale Above, the optical flow estimated at the previous scale. Upsample to the current scale, and use this upsampled optical flow through a bilinear sampling distortion operation. Preliminary alignment of historical features yields distorted historical features at this scale. ( , ); to the current scale features It is then stitched together with the distorted historical features and input into the optical flow network at the current scale. The residual optical flow is calculated in the middle, and this scale The final optical flow field The optical flow is obtained by adding the residual optical flow to the upsampled initial optical flow. This process is progressive, using coarse-scale optical flow to capture large-scale motion and fine-tuning with fine-scale optical flow to gradually improve alignment accuracy. Finally, after optical flow estimation is completed at the highest resolution scale, the final optical flow field is obtained. ; Utilizing the historical characteristics of the original resolution using this optical flow field The distortion is performed, and the final output is related to the current feature. Enhanced historical features of spatial alignment .

5. The multi-agent cooperative perception method for vehicle networking according to claim 4, characterized in that, The difference-aware temporal fusion module is specifically as follows: First, calculate the current features and the aligned enhanced historical features. The original difference tensor between them: This difference tensor It directly encodes the dynamic changes in the scene caused by the relative motion between the vehicle and the target; A bottleneck structure is adopted, firstly compressing the number of channels through dimensionality reduction convolution, then applying ReLU nonlinear activation and batch normalization, and finally constraining the feature values ​​to the [-1,1] interval through dimensionality increase convolution and Tanh activation function, highlighting significant positive and negative variation patterns; the process is expressed as follows: ReLU(BN( )) in and The convolution weights are for dimensionality reduction and dimensionality increase, respectively. The Tanh activation function ensures that the output can represent both positive and negative changes. Constructing a lightweight saliency detection network A significance map of change is generated based on the absolute value of the difference. This is used to quantify the dynamics of each spatial location: in The saliency plot is the Sigmoid function. The spatial region where significant movement occurred was identified; The current features and enhanced differential features are processed independently to improve their ability to perceive dynamic changes; for the current feature stream, spatial modulation is performed using a change saliency map to enhance the feature response in regions of change. For the differential feature stream, the same saliency plot weighting is used to highlight the contribution of the regions of change: in, and These are modulation networks for the current features and for enhancing differential features, respectively; Modulated current features Differences after modulation and aligned historical features The concatenation is performed along the channel dimension using a gated network based on global average pooling. Generate adaptive fusion weights: ( (GAP[ , , ])) in, It consists of two layers of dimensionality-reduced convolution and a ReLU nonlinear activation function; the weights It can be broken down into features for modulating the current characteristics. Differences after modulation The two feature branches are weighted separately. and Then, the two weighted features are concatenated and fed into a temporal fusion network. To obtain preliminary fusion results and achieve temporal enhancement of the current features; this process is represented as follows: ]) Finally, the temporal feature fusion results are... Compared with the original current features By combining residual connections, the residual strength is controlled by the average significance; the final output is the current feature after difference-aware temporal enhancement. The process is represented by the following formula: in, Represents the addition operator; For adaptive residual gating, It is a saliency plot exist The operation of averaging.

6. The multi-agent cooperative perception method for vehicle networking according to claim 5, characterized in that, In step S6, specifically: A sparsely guided multi-agent fusion module is constructed, whose network structure includes a sparse perception relative position encoding submodule, a cross-agent sparse cross attention submodule, and an adaptive multi-source feature fusion submodule. Sparse-aware relative position encoding submodule: Enhanced spatial modeling capabilities for key regions are achieved through directional difference sensing fields and sparse mask-based weight modulation; two independent learnable embedding tables are maintained. and ,in , The maximum relative distance in each dimension. For attention head dimension; For position pairs in the feature map and relative offset , After pruning, the index corresponds to the embedding. and ; Sparse mask Generating direction-specific modulation factors using a lightweight convolution modulator: in For the Sigmoid function, and They represent respectively by and Convolutional blocks are formed by convolutions; obtained by averaging along the width and height dimensions. Independent modulation of the position embedding in the height and width dimensions: in A learnable scaling factor; Adding attention to query and key features before attention computation enhances the spatial reasoning ability of subsequent cross-agent sparse cross attention for key regions. Cross-agent sparse cross-attention submodule: The self-aware agent generates query features through convolutional blocks. Each collaborative intelligent agent Key-value pairs are generated using two convolutional blocks. ;Will and Projected onto having In a multi-headed representation space, so as to... Channel dimensions are divided into Multiple subspaces are used to capture different contextual information; position encoding will be added to query and key features; The traditional dot product attention is decomposed into horizontal and vertical paths, thereby reducing the computational complexity from Reduce to In the horizontal path, Containing lines The query vector for all positions, and Corresponding sparse mask Selected Individual lines The key-value vector at the key position; the same applies to vertical paths; the calculation for cross-agent sparse cross attention is as follows: in, and They represent and The dimension; reducing computational complexity from Further down to ,in ,in Indicates the total number of important positions; when At this point, the aforementioned cross-agent sparse cross-attention computation degenerates into self-attention computation on the intermediate features of the self-agent; the outputs from all attention heads are concatenated and linearly projected to form interaction features on each path; finally, the interaction features on the horizontal and vertical paths are summed and connected by weighted sums and residuals to obtain the final representation, which is represented by the following equation: in For learnable scaling factor, The final interaction features are then fed into the adaptive multi-source feature fusion module, which then feeds the interaction features of the self-agent and the cooperative agents into the subsequent interaction features. Adaptive multi-source feature fusion: For each interaction feature, a weight calculation network is used. Generate a spatial importance graph Then, along The dimension is normalized using Softmax, and this process can be represented as: in, Represents a set of collaborative intelligent agents. Indexing the interaction pairs of self-aware agents and collaborative agents. It is an exponential function, whose function is to ensure that all weights are positive and to reinforce the relative differences between weights; the final fused features are obtained by weighted summation: 。 7. The multi-agent cooperative perception method for vehicle networking according to claim 6, characterized in that, In step S7, specifically: The vehicle-to-everything (V2X) perception model is a lightweight detection head network. This detection head employs a dual-branch parallel structure, responsible for object classification and bounding box regression tasks respectively. The classification branch uses a convolutional kernel with a size of [missing information]. The convolutional layer implementation is used to predict the confidence that each anchor box at each spatial location contains a target of a specific category; The regression branch also adopts Convolutional layers are responsible for regressing the precise geometric parameters of each object bounding box, including the center coordinates in 3D space. ,size and yaw angle The final fused global context features Input the above-mentioned detection head network and output the collaborative sensing results; The vehicle-to-everything (V2X) perception model is trained using a multi-task loss function, which consists of two parts: bounding box regression loss and classification loss. The overall optimization objective is formed by weighted summation. First, the bounding box regression loss employs a weighted smooth L1 loss function, which smoothly transitions between L1 and L2 losses and exhibits better robustness to outliers; its mathematical expression is: in, It is the index of the anchor box. To predict bounding box parameters, To correspond to the actual annotation box parameters, For indicator functions, The number of positive samples; the SmoothL1 function is defined as: Here This is a hyperparameter that controls the transition range between L1 and L2 losses; Secondly, the classification loss adopts the focus loss mechanism; the focus loss expression is: in, The target confidence level predicted by the model. To balance the weights for each category, This is a focusing parameter used to adjust the weight ratio between easy and difficult samples; Ultimately, the overall loss function is the weighted sum of the two losses mentioned above: in, and These are the weight coefficients for regression loss and classification loss, used to balance the contributions of the two tasks to model training.

8. A vehicle-to-everything (V2X) multi-agent collaborative perception system, characterized in that, include: Network building module: Constructs a collaborative perception network, which includes: a self-aware agent and multiple collaborative agents; Data acquisition module: Acquires and processes sensing data through a collaborative sensing network to obtain processed sensing data; Intermediate Feature Module: The collaborative sensing network extracts features from the processed sensing data to obtain intermediate features; specifically: The processed perception data includes point cloud data. The self-agent and collaborative agents use a shared-weight feature encoding network to extract features from the point cloud data. The feature encoding network uses a PointPillars backbone network to convert the point cloud data into structured cylindrical feature tensors, and further extracts them into high-order feature maps in a two-dimensional bird's-eye view format. Each agent ultimately extracts a feature map with a dimension of... intermediate features ,in The number of feature channels, and Spatial dimensions of the feature map Collaborative agent processing module: Multiple collaborative agents adaptively sparsify the features to obtain sparse features, compress the sparse features to obtain a compressed feature representation, and transmit the compressed feature representation to the self-agent agents; specifically: An adaptive sparse generator is constructed, through which each cooperative agent processes its local intermediate features. To generate a sparse mask corresponding to its intermediate features. In its implementation, the adaptive sparse generator first generates a confidence map from the intermediate input features through a lightweight network consisting of depthwise separable convolutions. The confidence map reflects the importance of the current intermediate features at each spatial location and in each channel; then, a spatially sparse map is obtained through nonlinear activation and spatial smoothing. The calculation process is as follows: in, For the Sigmoid function, It is a Gaussian smoothing function; Simultaneously, a dynamic sparsity rate predictor is introduced, which adaptively predicts a dynamic sparsity rate based on intermediate features and confidence information through a multilayer perceptron. Its value reflects the proportion of information that needs to be retained in the current scene; finally, a Top-Sparse Map is performed on the spatial sparse graph based on this sparsity rate. K Filter and generate a binary sparse mask. The total number of spatial locations that were selected ; Each cooperative agent performs sparsification on intermediate features based on the generated sparse mask, and then compresses and transmits the sparse features and the mask. In specific implementation, the cooperative agents first utilize the sparse mask... For intermediate features Sparsity processing is performed to obtain sparse features. In the sparse feature model, only feature values ​​at important spatial locations are retained to adapt to the limited communication bandwidth in real-world scenarios. Subsequently, based on the characteristics of the feature value distribution after sparsification, the sparse features are quantized and lossless entropy encoded to obtain a compressed representation. ; and sparse masking Binarization and bit packing are performed to obtain Next, compress the data packet. Transmitted to the self-intelligent agent via wireless communication; Enhancement module: At the self-agent end, the received collaborative agent features are first decompressed and reconstructed, and then the temporal feature enhancement is performed on all agent features based on the corresponding historical features to obtain the enhanced features; Fusion module: Fuses the enhanced features of all agents to obtain fused features; Perception module: Constructs a vehicle-to-everything (V2X) perception model, and the detection model realizes perception based on the fused features.

Images