Motion perception space-time compensation method and system for vehicle-road cooperation three-dimensional target detection

Abstract

The present application relates to a kind of motion perception space-time compensation method and system for vehicle-road cooperation three-dimensional target detection, belong to intelligent transportation and automatic driving perception technical field.The method and system first obtain the laser radar point cloud data of ego vehicle and collaboration node, and extract bird's eye view features and share collaboration features;Then according to communication delay and historical speed field calculation motion prior displacement, combined with the local correlation of current features and delayed collaboration features residual error prediction, obtain compensation offset and space-time deformation is carried out to delayed feature;Then fusion is compensated after multi-source feature, and utilize time sequence memory, motion prior and occlusion perception mechanism to enhance weak response area;Finally, according to the enhanced feature output three-dimensional target detection result, and update speed field for subsequent compensation.The present application can reduce the misplacement of dynamic target features caused by communication delay, improve the accuracy and robustness of three-dimensional target detection under occlusion and sparse scene.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation and autonomous driving perception technology, and relates to a motion perception spatiotemporal compensation method and system for vehicle-road cooperative three-dimensional target detection. In particular, it relates to a vehicle-road cooperative perception technology that performs delay compensation, temporal enhancement and multi-agent fusion on the bird's-eye view features of the vehicle, road and cooperative vehicles under the conditions of communication delay, positioning error, occlusion and sparse lidar observation. Background Technology

[0002] With the development of autonomous driving and intelligent transportation technologies, the ability of vehicles to perceive their surroundings in three dimensions has become one of the key technologies for ensuring the safety and reliability of autonomous driving. Existing autonomous vehicles typically rely on onboard sensors such as LiDAR, cameras, and millimeter-wave radar to complete tasks such as 3D target detection, road participant identification, and trajectory prediction. However, single-vehicle perception is limited by sensor installation location, detection distance, field of view, and occlusion in the traffic environment. In scenarios such as intersections, ramps, urban roads, and large vehicles, problems such as sparse point clouds of distant targets, invisible occluded vehicles, or weak bird's-eye view feature response can easily occur, thus affecting the accuracy and stability of target detection.

[0003] Vehicle-to-everything (V2X) cooperative perception technology enables vehicles to utilize supplementary observation information from other vehicles or roadside units through information sharing between vehicles and between vehicles and roadside infrastructure, thereby expanding the perception range and alleviating the problem of limited single-vehicle perspective. Existing V2X cooperative 3D target detection methods typically include early fusion, late fusion, and intermediate feature fusion. Among them, early fusion methods directly share raw point cloud or sensor data, which can retain relatively complete scene information, but the amount of communication data is large; late fusion methods share result-level information such as target detection boxes, with less communication overhead, but are prone to losing intermediate spatial features; intermediate feature fusion methods extract bird's-eye view features locally in each agent and then compress and share them, achieving a better balance between detection accuracy and communication overhead, and thus have become an important direction in V2X cooperative perception research in recent years. Existing research such as F-Cooper, V2VNet, OPV2V, DAIR-V2X, and V2X-ViT have promoted the development of this field from the perspectives of feature-level cooperative perception, vehicle-to-vehicle cooperative perception datasets, real-world V2X cooperative datasets, and Transformer-based multi-agent feature fusion, respectively. F-Cooper proposed a cooperative perception framework based on point cloud features, V2VNet studied the sharing of compressed depth features between vehicles to improve perception and prediction capabilities, OPV2V built an open dataset and fusion benchmark for vehicle-to-vehicle cooperative perception, DAIR-V2X provided real vehicle-road cooperative 3D target detection data, and V2X-ViT used a visual Transformer to model multi-agent interactions between vehicles and roadside infrastructure.

[0004] Building upon the intermediate feature fusion framework, some existing methods further address practical issues such as communication efficiency, pose error, and asynchronous perception. For example, Where2comm selects key perception regions for communication using spatial confidence maps to reduce communication bandwidth; How2comm focuses on communication redundancy, transmission latency, and cooperative heterogeneity, balancing perception performance with communication overhead; CoAlign addresses the pose error problem in cooperative perception by mitigating feature misalignment among multiple agents through a cooperative alignment mechanism. These methods have achieved certain results in multi-agent information aggregation, communication region selection, and pose error robustness.

[0005] However, in practical vehicle-road cooperative perception systems, the features transmitted by cooperating vehicles or roadside units are usually not completely synchronized with the vehicle's current moment, but rather generated from sensor observations at an earlier time. Due to factors such as communication transmission, data processing, feature compression, and network scheduling, the cooperative features received by the vehicle at the current moment often correspond to a delayed timestamp. If this delayed feature is directly transformed to the vehicle's coordinate system and fused, it will cause the position of dynamic targets in the bird's-eye view feature space to shift. Especially in high-speed traffic scenarios, different targets will produce different degrees of displacement under the same communication delay. For example, under a 200ms delay, a vehicle traveling at 80km / h will produce approximately 4.4m of displacement, a vehicle traveling at 30km / h will produce approximately 1.7m of displacement, while a stationary target will produce almost no displacement. This difference indicates that the feature misalignment caused by communication delay is not a simple global coordinate shift, but a non-uniform misalignment related to the target's own motion state.

[0006] To address the issues of communication latency and asynchronous perception, existing methods have proposed solutions such as latency-aware collaborative perception, feature flow prediction, temporal query modeling, and sparse alignment. For example, Latency-Aware Collaborative Perception adaptively aligns asynchronous perception features to the same timestamp; FFNet compensates for temporal asynchrony in vehicle-to-infrastructure (V2I) 3D target detection through feature flow prediction; and methods like StreamLTS and SparseAlign introduce temporal context or sparse feature modeling to improve the robustness of collaborative detection under asynchronous perception conditions. These solutions demonstrate that communication latency and temporal asynchrony have become significant technical challenges in V2I cooperative perception.

[0007] However, existing delay compensation methods still have the following shortcomings: First, some methods mainly rely on global spatial transformation, temporal coding, feature flow prediction, or learning-based temporal alignment, failing to fully utilize the physical relationship between target motion speed and communication delay, making it difficult to finely compensate for non-uniform displacements generated by targets at different speeds; Second, although some methods can model asynchronous features or historical information, there is a lack of closed-loop feedback between the compensation process for delayed collaborative features and the subsequent detection process, making it difficult to directly use the motion state estimated in the current frame for collaborative feature alignment in the next frame; Third, in occlusion or sparse point cloud scenarios, even with the introduction of vehicle-road cooperative information, the target's current frame bird's-eye view response may still be weak. Existing methods typically focus on multi-agent feature aggregation or communication compression, with insufficient joint utilization of historical detection results, motion priors, and adaptive enhancement of occluded regions, which can easily lead to missed detections or inaccurate localization of occluded vehicles.

[0008] Therefore, how to simultaneously handle the dynamic target feature misalignment caused by communication latency and the weak response of the current frame caused by occlusion in vehicle-road cooperative 3D target detection is a problem that existing technologies still need to solve. Summary of the Invention

[0009] In view of this, the purpose of the present invention is to provide a motion perception spatiotemporal compensation method and system for vehicle-road cooperative three-dimensional target detection, which solves the problems of existing vehicle-road cooperative perception methods under the conditions of communication delay, dynamic target motion, occlusion and sparse lidar observation, such as spatial misalignment between delayed cooperative features and the current frame features of the vehicle, weak bird's-eye view response of occluded targets and insufficient utilization of historical motion information.

[0010] Specifically, this invention aims to perform motion perception compensation on the delayed bird's-eye view features sent by cooperative vehicles or roadside units within an intermediate feature fusion framework, based on communication delay time, historical bird's-eye view velocity field, and current vehicle features. This enables the delayed cooperative features to be more accurately aligned to the current vehicle coordinate system. Simultaneously, by utilizing temporal memory, historical detection results, motion priors, and occlusion perception gating mechanisms, weak response regions in the current frame caused by occlusion or sparse point clouds are enhanced. This improves the detection accuracy, positioning stability, and robustness of vehicle-road cooperative 3D target detection in asynchronous communication and complex traffic environments.

[0011] The present invention further aims to construct a closed-loop motion-time compensation process, which caches the bird's-eye view velocity field estimated in the current frame and feeds it back to the delay feature compensation module of the next frame. This enables the cooperative perception system to continuously use historical motion states to correct the cooperative feature misalignment problem in subsequent frames, thereby reducing the missed detection, false detection and positioning deviation of dynamic targets caused by communication delay.

[0012] To achieve the above objectives, the present invention provides the following technical solution: On the one hand, a motion perception spatiotemporal compensation method for vehicle-road cooperative 3D target detection is provided, which includes the following steps: S1. Collect LiDAR point cloud data from the vehicle node and the collaborating node, and extract bird's-eye view features from each node. At the same time, the collaborating node shares the bird's-eye view features generated at delayed times with the vehicle node. S2. Calculate the prior motion displacement based on the historical velocity field and communication delay. Based on the local correlation between the current features of the vehicle and the delayed cooperation features, perform local feature prediction to obtain the residual offset. Finally, obtain the compensation offset based on the prior motion displacement and the learnable residual offset. S3. Based on the final compensation offset, perform differentiable sampling deformation on the delayed cooperation features to obtain the compensated cooperation features, and construct the set of compensated cooperation features for all cooperation nodes. S4. The vehicle node fuses the current features of the vehicle with the compensated collaborative feature set to obtain multi-source fused bird's-eye view features. S5. Construct an occlusion perception temporal enhancement mechanism, including a temporal memory branch, a motion prior branch, and an occlusion perception gating branch. The temporal memory branch updates the current hidden state with the current multi-source fused bird's-eye view features and the hidden state of the previous frame. The motion prior branch predicts the target center of the current frame based on the detected target center and target velocity of the previous frame. The occlusion perception gating branch generates gating weights based on the current fused features. The hidden state and the predicted target center are fused according to the weights to obtain the final enhanced bird's-eye view features. S6. Output the 3D target detection result of the current frame based on the final enhanced bird's-eye view features, and estimate the velocity field of the current frame bird's-eye view based on the current frame detection result, historical detection results and target correlation, as input for the next frame processing.

[0013] Furthermore, in step S1, the autonomous vehicle node and the collaborating nodes respectively collect LiDAR point cloud data from their respective perspectives, and generate bird's-eye view features through a local point cloud feature extraction network. The bird's-eye view features generated by the autonomous vehicle node at the current moment... As the main feature of the current frame, the bird's-eye view feature generated by the collaborating node at the delay time. As a collaborative feature to be shared, among which, Indicates the first Communication latency time corresponding to each collaborative node; The collaborating node encodes and compresses the generated bird's-eye view features and sends them to the vehicle node. The vehicle node then transforms the received collaborative bird's-eye view features into the vehicle's current coordinate system based on the spatial pose relationship between the collaborating node and the vehicle.

[0014] Furthermore, in step S2, the final compensation offset of the cooperative features includes two parts: the first part is the prior motion displacement calculated from the historical velocity field and communication delay, and the second part is the learnable residual offset predicted from feature correlation and local features. The prior motion displacement and the residual offset are added together to obtain the final compensation offset.

[0015] in, Indicates the first The prior displacement of each cooperating node. Indicates the first The residual offset of each cooperating node.

[0016] Furthermore, regarding the first The prior displacement of each cooperating node, based on its communication delay. and the velocity field of the previous frame calculate:

[0017] Among them, the velocity field of the previous frame This represents the target movement trend at each position in the feature grid of the bird's-eye view, calculated from the displacement of the target center in two adjacent frames; if the target's center positions in the previous and current frames are respectively... and The time interval between adjacent frames is The corresponding velocity field components are:

[0018]

[0019] in, These represent the width and height of the feature unit in the bird's-eye view, respectively.

[0020] Furthermore, regarding the first The residual offset of each cooperating node is first determined based on the current characteristics of the vehicle. and collaborating nodes Delay characteristics Calculate the correlation characteristics between the two within a local spatial range:

[0021] in, This represents a function for calculating local correlation. Then, based on local correlation, current vehicle features, delayed cooperation features, the velocity field of the previous frame, and communication delay time, the residual offset is predicted; the residual offset is expressed as:

[0022] in, The residual prediction network is used to predict the two-dimensional residual offset of delayed cooperation features on the bird's-eye view grid based on local correlation features, current bird's-eye view features of the vehicle, delayed cooperation bird's-eye view features, velocity field of the previous frame bird's-eye view, and communication delay time. The residual prediction network includes feature stitching units, convolutional feature encoding units, and offset regression units. Feature concatenation units are used to combine locally correlated features Current characteristics of the vehicle Delayed collaboration features Previous frame velocity field and communication delay time Extended delay feature map By concatenating the data along the channel dimension, we obtain the residual prediction input features:

[0023]

[0024] in, This means expanding the communication latency to a feature map with a spatial size consistent with the bird's-eye view feature map. This indicates splicing by channel dimension. and These represent the height and width of the feature map in the bird's-eye view, respectively. Convolutional feature encoding units are used to predict input features from residuals. Local spatial feature encoding is performed to extract the local mismatch relationship between the vehicle's current features and delayed cooperative features; the offset regression unit is used to output the two-dimensional residual offset field.

[0025] in, For offset regression operation, Its two channels represent the residual offsets in the x and y directions of the bird's-eye view, respectively, in the form of bird's-eye view feature units.

[0026] Furthermore, in step S3, the process of obtaining the compensated collaborative features is as follows: First, adjust the final compensation offset. Converted to offset in normalized sampling coordinates, for a width of Height is The bird's-eye view feature map is converted as follows:

[0027]

[0028] in, This refers to the sampling offset in the normalized coordinate system. Then the original sampling coordinates Updated to:

[0029]

[0030] Subsequently, based on the updated sampling coordinates, the delayed cooperative features are bilinearly differentiable to obtain the compensated cooperative features:

[0031] in, Represents a differentiable feature sampling deformation function. Indicates the target sampling location. This indicates the sampling offset corresponding to that position; Perform motion compensation on each of the collaborating nodes to obtain the compensated set of collaborative features. ,in, This represents the total number of all collaborating nodes.

[0032] Furthermore, in step S4, the fusion process of the multi-source fusion bird's-eye view features is as follows: The compensated set of collaborative features With the current characteristics of the vehicle By fusing the data, we obtain the multi-source fused bird's-eye view features:

[0033] in, This represents a multi-source feature fusion function.

[0034] Furthermore, in step S5, the processing procedure for the temporal memory branch is expressed as follows:

[0035] in, This represents the convolutional gated time-series update function. and These represent the temporal hidden states of the previous frame and the current frame, respectively; The process of handling the motion prior branch is expressed as follows:

[0036] in, Center of the target in the previous frame. The target speed of the previous frame. For the predicted target center in the current frame, The time interval between adjacent frames is defined; the predicted target center in the current frame and the target velocity in the previous frame are encoded as dynamic prior features:

[0037] in, This represents the motion prior encoding function; The processing procedure for the occlusion-sensing gating branch is represented as follows:

[0038] in, Represents the gated prediction function. Represents the normalized activation function. The generated gating weights are used to control the fusion ratio of temporal memory features and motion prior features; The final enhanced bird's-eye view features are represented as follows:

[0039] Here, ⊙ represents element-wise multiplication.

[0040] Furthermore, in step S6, based on the enhanced bird's-eye view features... Output the 3D object detection results for the current frame:

[0041] in, Indicates the first Confidence level of each target The parameters of the 3D detection bounding box representing the target are:

[0042] These represent the target's center position, length, width, height, and orientation angle, respectively. Simultaneously, based on the detection results of the current frame, historical detection results, and target correlation, the velocity field of the bird's-eye view in the current frame is estimated. This information is then used as input for the delay-cooperative feature compensation in the next frame.

[0043] On the other hand, a motion perception spatiotemporal compensation system for vehicle-road cooperative 3D target detection is also provided. This system executes the aforementioned motion perception spatiotemporal compensation method and includes: a vehicle-side data processing module, a cooperative-side data acquisition and sharing module, a motion perception compensation module, a multi-source feature fusion module, an occlusion perception temporal enhancement module, and a detection output and speed feedback module. The vehicle-side data processing module is used to collect the LiDAR point cloud data of the vehicle at the current moment, generate the current bird's-eye view features of the vehicle, and receive the delayed collaboration features sent by the collaborating end. The collaborative data acquisition and sharing module is used to acquire lidar point cloud data collected by collaborative vehicles and roadside infrastructure, and to complete point cloud preprocessing and bird's-eye view feature extraction at the collaborative end to generate corresponding intermediate features of collaborative nodes. The motion perception compensation module is used to dynamically compensate the delayed cooperation features based on the speed field of the historical bird's-eye view, the communication delay time, and the spatial correlation between the current vehicle features and the delayed cooperation features. The multi-source feature fusion module is used to fuse the current bird's-eye view features of the vehicle with the collaborative features after motion compensation to generate multi-agent fused bird's-eye view features. The occlusion perception temporal enhancement module is used to enhance the weak response regions in the current fused features by utilizing historical temporal information, target motion priors, and occlusion region feature states. The detection output and velocity feedback module is used to generate 3D target detection results based on the enhanced bird's-eye view features, and update the velocity field information required for motion compensation in the next frame based on the current detection results.

[0044] The beneficial effects of this invention are as follows: (1) Existing vehicle-road cooperative sensing methods often struggle to ensure complete synchronization between cooperative features and the current frame of the vehicle when receiving intermediate features from cooperative vehicles or roadside infrastructure. Directly fusing delayed cooperative features with the current features of the vehicle can cause positional shifts in dynamic targets due to communication delays, especially noticeable in high-speed scenarios. This invention calculates the prior displacement of the dynamic target during the delay period based on the communication delay time corresponding to the cooperative features and the velocity field of the previous frame's bird's-eye view, and then compensates for the delayed cooperative features by predicting residual offsets based on local feature correlations. Thus, the cooperative features are closer to the current vehicle coordinate system position before fusion, reducing missed detections, false detections, and positioning errors caused by delays in dynamic target detection. This invention decomposes the compensation offset into prior displacement and residual correction displacement. On one hand, the prior displacement provides initial compensation related to the target's motion state based on the velocity field and delay time; on the other hand, the residual correction displacement further corrects acceleration / deceleration, steering, positioning errors, and local feature matching errors. Therefore, this invention better adapts to the non-uniform displacements generated by targets at different speeds under asynchronous communication conditions, improving the stability of cooperative 3D target detection.

[0045] (2) In scenarios involving intersections, large vehicle occlusion, distant targets, and sparse point clouds, the target response in the current frame's bird's-eye view features may be weak. While existing collaborative perception methods can incorporate observation information from other agents, problems such as incomplete target features, weak center response, and unstable detection boxes may still occur if the current frame itself is occluded or the collaborative features are delayed. This invention introduces an occlusion perception temporal enhancement process after multi-source feature fusion, combining historical hidden states, historical target positions and velocities, current frame fusion features, and occlusion perception gating to adaptively enhance weak response regions. This approach can supplement target representation using historical temporal information and target motion trends when current frame observations are insufficient, thereby improving the detection reliability of occluded targets and distant sparse targets.

[0046] (3) The present invention caches the bird's-eye view velocity field estimated in the current frame and feeds it back to the delay cooperative feature compensation process in the next frame, so that the detection results and motion estimation of the current frame can participate in the feature alignment of subsequent frames. Thus, the system forms a closed loop process of "delay feature compensation - multi-source fusion - occlusion perception temporal enhancement - velocity field feedback - next frame compensation", which is beneficial to continuously reduce the dynamic target misalignment caused by communication delay.

[0047] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0048] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the overall process of the motion sensing spatiotemporal compensation method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of dynamic target displacement caused by communication delay in an embodiment of the present invention; Figure 3 This is a schematic diagram of the motion compensation process according to an embodiment of the present invention; Figure 4 This is a timing diagram of a weak response target in an occlusion scenario according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the occlusion perception temporal enhancement process according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the overall architecture of motion sensing spatiotemporal compensation according to an embodiment of the present invention. Detailed Implementation

[0049] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0050] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual images. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0051] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0052] Please see Figures 1-6 This is a method and system for motion perception spatiotemporal compensation of three-dimensional target detection in vehicle-road cooperative systems.

[0053] This invention proposes a motion perception spatiotemporal compensation method and system for vehicle-road cooperative 3D target detection. This method is applicable to vehicle-road cooperative perception scenarios composed of autonomous vehicles, collaborating vehicles, and roadside infrastructure. Each perception node collects LiDAR point cloud data and extracts intermediate features from the bird's-eye view locally. Collaborating vehicles and roadside infrastructure compress the intermediate features and send them to the autonomous vehicle. The autonomous vehicle performs motion compensation on the delayed collaborative features based on the timestamp, communication delay, pose relationship, and historical motion state of the collaborative features, then fuses them with the current features of the autonomous vehicle. Subsequently, historical temporal information and occlusion perception mechanisms are used to enhance the weak response areas of the current frame, finally outputting the 3D target detection result in the current autonomous vehicle coordinate system.

[0054] The technical solution of this invention includes: multi-source point cloud feature extraction, cooperative feature compression and sharing, cooperative feature coordinate unification, delayed cooperative feature motion compensation, multi-agent feature fusion, occlusion perception temporal enhancement, 3D target detection output, and velocity field feedback update. Its core idea is that instead of directly fusing delayed cooperative features with the vehicle's current features, it first estimates the dynamic target's displacement during the delay period based on historical velocity fields and communication delays. Then, it learns residual correction amounts through local feature matching, thereby compensating the delayed cooperative features to a position closer to the current moment. Simultaneously, it enhances weak response regions caused by occlusion or sparse point clouds based on historical detection results and temporal memory, and feeds back the velocity field estimated in the current frame to the compensation process in the next frame, forming a continuous motion compensation closed loop.

[0055] Example 1 This embodiment first provides a motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection, such as... Figure 1 As shown, the specific steps include: Step 1: Collect LiDAR point cloud data from the vehicle node and collaborating nodes, and extract bird's-eye view features from each node. At the same time, the collaborating nodes share the bird's-eye view features generated at delayed times with the vehicle node. The autonomous vehicle, collaborating vehicles, and roadside infrastructure each collect LiDAR point cloud data from their respective perspectives, and generate intermediate features for the bird's-eye view using a local 3D point cloud feature extraction network. The bird's-eye view features of the autonomous vehicle at the current moment are used as the main features of the current frame, and the bird's-eye view features generated by the collaborating vehicles and roadside infrastructure are used as collaborative features to be shared. Let the set of collaborative nodes consisting of the collaborating vehicles and roadside infrastructure be denoted as . ; The autonomous vehicle, collaborative vehicles, and roadside infrastructure each collect LiDAR point cloud data from their respective perspectives. For each sensing node, its point cloud data is denoted as:

[0056] Among them, subscript Indicates the self-vehicle node or collaborative nodes , ; For spatial coordinates, The point cloud reflection intensity, At the time of data acquisition, each sensing node transforms the raw point cloud into intermediate features of the bird's-eye view through a local 3D point cloud feature extraction network. The bird's-eye view features of the vehicle at the current time are denoted as... The bird's-eye view features generated by the collaborative nodes at the delay time are denoted as ,in, Indicates the first The communication latency time corresponding to each collaborative node.

[0057] In this embodiment, the point cloud feature extraction network can be a columnar voxel feature extraction network, a sparse voxel feature extraction network, or other 3D point cloud detection networks capable of outputting intermediate features of the bird's-eye view. The voxel size, downsampling factor, and bird's-eye view spatial resolution can be set according to the specific detection range and computing resources.

[0058] In one specific implementation, the point cloud voxel size is set to... The spatial downsampling factor of the bird's-eye view feature map is 4, therefore the actual spatial size corresponding to one bird's-eye view feature unit is... This spatial scale is used for subsequent conversion between the velocity field and the characteristic offset.

[0059] After the collaborating vehicles and roadside infrastructure extract bird's-eye view features locally, they encode and compress the intermediate features and send them to the vehicle's end via the vehicle-to-infrastructure communication link. The vehicle's end receives the intermediate features sent by the collaborating nodes and simultaneously obtains the timestamp corresponding to the feature, the pose of the collaborating node, the current pose of the vehicle, and the communication delay time.

[0060] Because communication transmission, feature extraction, data encoding, and network scheduling all require time, the cooperative features received by the vehicle at the current moment are usually not features generated at the current moment, but rather originate from an earlier moment. Therefore, directly fusing these cooperative features can lead to misalignment of dynamic targets within the bird's-eye view plane. Figure 2 As shown, for example, under a 200ms delay, a vehicle traveling at 80km / h experiences a displacement of approximately 4.4m, a vehicle traveling at 30km / h experiences a displacement of approximately 1.7m, while a stationary target experiences almost no displacement. This phenomenon indicates that the target misalignment caused by communication delay has a significant correlation with the motion state, rather than a simple fixed offset.

[0061] The vehicle-mounted system transforms the received collaborative bird's-eye view features into its current coordinate system based on the spatial pose relationship between the collaborating nodes and the vehicle. This step is used to eliminate coordinate system differences between different sensing nodes.

[0062] It should be noted that this step mainly handles the static coordinate transformation relationships between different nodes, including translation, rotation, and scale correspondence. For the positional changes of dynamic targets during communication delays, this invention does not employ simple global translation compensation in this step, but instead performs motion perception compensation based on the velocity field and communication delay in subsequent steps. This feature is already in the current bird's-eye view coordinate system of the vehicle, but the position of the dynamic target within it may still correspond to an earlier time.

[0063] Step 2: Calculate the prior motion displacement based on the historical velocity field and communication delay. Based on the local correlation between the current features of the vehicle and the delayed cooperation features, perform local feature prediction to obtain the residual offset. Finally, obtain the compensation offset based on the prior motion displacement and the learnable residual offset. like Figure 3 As shown, the vehicle-side reads the bird's-eye view velocity field from the previous frame's buffer and, combined with the communication delay time corresponding to the current cooperating node, calculates the prior displacement of the dynamic target during the delay period. This step utilizes the physical relationship of "velocity multiplied by time" to make a preliminary prediction of the dynamic target position in the delayed cooperation features. Therefore, the system can generate different compensation amounts for high-speed targets, low-speed targets, and stationary targets, instead of using a uniform global offset.

[0064] The vehicle reads the bird's-eye view velocity field from the previous frame's cache. This velocity field represents the target's motion trend at various locations within the feature grid of the bird's-eye view. In one specific implementation, the velocity field is calculated from the displacement of the target's center in two adjacent frames. If the target's center positions in the previous frame and the current frame are respectively... and The time interval between adjacent frames is The corresponding velocity field components are:

[0065]

[0066] in, The unit is the bird's-eye view feature unit per second, not meters per second; These are the width and height of the feature unit in the bird's-eye view, respectively; For the Each cooperating node, based on its communication latency and the velocity field of the previous frame Calculate its prior displacement:

[0067] in, This represents the prior displacement of motion in units of feature cells in the bird's-eye view. This displacement is used to roughly estimate the positional changes of a dynamic target during the communication delay.

[0068] The purpose of this step is to introduce the physical relationship of "velocity multiplied by time" into the collaborative feature compensation process, so that the compensation direction does not completely depend on network learning, but has clear motion prior constraints.

[0069] Because vehicle motion may involve acceleration / deceleration, steering, occlusion, positioning errors, and speed estimation errors, relying solely on prior displacement is insufficient to completely eliminate feature misalignment. Therefore, this invention further calculates the local correlation between the vehicle's current bird's-eye view features and delayed collaborative features.

[0070] For the current characteristics of the vehicle and collaborating nodes Delay characteristics Calculate the correlation characteristics between the two within a local spatial range:

[0071] in, This represents the function for calculating local correlation. This represents a local matching clue between the vehicle's current features and the cooperative delay features. This local correlation is used to determine the spatial correspondence between the target or local structure in the delayed cooperative features and the vehicle's current features, providing a basis for subsequent residual offset prediction.

[0072] The vehicle-side predicts the residual offset based on local correlation, current vehicle characteristics, delayed cooperation characteristics, the velocity field of the previous frame, and communication delay time. The residual offset is expressed as:

[0073] in, The residual prediction network is used to predict the two-dimensional residual offset of the delayed cooperation feature on the bird's-eye view grid based on local correlation features, the current bird's-eye view features of the vehicle, the delayed cooperation bird's-eye view features, the velocity field of the previous frame bird's-eye view, and the communication delay time.

[0074] The residual prediction network includes a feature concatenation unit, a convolutional feature encoding unit, and a shift regression unit. The feature concatenation unit is used to combine locally correlated features. Current characteristics of the vehicle Delayed collaboration features Previous frame velocity field and communication delay time Extended delay feature map By concatenating the data along the channel dimension, we obtain the residual prediction input features:

[0075]

[0076] in, This means expanding the communication latency to a feature map with a spatial size consistent with the bird's-eye view feature map. This indicates splicing by channel dimension. and These represent the height and width of the feature map in the bird's-eye view, respectively.

[0077] Convolutional feature encoding units are used to predict input features from residuals. Local spatial feature encoding is performed to extract the local mismatch relationship between the vehicle's current features and delayed cooperative features; the offset regression unit is used to output the two-dimensional residual offset field.

[0078] in, Its two channels represent the residual offsets in the x and y directions of the bird's-eye view, respectively, with units of bird's-eye view feature units. The convolutional feature encoding unit can consist of one or more layers of two-dimensional convolution, normalization layers, and nonlinear activation layers, and the offset regression unit can be implemented by two-dimensional convolution or equivalent linear mapping.

[0079] Subsequently, the prior motion displacement and the residual offset are added to obtain the final compensated offset. The prior motion displacement provides the main motion direction, while the residual offset corrects for nonlinear motion, positioning errors, velocity estimation errors, and local feature matching errors. Through this method, the compensation process for delayed cooperative features simultaneously possesses physical motion constraints and adaptive correction capabilities. This invention decomposes the final compensated offset of cooperative features into two parts: the first part is the prior motion displacement calculated from the historical velocity field and communication delay, and the second part is the learnable residual offset obtained from feature correlation and local feature prediction.

[0080] Specifically, the final compensation offset is expressed as .

[0081] This design is one of the key technical means of this invention. Compared with directly predicting the complete offset, this invention first uses the historical velocity field to give the physical motion direction, and then corrects nonlinear motion, velocity error, coordinate transformation error and local feature ambiguity through residual offset, thereby improving the stability and accuracy of delayed cooperative feature alignment.

[0082] Step 3: Perform differentiable sampling deformation on the delayed cooperative features based on the final compensated offset to obtain the compensated cooperative features; The vehicle-side system converts the final compensated offset into an offset in the bird's-eye view feature sampling coordinates, and performs differentiable sampling deformation on the delayed collaborative features based on the updated sampling coordinates to obtain the compensated collaborative features. After this step, the delayed features of the collaborative nodes are compensated to a position closer to the current vehicle coordinate system, thereby mitigating the misalignment of dynamic target features caused by communication delays.

[0083] Specifically, the final compensated offset is obtained. Then, it is converted into an offset in normalized sampling coordinates. For a width of Height is The bird's-eye view feature map is converted as follows:

[0084]

[0085] in, This is the sampling offset in the normalized coordinate system.

[0086] Original sampling coordinates Updated to:

[0087]

[0088] Subsequently, based on the updated sampling coordinates, the delayed cooperative features are bilinearly differentiable to obtain the compensated cooperative features:

[0089] in, Represents a differentiable feature sampling deformation function. Indicates the target sampling location. This indicates the sampling offset corresponding to that position.

[0090] For all cooperative vehicles and roadside infrastructure, perform the motion compensation process described above to obtain the compensated cooperative feature set. ,in, This represents the total number of all collaborating nodes, i.e., the total number of all collaborating vehicles and roadside infrastructure. After this step, the latency characteristics of the collaborating nodes are compensated to a vehicle coordinate system closer to the current moment, thereby mitigating dynamic target misalignment caused by communication latency.

[0091] Step 4: Fuse the features of the autonomous vehicle with the compensated collaborative features to obtain multi-source fused bird's-eye view features; In this embodiment, the fusion process can employ weighted fusion, convolutional fusion, attention fusion, confidence fusion, or region selection fusion. Through this step, the autonomous vehicle can utilize complementary observation information provided by cooperating vehicles and roadside infrastructure to expand its perception range and enhance the representation of target features.

[0092] For the compensated collaborative feature set obtained by the vehicle node This set is compared with the current characteristics of the vehicle. By fusing the data, we obtain the multi-source fused bird's-eye view features:

[0093] in, This represents a multi-source feature fusion function.

[0094] Step 5: Enhance weak response regions using temporal memory and occlusion perception mechanisms. like Figure 4 As shown, although the compensated collaborative features can alleviate spatial misalignment caused by communication delays, the target response may still be weak in the current frame's bird's-eye view features under conditions such as intersections, large vehicle occlusion, distant targets, or sparse point clouds. Therefore, this invention further introduces an occlusion perception temporal enhancement process after multi-source fusion.

[0095] After multi-source feature fusion, the vehicle-side further performs occlusion perception temporal enhancement. For intersections, large vehicle occlusion, distant targets, or sparse point cloud scenarios, the target response may still be weak in the fused features of the current frame. This embodiment utilizes historical temporal information saved in the hidden state of the previous frame to predict the current target motion trend based on historical target position and speed, and adjusts the enhancement ratio of the current fused features by controlling the historical temporal information and motion prior information through occlusion perception gating. Thus, the system can supplement the target representation of weak response areas using historical information and motion trends when the observation of the current frame is incomplete.

[0096] like Figure 5 As shown, the process consists of three branches.

[0097] First, the temporal memory branch. This branch updates the hidden state at the current time step based on the current fused features and the hidden state of the previous frame:

[0098] in, This represents the convolutional gated time-series update function. and These represent the temporal hidden states of the previous and current frames, respectively. This branch is used to save historical scene information, allowing the system to utilize historical context even when the observations of the current frame are incomplete.

[0099] Second, the motion prior branch. This branch predicts the target center in the current frame based on the target center and target velocity detected in the previous frame:

[0100] in, Center of the target in the previous frame. The target speed of the previous frame. The time interval between adjacent frames. The prediction center and velocity information are further encoded as dynamic prior features:

[0101] in, This represents the motion prior encoding function.

[0102] Third, the occlusion-aware gating branch. This branch generates gating weights based on the current fused features:

[0103] in, Represents the gated prediction function. Represents the normalized activation function. Used to control the fusion ratio of temporal memory features and motion prior features.

[0104] The final enhanced features are:

[0105] Here, ⊙ represents element-wise multiplication.

[0106] This step enables the system to adaptively decide to rely more on current fused observations, historical temporal memory, or motion prior predictions based on the current frame feature state, thereby enhancing the target representation in occluded and weak response regions.

[0107] Step 6: Output the 3D target detection results and feed back the current velocity field. The detection head outputs the 3D target detection results for the current frame based on the enhanced bird's-eye view features. The detection results include target category confidence, 3D center position, target size, and target orientation angle. Simultaneously, the vehicle-mounted system estimates the velocity field of the current frame's bird's-eye view based on the current frame's detection results, historical detection results, and target correlations, and caches this velocity field. During the processing of the next frame, this velocity field serves as input for delayed cooperative feature motion compensation, participating in the calculation of prior displacement for the next frame's motion.

[0108] The detection head uses enhanced bird's-eye view features Output the 3D object detection results for the current frame:

[0109] in, Indicates the first Confidence level of each target The parameters of the 3D detection bounding box representing the target are:

[0110] These represent the target's center position, length, width, height, and orientation angle, respectively.

[0111] Meanwhile, this invention estimates the velocity field of the current frame's bird's-eye view based on the detection results of the current frame, historical detection results, and target correlation. The velocity field is buffered and used as input for delay-cooperative feature compensation in the next frame. This forms the following closed-loop process: →Delayed feature motion compensation→Multi-source feature fusion→Temporal enhancement of occlusion perception→ →Motion compensation for delayed features in the next frame. Through this closed loop, the target motion state estimated in the current frame can continue to affect the cooperative feature alignment in subsequent frames, thereby reducing missed detections, false detections, and positioning errors caused by communication delays.

[0112] Example 2 This embodiment provides a motion perception spatiotemporal compensation system for vehicle-road cooperative three-dimensional target detection, such as... Figure 6 As shown, it includes at least: a vehicle-side data processing module, a collaborative data acquisition and sharing module, a motion perception compensation module, a multi-source feature fusion module, an occlusion perception temporal enhancement module, and a detection output and speed feedback module. These modules are connected via data transmission interfaces to achieve spatiotemporal compensation of delayed collaborative features, multi-source perception feature fusion, and output of 3D target detection results in a vehicle-road cooperative environment.

[0113] The vehicle-side data processing module is used to collect the LiDAR point cloud data of the vehicle at the current moment and generate the current bird's-eye view features of the vehicle, while receiving delayed collaborative features sent by the collaborating nodes. Specifically, the vehicle-side data processing module includes: a vehicle point cloud acquisition unit, a feature extraction unit, and a coordinate transformation unit. The vehicle point cloud acquisition unit acquires the 3D point cloud data corresponding to the vehicle's current position; the feature extraction unit generates the current bird's-eye view features of the vehicle through a 3D point cloud feature extraction network, which serves as the main features in the subsequent collaborative fusion process; the coordinate transformation unit transforms the received bird's-eye view features of the collaborating nodes to a unified vehicle coordinate system based on the spatial pose relationship between the vehicle and the collaborating nodes. It should be noted that the coordinate transformation unit is only used to eliminate static coordinate differences between different sensing nodes; dynamic target position offsets caused by communication delays are further processed by the motion perception compensation module.

[0114] The collaborative data acquisition and sharing module is used to acquire LiDAR point cloud data collected by collaborative vehicles and roadside infrastructure, and to perform point cloud preprocessing and bird's-eye view feature extraction at the collaborative end to generate corresponding intermediate features of collaborative nodes. Specifically, the collaborative data acquisition and sharing module includes: a point cloud acquisition unit, a point cloud feature extraction unit, a feature compression and encoding unit, and a collaborative communication transmission unit. The point cloud acquisition unit is used to acquire 3D point cloud data obtained by collaborative vehicles or roadside infrastructure within the current sensing area; the point cloud feature extraction unit is used to convert the raw point cloud data into intermediate features of the bird's-eye view using a 3D point cloud feature extraction network; the feature compression and encoding unit is used to compress and encode the generated intermediate features of the bird's-eye view to reduce the amount of data transmitted during vehicle-to-infrastructure communication; and the collaborative communication transmission unit is used to send the compressed collaborative features to the vehicle end, and simultaneously send the corresponding timestamp information, collaborative node pose information, and communication delay information. Through these modules, the vehicle end can acquire asynchronous collaborative sensing information from different sensing nodes, providing input data for subsequent delay compensation.

[0115] The motion perception compensation module is used to dynamically compensate for the delayed collaboration features based on the historical bird's-eye view velocity field, communication delay time, and the spatial correlation between the current vehicle features and the delayed collaboration features. Specifically, the motion perception compensation module includes: a historical velocity field reading unit, a motion prior displacement calculation unit, a local correlation analysis unit, a residual offset prediction unit, and a feature sampling deformation unit; wherein: the historical velocity field reading unit is used to read the previous moment's bird's-eye view velocity field from historical cache data; the motion prior displacement calculation unit is used to calculate the motion displacement generated by the dynamic target during the delay period based on the communication delay time and the historical velocity field; the local correlation analysis unit is used to calculate the local spatial matching relationship between the current bird's-eye view features of the vehicle and the delayed collaboration features; the residual offset prediction unit is used to predict the compensation residual caused by nonlinear motion, positioning error, and speed estimation error based on local correlation, the current features of the vehicle, and the delayed collaboration features; the feature sampling deformation unit is used to fuse the motion prior displacement and the residual offset to obtain the final compensation offset, and perform differentiable sampling deformation on the delayed collaboration features based on the compensation offset to obtain the compensated collaboration features. This module enables delayed collaboration features to be adjusted from their historical positions to their current positions, thereby improving the spatial alignment accuracy of dynamic targets.

[0116] The multi-source feature fusion module is used to fuse the current bird's-eye view features of the vehicle with the motion-compensated collaborative features to generate multi-agent fused bird's-eye view features. Specifically, the multi-source feature fusion module includes: a feature input unit, a fusion weight calculation unit, and a feature aggregation unit; wherein, the feature input unit is used to receive the current features of the vehicle and the compensated collaborative features of each collaborative node; the fusion weight calculation unit is used to generate corresponding fusion weights based on the feature quality, spatial position relationship, or confidence information of different nodes; the feature aggregation unit is used to aggregate the multi-source features using weighted fusion, convolutional fusion, attention fusion, or region selection fusion methods to obtain the fused bird's-eye view features.

[0117] The occlusion-aware temporal enhancement module utilizes historical temporal information, target motion priors, and the feature state of occluded regions to enhance weak response regions in the current fused features. Specifically, the occlusion-aware temporal enhancement module includes: a temporal memory update unit, a motion prior prediction unit, an occlusion-aware gating unit, and a feature enhancement unit. The temporal memory update unit updates the current temporal hidden state based on the current fused features and the hidden state of the previous frame to preserve historical scene information. The motion prior prediction unit predicts the target motion trend at the current moment based on the historical target center position and target velocity, and generates motion prior features. The occlusion-aware gating unit calculates gating weights based on the current fused features, adaptively adjusting the fusion ratio between current observation information, historical temporal information, and motion prior information. The feature enhancement unit fuses the temporal memory features and motion prior features according to the gating weights to generate enhanced bird's-eye view detection features. Through these modules, the target feature response intensity can be improved in scenarios involving vehicle occlusion, sparse point clouds, and long-distance detection.

[0118] The detection output and velocity feedback module is used to generate 3D target detection results based on the enhanced bird's-eye view features, and to update the velocity field information required for motion compensation in the next frame based on the current detection results. Specifically, the detection output and velocity feedback module includes: a 3D detection output unit, a target association unit, a velocity field estimation unit, and a historical state cache unit; wherein, the 3D detection output unit is used to output the target category, 3D center position, size parameters, and orientation angle information; the target association unit is used to establish the target correspondence based on the current detection results and historical detection results; the velocity field estimation unit is used to calculate the velocity field of the current frame's bird's-eye view based on the target's continuous motion trajectory; the historical state cache unit is used to store the current velocity field and use it as the motion prior input in the motion perception compensation process of the next frame.

[0119] The motion-aware spatiotemporal compensation system for 3D target detection in vehicle-road cooperative systems provided in this embodiment dynamically compensates for asynchronous cooperative features by combining communication latency with target motion state through a motion-aware compensation module; enhances weak response regions by utilizing historical detection information and motion priors through an occlusion-aware temporal enhancement module; and feeds back the target motion state estimated in the current frame to the compensation process in the next frame through a detection output and velocity feedback module. Therefore, this system can improve the spatial alignment accuracy, detection accuracy, and system robustness of 3D target detection in complex vehicle-road cooperative environments such as communication latency, dynamic target motion, occlusion, and sparse point clouds.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A motion-sensory compensation method for 3D target detection in vehicle-road cooperative systems, characterized in that: The method includes the following steps: S1. Collect LiDAR point cloud data from the vehicle node and the collaborating node, and extract bird's-eye view features from each node. At the same time, the collaborating node shares the bird's-eye view features generated at delayed times with the vehicle node. S2. Calculate the prior motion displacement based on the historical velocity field and communication delay. Based on the local correlation between the current features of the vehicle and the delayed cooperation features, perform local feature prediction to obtain the residual offset. Finally, obtain the compensation offset based on the prior motion displacement and the learnable residual offset. S3. Based on the final compensation offset, perform differentiable sampling deformation on the delayed cooperation features to obtain the compensated cooperation features, and construct the set of compensated cooperation features for all cooperation nodes. S4. The vehicle node fuses the current features of the vehicle with the compensated collaborative feature set to obtain multi-source fused bird's-eye view features. S5. Construct an occlusion perception temporal enhancement mechanism, including a temporal memory branch, a motion prior branch, and an occlusion perception gating branch. The temporal memory branch updates the current hidden state with the current multi-source fused bird's-eye view features and the hidden state of the previous frame. The motion prior branch predicts the target center of the current frame based on the detected target center and target velocity of the previous frame. The occlusion perception gating branch generates gating weights based on the current fused features. The hidden state and the predicted target center are fused according to the weights to obtain the final enhanced bird's-eye view features. S6. Output the 3D target detection result of the current frame based on the final enhanced bird's-eye view features, and estimate the velocity field of the current frame bird's-eye view based on the current frame detection result, historical detection results and target correlation, as input for the next frame processing.

2. The motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection according to claim 1, characterized in that: In step S1, the autonomous vehicle node and the collaborating node respectively collect LiDAR point cloud data from their respective perspectives, and generate bird's-eye view features through a local point cloud feature extraction network. The bird's-eye view features generated by the autonomous vehicle node at the current moment are... As the main feature of the current frame, the bird's-eye view feature generated by the collaborating node at the delay time. As a collaborative feature to be shared, among which, Indicates the first Communication latency time corresponding to each collaborative node; The collaborating node encodes and compresses the generated bird's-eye view features and sends them to the vehicle node. The vehicle node then transforms the received collaborative bird's-eye view features into the vehicle's current coordinate system based on the spatial pose relationship between the collaborating node and the vehicle.

3. The motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection according to claim 2, characterized in that: In step S2, the final compensation offset of the cooperative features consists of two parts: the first part is the prior motion displacement calculated from the historical velocity field and communication delay, and the second part is the learnable residual offset predicted from feature correlation and local features. The prior motion displacement and the residual offset are added together to obtain the final compensation offset. in, Indicates the first The prior displacement of each cooperating node. Indicates the first The residual offset of each cooperating node.

4. The motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection according to claim 3, characterized in that: For the The prior displacement of each cooperating node, based on its communication delay. and the velocity field of the previous frame calculate: Among them, the velocity field of the previous frame This represents the target movement trend at each position in the feature grid of the bird's-eye view, calculated from the displacement of the target center in two adjacent frames; if the target's center positions in the previous and current frames are respectively... and The time interval between adjacent frames is The corresponding velocity field components are: in, These represent the width and height of the feature unit in the bird's-eye view, respectively.

5. The motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection according to claim 3, characterized in that: For the The residual offset of each cooperating node is first determined based on the current characteristics of the vehicle. and collaborating nodes Delay characteristics Calculate the correlation characteristics between the two within a local spatial range: in, This represents a function for calculating local correlation. Then, based on local correlation, current vehicle features, delayed cooperation features, the velocity field of the previous frame, and communication delay time, the residual offset is predicted; the residual offset is expressed as: in, The residual prediction network is used to predict the two-dimensional residual offset of delayed cooperation features on the bird's-eye view grid based on local correlation features, current bird's-eye view features of the vehicle, delayed cooperation bird's-eye view features, velocity field of the previous frame bird's-eye view, and communication delay time. The residual prediction network includes feature stitching units, convolutional feature encoding units, and offset regression units. Feature concatenation units are used to combine locally correlated features Current characteristics of the vehicle Delayed collaboration features Previous frame velocity field and communication delay time Extended delay feature map By concatenating the data along the channel dimension, we obtain the residual prediction input features: in, This means expanding the communication latency to a feature map with a spatial size consistent with the bird's-eye view feature map. This indicates splicing by channel dimension. and These represent the height and width of the feature map in the bird's-eye view, respectively. Convolutional feature encoding units are used to predict input features from residuals. Local spatial feature encoding is performed to extract the local mismatch relationship between the vehicle's current features and delayed cooperative features; the offset regression unit is used to output the two-dimensional residual offset field. in, For offset regression operation, Its two channels represent the residual offsets in the x and y directions of the bird's-eye view, respectively, in the form of bird's-eye view feature units.

6. The motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection according to claim 3, characterized in that: In step S3, the process of obtaining the compensated collaborative features is as follows: First, adjust the final compensation offset. Converted to offset in normalized sampling coordinates, for a width of Height is The bird's-eye view feature map is converted as follows: in, This refers to the sampling offset in the normalized coordinate system. Then the original sampling coordinates Updated to: Subsequently, based on the updated sampling coordinates, the delayed cooperative features are bilinearly differentiable to obtain the compensated cooperative features: in, Represents a differentiable feature sampling deformation function. Indicates the target sampling location. This indicates the sampling offset corresponding to that position; Perform motion compensation on each of the collaborating nodes to obtain the compensated set of collaborative features. ,in, This represents the total number of all collaborating nodes.

7. The motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection according to claim 6, characterized in that: In step S4, the fusion process of the multi-source fusion bird's-eye view features is as follows: The compensated set of collaborative features With the current characteristics of the vehicle By fusing the data, we obtain the multi-source fused bird's-eye view features: in, This represents a multi-source feature fusion function.

8. The motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection according to claim 7, characterized in that: In step S5, the processing procedure for the temporal memory branch is as follows: in, This represents the convolutional gated time-series update function. and These represent the temporal hidden states of the previous frame and the current frame, respectively; The process of handling the motion prior branch is expressed as follows: in, Center of the target in the previous frame. The target speed of the previous frame. For the predicted target center in the current frame, The time interval between adjacent frames is defined; the predicted target center in the current frame and the target velocity in the previous frame are encoded as dynamic prior features: in, This represents the motion prior encoding function; The processing procedure for the occlusion-sensing gating branch is represented as follows: in, Represents the gated prediction function. Represents the normalized activation function. The generated gating weights are used to control the fusion ratio of temporal memory features and motion prior features; The final enhanced bird's-eye view features are represented as follows: Here, ⊙ represents element-wise multiplication.

9. A motion perception spatiotemporal compensation method for vehicle-road cooperative three-dimensional target detection according to claim 8, characterized in that: In step S6, based on the enhanced bird's-eye view features Output the 3D object detection results for the current frame: in, Indicates the first Confidence level of each target The parameters of the 3D detection bounding box representing the target are: These represent the target's center position, length, width, height, and orientation angle, respectively. Simultaneously, based on the detection results of the current frame, historical detection results, and target correlation, the velocity field of the bird's-eye view in the current frame is estimated. This information is then used as input for the delay-cooperative feature compensation in the next frame.

10. A motion-sensing spatiotemporal compensation system for vehicle-road cooperative three-dimensional target detection, which executes the motion-sensing spatiotemporal compensation method according to any one of claims 1-9, characterized in that: The system includes: a vehicle-side data processing module, a collaborative-side data acquisition and sharing module, a motion perception compensation module, a multi-source feature fusion module, an occlusion perception temporal enhancement module, and a detection output and speed feedback module; wherein... The vehicle-side data processing module is used to collect the LiDAR point cloud data of the vehicle at the current moment, generate the current bird's-eye view features of the vehicle, and receive the delayed collaboration features sent by the collaborating end. The collaborative data acquisition and sharing module is used to acquire lidar point cloud data collected by collaborative vehicles and roadside infrastructure, and to complete point cloud preprocessing and bird's-eye view feature extraction at the collaborative end to generate corresponding intermediate features of collaborative nodes. The motion perception compensation module is used to dynamically compensate the delayed cooperation features based on the speed field of the historical bird's-eye view, the communication delay time, and the spatial correlation between the current vehicle features and the delayed cooperation features. The multi-source feature fusion module is used to fuse the current bird's-eye view features of the vehicle with the collaborative features after motion compensation to generate multi-agent fused bird's-eye view features. The occlusion perception temporal enhancement module is used to enhance the weak response regions in the current fused features by utilizing historical temporal information, target motion priors, and occlusion region feature states. The detection output and velocity feedback module is used to generate 3D target detection results based on the enhanced bird's-eye view features, and update the velocity field information required for motion compensation in the next frame based on the current detection results.

Images