Vehicle-road collaborative cross-view target matching method based on topological structure scene expression

Abstract

The invention relates to the related technical field of vehicle-road collaboration, in particular to a vehicle-road collaboration cross-view target matching method based on topological structure scene expression, which comprises the following steps: firstly, correcting an original 3D bounding box of a vehicle end and a road side, and constructing a full-connection topological structure chart based on a relative position relation of a corrected target so as to lightly express a scene; secondly, aiming at each target node in the graph, extracting a view angle invariant geometric feature, a point cloud geometric feature and an image texture feature of the target node, then utilizing a graph attention network to aggregate a topological relation, fusing the multi-modal features through a gating Transform encoder, and generating a cross-modal scene descriptor robust to shielding and view angle change; and finally, constructing a similarity matrix of vehicle and road scene descriptors, solving an optimal transmission strategy by adopting a Sinkhorn algorithm with a garbage can mechanism, and outputting a matching result, thereby effectively reducing the dependence on high-precision positioning and the communication overhead, and improving the matching robustness and precision under the conditions of shielding, view angle change, similar appearance and the like.

Description

Technical Field

[0001] This invention relates to the field of vehicle-road cooperative technology, and in particular to a cross-view target matching method for vehicle-road cooperative based on topological scene expression. Background Technology

[0002] With the development of vehicle-to-everything (V2X) technology, the "single-vehicle intelligence" of a single vehicle is no longer sufficient to cope with the perception blind spots caused by complex urban scenarios (such as intersection occlusion and GNSS failure in tunnels). V2X cooperative perception, by fusing data from vehicle-mounted and roadside sensors, has become key to overcoming perception bottlenecks. However, the core prerequisite for achieving cooperative perception is cross-view target matching, that is, accurately associating the same physical target detected by both the vehicle and the roadside. Existing matching methods have many limitations. First, many methods heavily rely on high-precision positioning as a priori; if GNSS signals are poor or there are errors in the vehicle-road coordinate system, these geometrically constrained methods will completely fail. Second, even with feature-based matching methods, the inherently large viewpoint differences in V2X—the high viewpoint of the roadside and the low viewpoint of the vehicle—lead to drastic changes in the appearance and scale of targets and are highly susceptible to occlusion, significantly reducing the robustness of deep learning features. In dense traffic scenarios, the presence of numerous similar-looking targets (such as vehicles of the same model) further exacerbates the difficulty of differentiation and matching. More importantly, in order to ensure accuracy, traditional methods often require the transmission of the full amount of point cloud data or high-dimensional image features, which brings huge communication overhead and makes it difficult to meet the strict requirements of low latency in V2X scenarios. Summary of the Invention

[0003] The purpose of this invention is to provide a vehicle-road cooperative cross-view target matching method based on topological scene expression, so as to solve the problems of high dependence on high-precision positioning, poor robustness to occlusion, large communication overhead, and difficulty in distinguishing similar targets in the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a vehicle-road cooperative cross-view target matching method based on topological scene representation, comprising the following steps:

[0005] Step S1, Topology Scene Construction and Boundary Box Correction: On the vehicle and roadside, the original 3D bounding boxes of the targets to be detected are corrected, and a fully connected topology map is constructed based on the relative positional relationship between the corrected targets to express the scene in a lightweight way and reduce the dependence on high-precision positioning.

[0006] Step S2, Multi-level Scene Encoding: For each target node in the topology graph, DeepSDF is used to infer its view-invariant geometric features, and Point Transformer is used to extract point cloud geometric features and Vision Transformer is used to extract image texture features to generate multimodal basic features of the target.

[0007] Step S3, Topology Aggregation and Feature Fusion: A graph attention network (GATv2) is used to aggregate the topological relationship features of neighboring nodes, and a gated Transformer encoder is used to fuse the view-invariant features, geometric features, texture features and topological features to generate a cross-modal scene descriptor that is robust to occlusion and viewpoint changes.

[0008] Step S4, Optimal Transmission Matching: Construct a similarity matrix between the vehicle-side and roadside scene descriptors, and to address the issues of asymmetric target numbers and occlusion, use the Sinkhorn algorithm with a "trash can" mechanism to solve for the optimal transmission strategy, outputting robust cross-view target matching results.

[0009] Preferably, the topology scene construction and bounding box correction in step S1 include:

[0010] S101, Bounding Box Correction: The PointNet network is used to process the point cloud within the input 3D bounding box and regress to predict a 6-DOF (6-DoF) correction vector to optimize the position and pose errors of the original bounding box caused by detection or localization bias.

[0011] S102. Construction of Topology Graph: Each target after correction in S101 is used as a graph node, and the relative positional relationship between any two target nodes is used as the edge of the graph to construct a fully connected topology graph. This graph only contains the relative spatial distribution between targets and does not depend on global absolute coordinates.

[0012] Preferably, the multi-level scene encoding in step S2 includes:

[0013] S201, Viewpoint Invariant Feature Encoding: The DeepSDF (Deep Signed Distance Function) module is used to extract compact 3D shape feature representations by learning continuous SDF, enabling it to infer viewpoint-independent global geometric features from incomplete or noisy point clouds.

[0014] S202, Geometric Feature Encoding: The Point Transformer network is used to capture the global and local structure of the target point cloud through its self-attention mechanism, and to extract the explicit geometric features of the target.

[0015] S203. Texture Feature Encoding: The Vision Transformer (ViT) network is used to extract global context and local texture details from the 2D cropped image corresponding to the target as the texture features of the target.

[0016] Preferably, the topology aggregation and feature fusion in step S3 include:

[0017] S301. Topological Feature Aggregation: Using the GATv2 network, information is propagated across the fully connected topological graph, and the relative position vectors between targets are explicitly incorporated as dynamic topological constraints. Global contextual information is aggregated, and its attention score is defined as:

[0018]

[0019] in, , For node features, It is a relative position vector. For learnable topological embeddings;

[0020] S302, Cross-modal feature fusion: Design a Transformer-based fusion module that takes the view-invariant features, geometric features, texture features and topological features obtained in S2 and S301 as independent token sequences as input;

[0021] S303, Gating Mechanism and Scene Descriptor: Modality priority weight and cross-modality gating mechanism are introduced in the fusion module to dynamically balance the contribution of each modality, and complementary information between modalities is captured through the Transformer coding layer, and finally aggregated to generate L2 normalized scene descriptors.

[0022] Preferably, the optimal transmission matching in step S4 includes:

[0023] S401. Similarity Matrix Construction: Calculate the cosine similarity between N target descriptors in the vehicle-side scene and M target descriptors in the roadside scene, and construct a similarity matrix. Similarity matrix ;

[0024] S402, Trash Can Expansion: For handling Asymmetric matching and occlusion leading to unmatched targets will affect the similarity matrix. Expand into one The matrix is ​​augmented with learnable "dustbin" rows and columns to absorb unmatched targets;

[0025] S403, Sinkhorn Iterative Solution: The Sinkhorn algorithm is used to iteratively optimize the expanded matrix. Through row and column normalization, a differentiable optimal transmission strategy matrix is ​​obtained. The matrix The former The submatrix represents the final matching result.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] 1. A lightweight scene description method is proposed. By constructing a topology graph, the scene is expressed using only the target categories and their relative positional relationships. It does not rely on the global absolute coordinates of the targets, which significantly reduces communication overhead and fundamentally weakens the dependence on high-precision global positioning.

[0028] 2. A multi-level scene encoding mechanism was designed, which uses GATv2 to transmit information on the topology graph. It efficiently aggregates the view-invariant features, geometric features, texture features and topological relationship features of the target, and generates a scene-level descriptor that is robust to occlusion and viewpoint changes, effectively solving the problem of distinguishing similar targets.

[0029] 3. An optimal transmission strategy is introduced, and the Sinkhorn algorithm with a "trash can" mechanism is adopted, which can efficiently match cross-view scene content and effectively solve the problem of asymmetric target quantity caused by occlusion or different detection ranges at both ends of the vehicle and road.

[0030] 4. An integrated bounding box correction module is used to optimize the localization deviation of the input 3D bounding boxes through PointNet, ensuring the accuracy of the topology map construction, providing high-precision input for subsequent robust matching, and improving the overall performance of the framework. Attached Figure Description

[0031] Figure 1 This is the overall architecture diagram of the "Vehicle-Road Cooperative Cross-Perspective Matching Framework Based on Topological Scene Expression" of the present invention;

[0032] Figure 2 This is a schematic diagram illustrating the principle of "feature aggregation based on topology" in this invention;

[0033] Figure 3 This is a schematic diagram illustrating the principle of "optimal transmission matching" in this invention;

[0034] Figure 4 A visualization of the matching results of the method of the present invention;

[0035] Figure 5 This is a schematic diagram illustrating the construction of the "topology diagram" of the method of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Please see Figure 1-5 This invention provides a technical solution: a vehicle-road cooperative cross-view target matching method based on topological scene representation, specifically including the following steps:

[0038] Step 1: Topology Scene Construction and Bounding Box Correction

[0039] To reduce the communication overhead of full data transmission and weaken the reliance on high-precision positioning, such as Figure 1 and Figure 5 As shown, a scene representation method based on topology is proposed. This step first optimizes the input accuracy through bounding box correction, and then constructs a lightweight topology graph to represent the scene.

[0040] Step 1 specifically includes the following sub-steps:

[0041] Step 11, Bounding Box Correction: Due to potential positioning errors in sensor detection results, a PointNet-based regressor is first used to process the point cloud within the input 3D bounding box. This network is trained to predict a 6-DoF (translation and rotation) value. The correction vector is used to reverse the bounding box offset. During inference, this correction vector is applied to the original bounding box to obtain a more accurate target pose.

[0042] Step 12, Topology Graph Construction: Based on the corrections made in Step 11 One goal, such as Figure 5 As shown, construct a fully connected graph. , among which, nodes The attributes of each edge represent each target. Represent any two targets and The relative positional relationship between targets (such as relative distance, azimuth angle, etc.) is expressed in this way. This way of expressing only cares about "what direction target A is in from target B", and does not care about "the global coordinates of target A". Therefore, it is naturally robust to global positioning errors and the amount of data is extremely small.

[0043] Step 2: Multi-level scene coding mechanism

[0044] To address the significant differences in viewpoints, incomplete point clouds caused by occlusion, and changes in appearance in vehicle-to-infrastructure (V2I) communication, such as... Figure 1 As shown, a multi-level, multi-modal feature encoding mechanism is designed to extract robust features for each target from different dimensions.

[0045] Step 2 specifically includes the following sub-steps:

[0046] Step 21, View-Invariant Feature Encoding: To address the issue of incomplete point clouds caused by occlusion, a DeepSDF module is introduced. A decoder (multi-layer MLP) is trained to learn a continuous signed distance function (SDF). This function can infer the complete, view-independent 3D shape of the target from the incomplete point cloud input and encode it into a compact latent vector. Its decoder function is defined as:

[0047]

[0048] in For spatial coordinates, These are latent vectors. The final output is the view-invariant feature. ;

[0049] Step 22, Geometric Feature Encoding: A Point Transformer network is used to process the point cloud within the corrected bounding box. This network utilizes a self-attention mechanism to capture the global and local geometric structure of the point cloud and outputs the explicit geometric features of the target. ;

[0050] Step 23, Texture Feature Encoding: Using the Vision Transformer (ViT) network, the input is the cropped region corresponding to the target in the 2D image. ViT effectively captures the texture details such as color and pattern of the target through its global attention mechanism, and outputs texture features. .

[0051] Step 3: Topology Aggregation and Feature Fusion

[0052] To effectively distinguish similar targets in dense scenes (such as vehicles of the same model) and generate a final scene descriptor robust to occlusion and viewpoint changes, such as... Figure 1 and Figure 2 As shown, an aggregation mechanism that integrates topological relationships and multimodal features is proposed.

[0053] Step 3 specifically includes the following sub-steps:

[0054] Step 31, Feature aggregation based on topology: such as Figure 2As shown, to effectively distinguish targets with similar appearances, the global topological constraints of the scene are utilized, and a GATv2 network is employed to propagate information on the fully connected topology graph constructed in step 12. Unlike the standard GATv2, this invention explicitly incorporates the relative position vectors between targets when calculating attention weights. As a dynamic topological constraint, this enables the network to perceive that "similar car A is on the left side of the truck, and similar car B is on the right side of the truck," thereby distinguishing them and outputting topological features. ;

[0055] Step 32, Cross-modal feature fusion: To achieve complementarity of multimodal features, a Transformer-based fusion module is designed to fuse the four features obtained in Step 2 and Step 31. , , , As an independent token sequence input;

[0056] Step 33, Gating Mechanism and Scene Descriptor: To dynamically balance the contributions of different modalities (e.g., reducing texture features at night). (Weights), this step introduces learnable modality preference weights, defined as:

[0057]

[0058] Furthermore, a cross-modal gating mechanism is applied before self-attention computation. After processing by a multi-layer Transformer encoder, the output tokens are aggregated and L2 normalized to obtain the final scene descriptor. .

[0059] Step 4: Optimal transmission matching strategy

[0060] like Figure 3 As shown, in order to achieve efficient and robust matching of scene descriptors between the vehicle and the roadside, and to solve the problem of asymmetry in the number of targets at both ends due to perception range or occlusion, an optimal transmission strategy is introduced.

[0061] Step 4 specifically includes the following sub-steps:

[0062] Step 41, Similarity Matrix and Trash Can Expansion: Calculate the set of vehicle-side scene descriptors roadside scene descriptor set Cosine similarity between them, construct a similarity matrix :

[0063]

[0064] To handle the asymmetry of target quantity ( And unmatched targets caused by occlusion, Expand to Add a row and a column as "trash cans", and the trash can learning score is a learnable parameter;

[0065] Step 42, Sinkhorn Iterative Solution: The Sinkhorn algorithm is used to efficiently solve for the optimal transmission strategy. The algorithm obtains the final matching matrix through T iterations of row and column normalization operations. (That is, the first N rows and the first M columns of the extended matrix):

[0066] ;

[0067] Step 43, Matching Loss and Result Output: After T iterations, convergence, Center front submatrix This is the final soft-match probability matrix. During training, the matching loss (negative log-likelihood loss) is used to... Matching matrix with truth values Supervision:

[0068]

[0069] In reasoning, directly from The matching result is decoded from the code.

[0070] More specifically, Figure 1 The overall framework of the invention is shown, using vehicle-side and roadside sensor (camera, lidar) data and their corresponding 3D bounding boxes as input.

[0071] First, the original 3D bounding box is optimized for accuracy using a PointNet bounding box correction module. Then, as... Figure 5 As shown, based on the corrected targets and their relative positions, a fully connected topology graph is constructed to lightweightly represent the scene. Then, for each target in the topology graph, four features are extracted in parallel: 1) view-invariant implicit shape features are extracted using DeepSDF; 2) point cloud geometric features are extracted using Point Transformer; 3) image texture features are extracted using ViT. Then, as... Figure 2 As shown, the three extracted features, along with the target's topological relationships, are input into a GATv2 network for aggregation to obtain topology-aware features. Finally, all four features are input into a Transformer fusion module with a gating mechanism to generate the final cross-modal scene descriptor, such as... Figure 3As shown, the generated vehicle-side and roadside scene descriptors are then used to construct a similarity matrix, and the optimal transmission strategy is solved using the Sinkhorn algorithm to output stable and reliable matching pairs.

[0072] Example:

[0073] This invention was validated on the publicly available DAIR-V2X dataset, which contains real-world vehicle-road cooperative scenarios on urban roads and intersections. High-precision annotations were used as ground truth, and average detection accuracy (mAP) and AP50 (average accuracy when IoU_threshold=0.5) were used as the core evaluation metrics.

[0074] The method of this invention (Ours) is compared with current advanced vehicle-road cooperative perception methods, including TCLF, Where2comm and V2X-ViT. The test results are shown in Table I.

[0075] Table I. Comparison of matching accuracy with mainstream vehicle-road cooperative perception methods

[0076]

[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A vehicle-road cooperative cross-view target matching method based on topological scene representation, characterized in that, Includes the following steps: Step S1, Topology Scene Construction and Boundary Box Correction: On the vehicle and roadside, the original 3D bounding boxes of the targets to be detected are corrected, and a fully connected topology map is constructed based on the relative positional relationship between the corrected targets to express the scene in a lightweight way and reduce the dependence on high-precision positioning. Step S2, Multi-level Scene Encoding: For each target node in the topology graph, DeepSDF is used to infer its view-invariant geometric features, and Point Transformer is used to extract point cloud geometric features and Vision Transformer is used to extract image texture features to generate multimodal basic features of the target. Step S3, Topology Aggregation and Feature Fusion: A graph attention network (GATv2) is used to aggregate the topological relationship features of neighboring nodes, and a gated Transformer encoder is used to fuse the view-invariant features, geometric features, texture features and topological features to generate a cross-modal scene descriptor that is robust to occlusion and viewpoint changes. Step S4, Optimal Transmission Matching: Construct a similarity matrix between the vehicle-side and roadside scene descriptors, and to address the issues of asymmetric target numbers and occlusion, use the Sinkhorn algorithm with a "trash can" mechanism to solve for the optimal transmission strategy, outputting robust cross-view target matching results.

2. The vehicle-road cooperative cross-view target matching method based on topological scene representation according to claim 1, characterized in that, The topology scene construction and bounding box correction in step S1 include: S101, Bounding Box Correction: The PointNet network is used to process the point cloud within the input 3D bounding box and regress to predict a 6-DOF (6-DoF) correction vector to optimize the position and pose errors of the original bounding box caused by detection or localization bias. S102. Construction of Topology Graph: Each target after correction in S101 is used as a graph node, and the relative positional relationship between any two target nodes is used as the edge of the graph to construct a fully connected topology graph. This graph only contains the relative spatial distribution between targets and does not depend on global absolute coordinates.

3. The vehicle-road cooperative cross-view target matching method based on topological scene representation according to claim 1, characterized in that, The multi-level scene encoding in step S2 includes: S201, Viewpoint Invariant Feature Encoding: The DeepSDF (Deep Signed Distance Function) module is used to extract compact 3D shape feature representations by learning continuous SDF, enabling it to infer viewpoint-independent global geometric features from incomplete or noisy point clouds. S202, Geometric Feature Encoding: The Point Transformer network is used to capture the global and local structure of the target point cloud through its self-attention mechanism, and to extract the explicit geometric features of the target. S203. Texture Feature Encoding: The Vision Transformer (ViT) network is used to extract global context and local texture details from the 2D cropped image corresponding to the target as the texture features of the target.

4. The vehicle-road cooperative cross-view target matching method based on topological scene representation according to claim 1, characterized in that, The topology aggregation and feature fusion in step S3 include: S301. Topological Feature Aggregation: Using the GATv2 network, information is propagated across the fully connected topological graph, and the relative position vectors between targets are explicitly incorporated as dynamic topological constraints. Global contextual information is aggregated, and its attention score is defined as: in, , For node features, It is a relative position vector. For learnable topological embeddings; S302, Cross-modal feature fusion: Design a Transformer-based fusion module that takes the view-invariant features, geometric features, texture features and topological features obtained in S2 and S301 as independent token sequences as input; S303, Gating Mechanism and Scene Descriptor: Modality priority weight and cross-modality gating mechanism are introduced in the fusion module to dynamically balance the contribution of each modality, and complementary information between modalities is captured through the Transformer coding layer, and finally aggregated to generate L2 normalized scene descriptors.

5. The vehicle-road cooperative cross-view target matching method based on topological scene representation according to claim 1, characterized in that, The optimal transmission matching in step S4 includes: S401. Similarity Matrix Construction: Calculate the cosine similarity between N target descriptors in the vehicle-side scene and M target descriptors in the roadside scene, and construct a similarity matrix. Similarity matrix ; S402, Trash Can Expansion: For handling Asymmetric matching and occlusion leading to unmatched targets will affect the similarity matrix. Expand into one The matrix is ​​augmented with learnable "dustbin" rows and columns to absorb unmatched targets; S403, Sinkhorn Iterative Solution: The Sinkhorn algorithm is used to iteratively optimize the expanded matrix. Through row and column normalization, a differentiable optimal transmission strategy matrix is ​​obtained. The matrix The former The submatrix represents the final matching result.

Images