Safe driving method, system and device based on multi-vehicle millimeter wave cooperative perception

Abstract

The application relates to the technical field of automatic driving and discloses a multi-vehicle millimeter wave cooperative perception safe driving method, system and equipment. The method comprises the following steps: acquiring point cloud data of a local vehicle and cooperative vehicles, selecting and separating the point cloud data based on a speed auxiliary selection and a lightweight view cone separation scheme to obtain point cloud data of a moving target; constructing a cross-space information transmission network; performing cross-view angle correlation and information updating on the point cloud features selected by the local vehicle and the cooperative vehicles based on the cross-space information transmission network, so that a co-view vehicle is determined; and optimizing the radar point cloud of the co-view vehicle based on a background constraint alignment optimization algorithm, so that multi-vehicle cooperative perception is realized. The clear point cloud data of the moving target is efficiently acquired based on the speed auxiliary point cloud selection scheme, and the radar point cloud data of the moving vehicle is accurately filtered and separated in combination with the lightweight view cone separation scheme. The cross-space information transmission network is adopted for co-view target vehicle detection, so that the co-view vehicle in the cross-view angle is uniquely and accurately identified. The background constraint alignment optimization algorithm is adopted to improve the precision and the accuracy of the spatial calibration. The radar point cloud of the co-view target is used to calculate the transfer matrix between the vehicle view angles, so that accurate, real-time and adaptive multi-vehicle cooperative perception and safe driving are realized.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a safe driving method, system and device for multi-vehicle millimeter-wave collaborative perception. Background Technology

[0002] Autonomous driving is considered a promising technology that will transform daily transportation. To ensure the reliability and safety of driving systems, various sensors, such as LiDAR, cameras, and radar, are deployed to perceive and understand the surrounding traffic conditions. However, single-vehicle perception systems face challenges such as obstacle occlusion and limited perception range, severely impacting their understanding of the surrounding environment and safe driving planning. To address this, multi-vehicle cooperative perception technology has opened up a new paradigm: multiple interconnected autonomous vehicles share sensor data to overcome the limitations of individual perception and improve the safety of autonomous driving. Figure 1 As shown, at a complex intersection, local vehicles cannot detect other vehicles due to occlusion. In contrast, by sharing sensor information with collaborating vehicles, local vehicles can successfully perceive and detect occluded vehicles and accurately locate their spatial distribution through spatial calibration alignment.

[0003] Similar to real-time localization and mapping (SLAM) systems, cross-view sensor data alignment is crucial for achieving multi-vehicle cooperative perception and safe driving. This involves the local vehicle acquiring a spatial transfer matrix between itself and collaborating vehicles to perform cross-view calibration and alignment of shared sensor data, ultimately transforming it to the local viewpoint to improve vehicle perception performance. However, designing robust and efficient cross-view alignment schemes is more challenging for multi-vehicle cooperative systems than single-node SLAM. It must meet three key requirements: First, to meet the demands of various safe driving tasks such as path planning, lane changing, and overtaking, the cross-view alignment scheme must achieve decimeter-level calibration accuracy. Second, system processing latency must be completed within tens of milliseconds to avoid catastrophic accidents. Finally, dynamic environments and viewpoint differences significantly impact data alignment performance, thus requiring the perception scheme to be highly adaptable to these challenging conditions.

[0004] Traditional multi-vehicle cross-view alignment schemes mainly focus on: 1) relying on the precise attitude of vehicles to calibrate and align sensor data. However, the average positioning error measured using a Global Navigation Satellite System (GPS) is several meters, which is difficult to guarantee the accuracy requirements for many driving tasks in real-world traffic scenarios. 2) performing cross-view alignment based on extracting representative features from dense LiDAR point clouds. However, sharing high-density LiDAR points under limited bandwidth poses a significant challenge to latency. In addition, LiDAR modules with dense point clouds are expensive and easily affected by smoke and haze. 3) performing spatial calibration by sharing landmark key points extracted from visual images. However, the method of extracting landmarks based on ground images is easily affected by ambient lighting, texture, and vehicle occlusion, leading to incorrect detection and extraction of objects and their geometry, thereby reducing the accuracy of cross-view alignment. Therefore, how to design an accurate, real-time, and adaptive cross-view alignment scheme to improve the performance of multi-vehicle cooperative perception is a key problem that still needs to be solved in safe driving systems. Summary of the Invention

[0005] The main objective of this invention is to provide a safe driving method, system, and device for multi-vehicle millimeter-wave collaborative perception, aiming to solve at least one of the aforementioned technical problems.

[0006] To achieve the above objectives, the present invention provides a safe driving method based on multi-vehicle millimeter-wave cooperative perception, comprising:

[0007] The point cloud data of local vehicles and cooperative vehicles are acquired, and the point cloud data is selected and separated based on the speed-assisted selection and lightweight view cone separation scheme to obtain the point cloud data of the moving target.

[0008] Construct a cross-space information transmission network;

[0009] Based on the cross-space information transmission network, cross-view correlation and information update are performed on the point cloud features selected by local vehicles and cooperative vehicles to determine the co-view vehicles.

[0010] An alignment optimization algorithm based on background constraints is used to optimize the radar point cloud of the shared-view vehicle to achieve multi-vehicle cooperative perception.

[0011] In some embodiments, acquiring point cloud data of local vehicles and cooperative vehicles, and selecting and separating the point cloud data based on a speed-assisted selection and lightweight view frustum separation scheme to obtain point cloud data of the moving target, includes:

[0012] Point cloud data of local vehicles and cooperative vehicles are acquired separately; wherein, the point cloud data includes image frames and radar point cloud frames;

[0013] Point cloud selection is performed based on a velocity-assisted selection scheme to distinguish between stationary objects and moving targets, and to identify radar points of moving targets;

[0014] A lightweight separation scheme based on view cone projection separates points within the view cone of the region of interest from the image frame and the radar point cloud frame.

[0015] Based on the points within the view frustum of the region of interest, noise processing is performed to obtain point cloud data of the same moving target, thereby improving the accuracy of point cloud separation.

[0016] In some embodiments, the velocity-assisted selection scheme for point cloud selection to distinguish between stationary objects and moving targets, and to identify radar points of moving targets, includes:

[0017] Establish a coordinate system based on the vehicle's direction of travel and determine the offset angle of the radar's installation direction:

[0018] Based on the offset angle and the collected point cloud information, the directional angles and corresponding radial velocities of all objects in the field of view are obtained;

[0019] Based on the relationship between the radial velocity of all objects and the coordinates of the corresponding target direction, a sine curve is fitted to determine the radar point of stationary objects.

[0020] Based on the fitted sine curve, radar point clouds that deviate from the sine curve are identified as radar points corresponding to moving targets.

[0021] In some embodiments, the lightweight separation scheme based on frustum projection, which separates points within the frustum of the region of interest based on the image frame and the radar point cloud frame, includes:

[0022] Establish a cone propagation model;

[0023] The image frame is input into the target detection algorithm for preprocessing to obtain heat map information, depth information, and bounding box information;

[0024] Based on the heatmap information, depth information, and bounding box information, the bounding box and estimated depth of the object are determined, and the region of interest frustum is determined and the depth of the region of interest frustum is expanded according to the bounding box and estimated depth.

[0025] Point cloud selection is performed based on the radar point cloud frame, the selected point cloud data is projected onto the bird's-eye view, and the points projected onto the view frustum of the region of interest are determined.

[0026] In some embodiments, obtaining point cloud data of the same moving target by performing noise processing based on points within the view frustum of the region of interest includes:

[0027] Collect the radar points of the identified moving targets and the points projected into the view cone of the region of interest to obtain the initial point cloud data of the moving targets;

[0028] Using the position and velocity information of the point cloud as input constraints, a filter based on spatial-Doppler two-dimensional density clustering is designed to clean the initial point cloud data, thereby filtering out noise and obtaining the point cloud data of the moving target.

[0029] In some embodiments, constructing a cross-space information transmission network includes:

[0030] Construct a graph structure based on the detected target; wherein the nodes in the graph structure correspond to semantic features in the image, and the edges in the graph structure represent the connection relationships established using spatial information of millimeter-wave point clouds;

[0031] At the node end of the graph structure, embedded features are extracted by an encoder based on a multilayer perceptron;

[0032] The edges of the graph structure are constructed in two phases: local vehicle construction and cross-vehicle space construction.

[0033] In some embodiments, the step of performing cross-view correlation and information update on point cloud features selected by local vehicles and cooperative vehicles based on the cross-spatial information transmission network to determine co-view vehicles includes:

[0034] In the cross-spatial information transmission network, edge embedding feature updates and node embedding feature updates are performed in both the local and cross-view perspectives.

[0035] When updating edge embedding features, edges are divided into those connecting target nodes in the local vehicle view and those connecting target nodes across vehicle space, and edge embedding feature updates are performed to obtain the updated edge embedding feature set.

[0036] When updating node embedding features, iterate the connection edges and node embedding features to obtain the updated node embedding features;

[0037] The system constructs connections between all targets and learns the associations between all targets by weighted cross-view edges, thereby identifying the co-view vehicle.

[0038] In some embodiments, the background-constrained alignment optimization algorithm optimizes the radar point cloud of the shared-view vehicle to achieve multi-vehicle cooperative perception, including:

[0039] Clustering is used to filter out outliers caused by noise and multipath reflections from static background point clouds, retaining only background point cloud pairs containing roadside and building elements.

[0040] The initial transformation and initial rotation matrix are obtained by performing preliminary iterations based on the processed common-view vehicle radar point cloud.

[0041] The alignment optimization algorithm based on background constraints combines the point cloud pairs of the co-viewing vehicles and the radar point pairs of the background for continuous iteration, and calculates the transition matrix between the two viewpoints to improve the registration accuracy and realize multi-vehicle cooperative perception.

[0042] Furthermore, to achieve the above objectives, this invention also proposes a multi-vehicle millimeter-wave cooperative perception safety driving system, comprising:

[0043] The point cloud separation and selection module is used to acquire point cloud data of local vehicles and cooperative vehicles, and select and separate the point cloud data based on the speed-assisted selection and lightweight view cone separation scheme to obtain the point cloud data of the moving target.

[0044] Cross-space network building module, used to build cross-space information transmission networks;

[0045] The common-view vehicle detection module is used to perform cross-view correlation and information update on the point cloud features selected by local vehicles and cooperative vehicles based on the cross-spatial information transmission network, thereby determining the common-view vehicles.

[0046] The background constraint alignment module is used to optimize the radar point cloud of the shared-view vehicle based on the alignment optimization algorithm of the background constraint, so as to realize multi-vehicle cooperative perception.

[0047] Furthermore, to achieve the above objectives, the present invention also proposes an electronic device comprising: a memory, a processor, and a multi-vehicle millimeter-wave collaborative perception safety driving program stored in the memory and executable on the processor, wherein the multi-vehicle millimeter-wave collaborative perception safety driving program is configured to implement the multi-vehicle millimeter-wave collaborative perception safety driving method as described above.

[0048] This invention provides a safe driving method for multi-vehicle millimeter-wave cooperative perception, comprising: acquiring point cloud data of local vehicles and cooperative vehicles; selecting and separating the point cloud data based on a speed-assisted selection and lightweight cone separation scheme to obtain point cloud data of moving targets; constructing a cross-spatial information transmission network; performing cross-view correlation and information update on the point cloud features selected by local vehicles and cooperative vehicles based on the cross-spatial information transmission network to determine the co-viewing vehicles; and optimizing the radar point cloud of the co-viewing vehicles using a background-constrained alignment optimization algorithm to achieve multi-vehicle cooperative perception. Compared with traditional multi-vehicle cooperative perception schemes, this invention's speed-assisted point cloud selection scheme efficiently acquires clear point cloud data of moving targets, and combined with a lightweight cone separation scheme, accurately filters and separates the radar point cloud data of moving vehicles. The use of a cross-spatial information transmission network for co-viewing target vehicle detection achieves unique and accurate identification of co-viewing vehicles across different viewpoints. Furthermore, the use of a background-constrained alignment optimization algorithm improves accuracy, thereby enhancing the accuracy of spatial calibration. Based on this, the vehicle perspective transfer matrix is ​​achieved by computing radar points of the shared target, thereby realizing accurate, real-time and adaptive multi-vehicle cooperative perception and safe driving. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of multi-vehicle collaborative perception involved in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiments of the present invention;

[0051] Figure 3 This is a flowchart illustrating an embodiment of the multi-vehicle millimeter-wave collaborative sensing safe driving method of the present invention;

[0052] Figure 4 This is a schematic diagram of the technical architecture of the multi-vehicle millimeter-wave collaborative perception safe driving method involved in the embodiments of the present invention;

[0053] Figure 5 This is a schematic diagram of the lightweight separation scheme based on view cone projection involved in the embodiments of the present invention;

[0054] Figure 6 This is a schematic diagram of a cross-space information transmission network involved in an embodiment of the present invention;

[0055] Figure 7 This is a schematic diagram illustrating the edge embedding feature update and node embedding feature update involved in the embodiments of the present invention;

[0056] Figure 8 This is a structural block diagram of an embodiment of the multi-vehicle millimeter-wave collaborative perception safety driving system of the present invention.

[0057] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0059] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0060] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention.

[0061] Reference Figure 2 , Figure 2 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention.

[0062] like Figure 2As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0063] Those skilled in the art will understand that Figure 2 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0064] like Figure 2 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a safety driving program for multi-vehicle millimeter-wave collaborative perception.

[0065] exist Figure 2 In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device, and the electronic device calls the multi-vehicle millimeter-wave cooperative perception safety driving program stored in the memory 1005 through the processor 1001, and executes the multi-vehicle millimeter-wave cooperative perception safety driving method provided in the embodiment of the present invention.

[0066] This invention proposes a safe driving method, system, and device for multi-vehicle millimeter-wave collaborative sensing.

[0067] This invention provides a safe driving method based on multi-vehicle millimeter-wave collaborative perception, referring to... Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of the multi-vehicle millimeter-wave collaborative sensing safe driving method of the present invention.

[0068] like Figure 3As shown, the multi-vehicle millimeter-wave collaborative perception safe driving method includes:

[0069] Step S100: Acquire point cloud data of local vehicles and cooperative vehicles, and select and separate the point cloud data based on the speed-assisted selection and lightweight view frustum separation scheme to obtain point cloud data of moving targets;

[0070] Step S200: Construct a cross-space information transmission network;

[0071] Step S300: Based on the cross-space information transmission network, cross-view association and information update are performed according to the point cloud features selected by the local vehicle and the cooperative vehicle, thereby determining the co-view vehicle;

[0072] Step S400: The radar point cloud of the co-view vehicle is optimized by an alignment optimization algorithm based on background constraints to achieve multi-vehicle cooperative perception.

[0073] It should be noted that the execution subject in this embodiment can be an electronic device, which can be a computer device with data processing function, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, a computer device is used as an example for explanation.

[0074] It is understood that this embodiment uses millimeter-wave radar point cloud alignment in an autonomous driving scenario as an example for illustration. Figure 4 As shown, the process of the multi-vehicle millimeter-wave cooperative perception safe driving method includes the following three key steps: radar selection and separation, common-view target vehicle detection, and background constraint alignment. These steps are explained in detail below.

[0075] In one embodiment, point cloud data of local vehicles and cooperative vehicles are acquired, and the point cloud data is selected and separated based on a speed-assisted selection and lightweight view cone separation scheme to obtain point cloud data of moving targets. This includes: acquiring point cloud data of local vehicles and cooperative vehicles respectively; wherein the point cloud data includes image frames and radar point cloud frames; selecting points based on a speed-assisted selection scheme to distinguish between stationary objects and moving targets, and identifying radar points of moving targets; separating points within the view cone of the region of interest based on a lightweight view cone projection scheme, according to the image frames and radar point cloud frames; and performing noise processing on the points within the view cone of the region of interest to obtain point cloud data of the same moving target, thereby improving the accuracy of point cloud separation.

[0076] Specifically, radar selection and separation: Due to specular reflection and interference, simply using all point cloud data for viewpoint alignment introduces serious matching errors and complex computational problems. Therefore, this embodiment first designs a velocity-assisted point cloud selection scheme to efficiently acquire clear point cloud data of moving targets. Subsequently, unlike radar-vision methods that require retraining through large labeled datasets and fine-grained feature extraction, this embodiment combines a monocular camera to design a lightweight separation scheme based on view cone projection to accurately filter and separate the radar point cloud data of moving vehicles.

[0077] In one embodiment, a speed-assisted selection scheme is used to select point clouds to distinguish between stationary objects and moving targets, and to identify radar points of moving targets. This includes: establishing a coordinate system based on the vehicle's driving direction and determining the offset angle of the radar installation direction; obtaining the directional angles and corresponding radial velocities of all objects in the field of view based on the offset angle and the collected point cloud information; fitting a sine curve based on the coordinate relationship between the radial velocities of all objects and the corresponding target directions to determine radar points of stationary objects; and identifying radar point clouds that deviate from the fitted sine curve as radar points corresponding to moving targets.

[0078] Specifically, due to specular reflection and interference, simply using all point cloud data for viewpoint alignment introduces serious matching errors and complex computational challenges. Therefore, this embodiment first designs a velocity-assisted selection scheme to efficiently acquire clear point cloud data of moving targets. The core of this scheme lies in the fact that radar point cloud data of stationary backgrounds and moving targets exhibit significantly different patterns in radial velocity, and these patterns can be used to identify the point cloud data of moving targets.

[0079] For example, assuming the local vehicle is moving along the Y-axis in a planar XY coordinate system, the mounting direction of the onboard radar is offset by an angle θ from the vehicle's direction of travel. Since Doppler radar can only capture radial velocity, the velocity of any stationary object can be expressed as v. t However, the direction is opposite to that of the radar module. At this point, the velocity of the stationary target can be projected onto the X and Y axes, yielding its velocity components as v. x =-v t sin(θ) and v y =-v t cos(θ). Meanwhile, for any point i (whose azimuth angle is ω) i Its radial velocity can be obtained as follows:

[0080]

[0081] According to the above formula, all stationary radar points will fall on a sine curve. In contrast, for a moving vehicle with a velocity angle of Δα, its radial velocity is:

[0082]

[0083] Comparing the two equations above, radar points from moving vehicles will deviate from the sine curve due to the additional term generated by their motion, becoming outliers. These point clouds differ significantly from stationary targets due to the radial velocity changes caused by their motion, thus effectively distinguishing moving targets from stationary background objects.

[0084] In one embodiment, a lightweight separation scheme based on frustum projection separates points within the region of interest (ROI) frustum according to the image frame and radar point cloud frame, including: establishing a frustum propagation model; inputting the image frame into a target detection algorithm for preprocessing to obtain heatmap information, depth information, and bounding box information; determining the object's bounding box and estimated depth based on the heatmap information, depth information, and bounding box information; determining the ROI frustum and extending its depth based on the bounding box and estimated depth; selecting points based on the radar point cloud frame; projecting the selected point cloud data onto a bird's-eye view; and determining the points projected into the ROI frustum.

[0085] Specifically, this embodiment first designs a velocity-assisted selection scheme to efficiently acquire clear point cloud data of moving targets. Then, unlike radar-vision fusion methods that require retraining through large labeled datasets and fine-grained feature extraction, this embodiment combines a monocular camera and designs a lightweight separation scheme based on view cone projection to accurately filter and separate radar point cloud data of moving vehicles. The basic idea is that both radar and camera exhibit view cone propagation patterns, and they can work together to determine the distribution of point cloud data.

[0086] It's important to note that in computer vision and 3D reconstruction, the frustum propagation model is a model that utilizes sensor data from cameras and radar to understand spatial relationships within a scene. In this model, a 2D heatmap is a data representation used to visualize and process data from sensors. A 2D heatmap represents 3D spatial information on a 2D plane; specifically, it's a 2D matrix where each element represents a pixel, and the pixel's value represents a metric at that location, such as depth, probability, or other scene-related information. A 2D heatmap is generated by projecting point cloud data in 3D space onto a 2D plane. This process typically involves projecting the point cloud data according to the camera's frustum (i.e., field of view).

[0087] For example, such as Figure 5 As shown, first, a view cone propagation model is created, and its 2D heatmap is defined. Where W and H represent the width and height of the image, respectively, K is the camera intrinsic parameter matrix, and L = [X, Y, Z] represents the position within the view frustum. Then, by combining the object's bounding box and its estimated depth d = ||L||, a region of interest (RoI) view frustum can be determined, indicating the possible distribution space of the point cloud data. In this embodiment, to mitigate the impact of inaccurate depth estimation, the depth of the RoI view frustum is expanded to Δd to include more adjacent point cloud data. Furthermore, measurement errors in the height dimension can lead to incorrect separation of point cloud data; therefore, as... Figure 5 As shown, this embodiment projects point cloud data onto a bird's-eye view (BEV) and determines the projection onto the RoI view cone F. i Points within the region of interest (points projected onto the view frustum of the region of interest) can be represented as: Finally, a series of point cloud data were collected. They correspond to this moving target vehicle i, such as Figure 5 As shown, This represents the point cloud dataset obtained from the separation of visual targets.

[0088] In this embodiment, a scheme based on velocity-assisted selection and lightweight frustum separation is adopted to extract reliable and clear point cloud data for alignment, which effectively reduces the impact of sparsity and susceptibility to interference of millimeter-wave point clouds.

[0089] In one embodiment, noise processing is performed on points within the view frustum of the region of interest to obtain point cloud data of the same moving target. This includes: collecting radar points of the identified moving target and points projected onto the view frustum of the region of interest to obtain initial point cloud data of the moving target; using the position and velocity information of the point cloud as input constraints, designing a filter based on spatial-Doppler two-dimensional density clustering to clean the initial point cloud data, thereby filtering out noise and obtaining point cloud data of the moving target.

[0090] In this embodiment, to address the challenges of dataset collection, labeling, and pre-training, a lightweight spatial-Doppler filtering method is designed to remove noise. The core idea is that when radar point cloud data originates from the same target, the point clouds cluster together to form a clustered distribution, while point clouds contaminated by noise may exhibit a random distribution. Furthermore, radar point clouds from the same vehicle will show similarity in both velocity and position, causing the point clouds to exhibit a clustered pattern in both dimensions. Therefore, as... Figure 5 As shown, this embodiment uses the position and velocity information of the point cloud as input constraints and employs a filter based on a density clustering algorithm. This process cleans the point cloud data. In this way, noisy points can be effectively filtered out while retaining point clouds from the same target, thus improving the accuracy of point cloud separation.

[0091] Specifically, such as Figure 4 As shown, after point cloud selection and separation, the cooperative vehicle shares and transmits image-radar frames (image frames and radar point cloud frames) to the local vehicle for perception fusion. Due to the sparsity of point cloud data, the cooperative vehicle can transmit not only point cloud data of moving targets but also point cloud data of stationary backgrounds to improve accuracy. For image frames, since the raw data would occupy a large amount of bandwidth, this embodiment compresses the data, sharing only the feature maps of the detected vehicles. This approach effectively reduces the amount of data transmitted while retaining key perception information for efficient cross-view alignment calibration.

[0092] In one embodiment, constructing a cross-spatial information transmission network includes: constructing a graph structure based on the detected target; wherein the nodes in the graph structure correspond to semantic features in the image, and the edges in the graph structure represent the connection relationships established using spatial information from millimeter-wave point clouds; at the node end of the graph structure, embedding features are extracted by an encoder based on a multilayer perceptron; and the edges of the graph structure are constructed using two stages: local vehicle construction and cross-vehicle spatial construction.

[0093] In one embodiment, based on the cross-spatial information transmission network, cross-view association and information update are performed on the point cloud features selected by local vehicles and cooperative vehicles to determine the co-view vehicle. This includes: in the cross-spatial information transmission network, performing edge embedding feature updates and node embedding feature updates in the local view and cross-view respectively; during edge embedding feature updates, the edges are divided into those connecting target nodes in the local vehicle view and those connecting target nodes in the cross-vehicle space, and edge embedding feature updates are performed to obtain an updated edge embedding feature set; during node embedding feature updates, the connection edges and node embedding features are iteratively updated to obtain updated node embedding features; connections between all targets are constructed, and the association between all targets is learned by weighting the cross-view edges to determine the co-view vehicle.

[0094] It is understandable that, such as Figure 4As shown, for common-view vehicle detection: To achieve unique and accurate identification of common-view vehicles across different viewpoints, this embodiment designs a cross-spatial information transfer network to learn and understand the association characteristics between vehicles. Specifically, this embodiment first constructs a graph structure association based on the detected targets, where nodes correspond to semantic features in the image, and edges represent connections established using spatial information from millimeter-wave point clouds. Furthermore, to construct cross-view associations, a cross-spatial information transfer network is designed to update the nodes and edges in the graph, thereby understanding the association characteristics between target vehicles. Finally, based on a predefined multi-label classifier, association weights are measured and common-view vehicles are identified.

[0095] Specifically, to achieve unique and accurate identification of co-existing vehicles across different perspectives, this embodiment designs a cross-spatial information transmission network for constructing and understanding the correlation characteristics between vehicles. For example... Figure 6 As shown, the design first constructs a graph structure G = (V, E) based on the detected target, where nodes v i ∈V corresponds to the semantic features in the image, and edge e i,j ∈E represents the connection relationship established using spatial information from millimeter-wave point clouds. Specifically, at the node end, an encoder ε based on a multilayer perceptron (MLP) is used. v Directly extract the embedding features, where the initial node embedding features of node i are: Regarding the construction of edges, due to different reference frames, in this embodiment, the design of edges is divided into two stages: local vehicle construction and cross-vehicle space construction.

[0096] For example, during the local vehicle construction phase, nodes are connected by relevant distances and angles:

[0097]

[0098] Where, d i,j d represents the distance between nodes i and j. max θ is the maximum distance across all edges. i and θ j These are the angles from the sensor's perspective, and θ is the angle of the edge e. i,j The angle. Then, exemplarily, during the cross-vehicle space construction phase, the nodes are assigned an initial value of 0. Subsequently, the edges are input to the encoder ε based on a multilayer perceptron (MLP). e In order to obtain the initial edge embedding features. During the embedding layer processing, the image features of each vehicle are encoded as 1×32 node embedding features, while the corresponding radar point cloud data is encoded as 1×16 edge embedding features. Finally, connections between all targets are constructed, and the associations between all targets are learned by weighted cross-view edges, thereby identifying the co-view vehicles.

[0099] It should be noted that, unlike continuous frame systems in a single node, nodes and edges in a cooperative vehicle system cannot directly span two independent spaces to update node and edge information. To address this issue, such as... Figure 7 As shown, this embodiment designs a cross-spatial information transmission network for information updating. In this cross-spatial information transmission network, edge embedding feature updates and node embedding feature updates are performed from both the local and cross-viewpoints. During the edge embedding feature update stage, each edge e... i,j ∈E connects two node embedding features, which can therefore be represented as source node embedding features and target node embedding features, respectively. Therefore, updating edge embedding features requires considering the embedding features of the source node, target node, and edge simultaneously. Since the nodes between the local vehicle and the cooperating vehicle are in different spaces, edges can be divided into two categories: one category represents connecting target nodes in the local vehicle's view, and the other category represents connecting target nodes across vehicle spaces. Edge embedding feature updates are then performed on these categories.

[0100] Specifically, assume that the cross-spatial information transfer network performs N information transfer steps in the graph. Therefore, from a local perspective, the edge embedding feature update of a set of nodes in the nth iteration... It can be represented as:

[0101]

[0102] For edges that cross viewpoints, the edge embeddings can be updated by performing a similar operation, where the updated edge embedding features in the nth iteration are represented as... Therefore, by traversing all edges in the graph, the updated edge embedding feature set is finally obtained. During the node embedding feature update phase, each node v i All nodes ∈V are connected by a set of edges, therefore updating the node embedding features requires iterating over the connecting edges and the node embedding features themselves. Understandably, similar to edge embedding feature updates, the update operation can be performed in both local and cross-view scenarios.

[0103] Specifically, the edge embedding features are first updated from the local perspective. The update formula can be expressed as:

[0104]

[0105] in, This represents the previous node embedding feature, while This is the current edge embedding feature. Following this update rule, we can obtain cross-view edge embedding features. Finally, the node embedding features can be updated. And by combining these two types of edge embedding features to connect the two spaces, the update formula is:

[0106]

[0107] After N iterations of message passing, all nodes in the graph have shared each other's information. The ultimate goal is to extract information from a series of edges connecting two independent viewpoint spaces. Find the co-visible target. To do this, embed all output edges into features. Input to a classifier based on a multilayer perceptron (MLP) In this process, the classifier predicts the edge e for each edge. i,j The association probability of ∈E is In this classifier, it can be determined whether an edge originates from the same target from two different viewpoints by comparing it with a threshold η. The output of the classifier is:

[0108]

[0109] To train the model, the loss function is defined as weighted binary cross-entropy, used for predicting common-view vehicles in the last information passing iteration n. The loss function can be expressed as:

[0110]

[0111] Wherein, ω is a weighting factor used to compensate for the height imbalance between the shared-view vehicle edge and other edges, and it is defined as:

[0112]

[0113] Here, 1(x) represents an indicator function used to indicate whether a certain condition is true or false. It returns 1 when condition x is met, and 0 otherwise. Therefore, this embodiment tends to assign larger weights to edge classes that connect to the same shared-view vehicles.

[0114] In one embodiment, a background-constrained alignment optimization algorithm optimizes the radar point cloud of the common-view vehicle to achieve multi-vehicle cooperative perception. This includes: filtering out abnormal points caused by noise and multipath reflections from the static background point cloud using clustering, retaining only background point cloud pairs containing roadside and building elements; performing initial iterations based on the processed common-view vehicle radar point cloud to obtain an initial transformation and initial rotation matrix; acquiring and extracting features from the background point clouds of the local vehicle and the cooperative vehicle respectively to obtain background radar point pairs; and continuously iterating the background-constrained alignment optimization algorithm using the common-view vehicle point cloud pairs and the background radar point pairs to calculate the transfer matrix between the two viewpoints, thereby improving registration accuracy and achieving multi-vehicle cooperative perception.

[0115] Specifically, such as Figure 4As shown, background constraint alignment: To avoid inconsistencies in point cloud coverage space caused by different viewpoints, this embodiment further designs an alignment optimization algorithm based on background constraints to improve accuracy. Specifically, since the point cloud data mapped from both sides of the road and other stationary objects have significant structural features, these features can be used to constrain the alignment process, thereby improving the accuracy of spatial calibration.

[0116] It should be noted that, finally, to avoid inconsistencies in point cloud coverage space caused by different perspectives, this embodiment further designs an alignment optimization algorithm based on background constraints to improve accuracy. The core of this algorithm lies in the fact that radar point clouds from roadsides and other static objects exhibit unique structural features, which can be used to constrain registration. However, typical road types, such as straight roads and intersections, have relatively ideal axisymmetric and rotational symmetric properties, which may lead to misregistration, thus biasing towards the opposite direction or rotationally symmetric direction. Therefore, directly combining radar point pairs of co-view vehicles and the background may have a negative impact on registration. To address this issue, this embodiment first uses clustering to filter out outliers caused by noise and multipath reflections, retaining point clusters from roadsides and buildings. Then, instead of directly performing nearest neighbor point cloud iteration (ICP) processing by combining radar point clouds, it first iterates only on radar point pairs of co-view vehicles to obtain the initial transformation T0. The iteration period is defined as min{L,N(D max )}, where L represents half of the total number of iterations for the common-view vehicle radar point pairs and N(D max ) is when the convergence distance threshold D is reached. max The number of iterations is determined by the spatial distribution of the vehicles in the same line of sight. Since the spatial distribution of the vehicles exhibits a good geometric structure, an approximate rotation matrix R0 can be obtained, thus avoiding registration errors caused by roadside symmetry. Finally, by combining the point pairs of the vehicles in the same line of sight... and and background point clouds and The iteration continues. During this process, the following optimization rules were designed in this embodiment to further improve registration accuracy:

[0117]

[0118] Among them, E n and E g These represent the estimation errors caused by the co-view point cloud pair and the background point cloud, respectively. and These are the weights assigned to them. Specifically, the core idea is that if vehicle detection achieves a high score through CenterNet (a deep learning-based object detection algorithm), then higher weights are assigned to the point cloud pairs of the co-viewing vehicles. Conversely, if only a small number of co-viewing vehicles N are detected, higher weights are assigned.v This will rely more on the background radar point cloud and assign it a higher weight. Through this method, this embodiment can accurately calculate the transition matrix between two viewpoints, achieving precise, real-time, and adaptive multi-vehicle cooperative perception and safe driving.

[0119] Understandably, spatial transformations can typically convert point cloud data from one viewpoint to another. The transition matrix between the two viewpoints can be calculated using the following steps: **Selecting a reference viewpoint:** First, determine a reference viewpoint, usually one of the viewpoints chosen as a reference. **Feature extraction:** Extract features from the point cloud data of both viewpoints. These features can be geometric features of the point cloud, color information, or high-level features extracted through a deep learning model. **Alignment strategy:** Determine the alignment strategy. Feature-based registration algorithms can be used, including but not limited to the Iterative Closest Point (ICP) algorithm, or deep learning-based registration methods.

[0120] For example, feature extraction: for point cloud data of each viewpoint, extract key features; initial registration: use an initial registration algorithm to obtain the initial alignment between the point clouds of the two viewpoints, which can be done by selecting corresponding point pairs or using a feature-based registration method; iterative optimization: use an iterative algorithm to optimize the registration result; output the final transformation matrix: once the iterative process converges, the final transformation matrix T is the transformation matrix between the two viewpoints.

[0121] This embodiment designs the first method for millimeter-wave radar point cloud alignment, enabling accurate and real-time multi-vehicle cooperative perception and safe driving in challenging autonomous driving scenarios using sparse point cloud data. Furthermore, this embodiment designs a scheme based on speed-assisted selection and lightweight cone separation to extract reliable and clear point cloud data for alignment, effectively reducing the impact of the sparsity and susceptibility to interference inherent in millimeter-wave point clouds. Finally, this embodiment proposes a cross-spatial information transmission network to construct and understand target relationships, and designs a background-constrained alignment optimization method to address the sparsity problem, thereby reducing latency while improving alignment accuracy. Compared to traditional multi-vehicle cooperative perception schemes, the multi-vehicle millimeter-wave cooperative perception safety driving mechanism proposed in this embodiment achieves accurate, real-time, and adaptive multi-vehicle cooperative perception and safe driving.

[0122] The method described in this embodiment can accurately align and locate target vehicles captured in images from different viewpoints using millimeter-wave radar. The core idea of ​​this method is based on fine-grained spatial information provided by radar, such as position, speed, and angle. Local and cooperative vehicles can construct unique cross-view associations for all target vehicles from different perspectives. In this way, local and cooperative vehicles can accurately capture and understand the connection topology between each target vehicle and its neighbors, the local role of each target vehicle within the structure, and its global position within the entire viewpoint. Furthermore, local and cooperative vehicles can uniquely identify each target and find commonly visible vehicles. Finally, based on the millimeter-wave point cloud corresponding to the common vehicles from the cross-viewpoints of local and cooperative vehicles, the transition matrix between the two viewpoints can be accurately calculated, thereby achieving precise, real-time, and adaptive multi-vehicle cooperative perception.

[0123] This embodiment provides a safe driving method for multi-vehicle millimeter-wave cooperative perception, including: acquiring point cloud data of local vehicles and cooperative vehicles; selecting and separating the point cloud data based on a speed-assisted selection and lightweight cone separation scheme to obtain point cloud data of moving targets; constructing a cross-spatial information transmission network; performing cross-view correlation and information update on the point cloud features selected by local vehicles and cooperative vehicles based on the cross-spatial information transmission network to determine the co-viewing vehicles; and optimizing the radar point cloud of the co-viewing vehicles using a background-constrained alignment optimization algorithm to achieve multi-vehicle cooperative perception. Compared with traditional multi-vehicle cooperative perception schemes, the method described in this embodiment efficiently acquires clear point cloud data of moving targets based on a speed-assisted point cloud selection scheme, and accurately filters and separates the radar point cloud data of moving vehicles by combining a lightweight cone separation scheme. The use of a cross-spatial information transmission network for co-viewing target vehicle detection achieves unique and accurate identification of co-viewing vehicles across different viewpoints. Furthermore, the use of a background-constrained alignment optimization algorithm improves accuracy, thereby enhancing the accuracy of spatial calibration. Based on this, the vehicle perspective transfer matrix is ​​achieved by computing radar points of the shared target, thereby realizing accurate, real-time and adaptive multi-vehicle cooperative perception and safe driving.

[0124] Furthermore, this embodiment of the invention also proposes a storage medium storing a multi-vehicle millimeter-wave collaborative perception safety driving program. When the multi-vehicle millimeter-wave collaborative perception safety driving program is executed by a processor, it implements the steps of the multi-vehicle millimeter-wave collaborative perception safety driving method described above.

[0125] Reference Figure 8 , Figure 8 This is a structural block diagram of an embodiment of the multi-vehicle millimeter-wave collaborative perception safety driving system of the present invention.

[0126] like Figure 8As shown, the multi-vehicle millimeter-wave collaborative perception safety driving system includes:

[0127] The point cloud separation and selection module 10 is used to acquire point cloud data of local vehicles and cooperative vehicles, and select and separate the point cloud data based on the speed-assisted selection and lightweight view cone separation scheme to obtain the point cloud data of the moving target.

[0128] Cross-space network construction module 20 is used to construct a cross-space information transmission network;

[0129] The common-view vehicle detection module 30 is used to perform cross-view correlation and information update on the point cloud features selected by local vehicles and cooperative vehicles based on the cross-space information transmission network, thereby determining the common-view vehicles.

[0130] Background constraint alignment module 40 is used to optimize the radar point cloud of the co-view vehicle based on the alignment optimization algorithm of background constraint, so as to realize multi-vehicle cooperative perception.

[0131] This embodiment designs the first system based on millimeter-wave radar point cloud alignment, enabling accurate and real-time multi-vehicle cooperative perception and safe driving in challenging autonomous driving scenarios using sparse point cloud data. Furthermore, this embodiment designs a scheme based on speed-assisted selection and lightweight cone separation to extract reliable and clear point cloud data for alignment, effectively reducing the impact of the sparsity and susceptibility to interference of millimeter-wave point clouds. Finally, this embodiment proposes a cross-spatial information transmission network that can construct and understand target relationships, and designs a background-constrained alignment optimization method to address the sparsity problem, thereby reducing latency while improving alignment accuracy. Compared to traditional multi-vehicle cooperative perception schemes, the multi-vehicle millimeter-wave cooperative perception safety driving mechanism proposed in this embodiment achieves accurate, real-time, and adaptive multi-vehicle cooperative perception and safe driving.

[0132] This embodiment provides a multi-vehicle millimeter-wave cooperative perception safety driving system. The system efficiently acquires clear point cloud data of moving targets based on a speed-assisted point cloud selection scheme. Combined with a lightweight view cone separation scheme, it accurately filters and separates the radar point cloud data of moving vehicles. A cross-spatial information transmission network is used for common-view target vehicle detection, achieving unique and accurate identification of common-view vehicles across different viewpoints. Furthermore, a background-constrained alignment optimization algorithm is employed to improve accuracy, thereby enhancing the accuracy of spatial calibration. Based on this, the system calculates the vehicle viewpoint transfer matrix using radar point cloud data of common-view targets, thereby achieving accurate, real-time, and adaptive multi-vehicle cooperative perception and safe driving.

[0133] It should be noted that technical details not described in detail in this embodiment of the multi-vehicle millimeter-wave cooperative perception safe driving system can be found in any embodiment of the present invention for the safe driving method applied to the multi-vehicle millimeter-wave cooperative perception as described above, and will not be repeated here.

[0134] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0135] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0136] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0137] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0139] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for safe driving of multi-vehicle millimeter wave cooperative perception, characterized in that, include: The point cloud data of local vehicles and cooperative vehicles are acquired, and the point cloud data is selected and separated based on the speed-assisted selection and lightweight view cone separation scheme to obtain the point cloud data of the moving target. Construct a cross-space information transmission network; Based on the cross-space information transmission network, cross-view correlation and information update are performed on the point cloud features selected by local vehicles and cooperative vehicles to determine the co-view vehicles. An alignment optimization algorithm based on background constraints is used to optimize the radar point cloud of the shared-view vehicle to achieve multi-vehicle cooperative perception. The method involves acquiring point cloud data of local vehicles and cooperative vehicles, and then selecting and separating the point cloud data based on a speed-assisted selection and lightweight view cone separation scheme to obtain point cloud data of moving targets. This includes: acquiring point cloud data of local vehicles and cooperative vehicles respectively; wherein the point cloud data includes image frames and radar point cloud frames; selecting points based on a speed-assisted selection scheme to distinguish between stationary objects and moving targets, and identifying radar points of moving targets; using a lightweight separation scheme based on view cone projection to separate points within the view cone of the region of interest based on the image frames and radar point cloud frames; and performing noise processing on the points within the view cone of the region of interest to obtain point cloud data of the same moving target, thereby improving the accuracy of point cloud separation. Based on the points within the view frustum of the region of interest, noise processing is performed to obtain point cloud data of the same moving target. This includes: collecting the radar points of the identified moving target and the points projected onto the view frustum of the region of interest to obtain the initial point cloud data of the moving target; using the position and velocity information of the point cloud as input constraints, a filter based on spatial-Doppler two-dimensional density clustering is designed to clean the initial point cloud data to remove noise and obtain the point cloud data of the moving target. Constructing a cross-spatial information transmission network includes: constructing a graph structure based on the detected target; wherein the nodes in the graph structure correspond to semantic features in the image, and the edges in the graph structure represent the connection relationships established using spatial information from millimeter-wave point clouds; at the node end of the graph structure, embedding features are extracted by an encoder based on a multilayer perceptron; and the edges of the graph structure are constructed using two stages: local vehicle construction and cross-vehicle spatial construction.

2. The method of claim 1, wherein, The velocity-assisted selection scheme performs point cloud selection to distinguish between stationary objects and moving targets, and identifies radar points of moving targets, including: Establish a coordinate system based on the vehicle's direction of travel and determine the offset angle of the radar's installation direction; Based on the offset angle and the collected point cloud information, the directional angles and corresponding radial velocities of all objects in the field of view are obtained; Based on the relationship between the radial velocity of all objects and the coordinates of the corresponding target direction, a sine curve is fitted to determine the radar point of stationary objects. Based on the fitted sine curve, radar point clouds that deviate from the sine curve are identified as radar points corresponding to moving targets.

3. The method of claim 1, wherein, The lightweight separation scheme based on frustum projection separates points within the region of interest frustum according to the image frame and radar point cloud frame, including: Establish a cone propagation model; The image frame is input into the target detection algorithm for preprocessing to obtain heat map information, depth information, and bounding box information; Based on the heatmap information, depth information, and bounding box information, the bounding box and estimated depth of the object are determined, and the region of interest frustum is determined and the depth of the region of interest frustum is expanded according to the bounding box and estimated depth. Point cloud selection is performed based on the radar point cloud frame, the selected point cloud data is projected onto the bird's-eye view, and the points projected onto the view frustum of the region of interest are determined.

4. The method of claim 1, wherein, The step of determining co-view vehicles by performing cross-view correlation and information update on point cloud features selected from local vehicles and cooperative vehicles based on the cross-spatial information transmission network includes: In the cross-spatial information transmission network, edge embedding feature updates and node embedding feature updates are performed in both the local and cross-view perspectives. When updating edge embedding features, edges are divided into those connecting target nodes in the local vehicle view and those connecting target nodes across vehicle space, and edge embedding feature updates are performed to obtain the updated edge embedding feature set. When updating node embedding features, iterate the connection edges and node embedding features to obtain the updated node embedding features; The system constructs connections between all targets and learns the associations between all targets by weighted cross-view edges, thereby identifying the co-view vehicle.

5. The method of claim 1, wherein, The background-constrained alignment optimization algorithm optimizes the radar point cloud of the shared-view vehicle to achieve multi-vehicle cooperative perception, including: Clustering is used to filter out outliers caused by noise and multipath reflections from static background point clouds, retaining only background point cloud pairs containing roadside and building elements. The initial transformation and initial rotation matrix are obtained by performing preliminary iterations based on the processed common-view vehicle radar point cloud. The alignment optimization algorithm based on background constraints combines the point cloud pairs of the co-viewing vehicles and the radar point pairs of the background for continuous iteration, and calculates the transition matrix between the two viewpoints to improve the registration accuracy and realize multi-vehicle cooperative perception.

6. A multi-vehicle millimeter wave cooperative perception based safe driving system based on the method of any one of claims 1-5, characterized in that, include: The point cloud separation and selection module is used to acquire point cloud data of local vehicles and cooperative vehicles, and select and separate the point cloud data based on the speed-assisted selection and lightweight view cone separation scheme to obtain the point cloud data of the moving target. Cross-space network building module, used to build cross-space information transmission networks; The common-view vehicle detection module is used to perform cross-view correlation and information update on the point cloud features selected by local vehicles and cooperative vehicles based on the cross-spatial information transmission network, thereby determining the common-view vehicles. The background constraint alignment module is used to optimize the radar point cloud of the shared-view vehicle based on the alignment optimization algorithm of the background constraint, so as to realize multi-vehicle cooperative perception.

7. An electronic device, comprising: The electronic device includes: a memory, a processor, and a multi-vehicle millimeter-wave collaborative perception safety driving program stored in the memory and executable on the processor, the multi-vehicle millimeter-wave collaborative perception safety driving program being configured to implement the multi-vehicle millimeter-wave collaborative perception safety driving method as described in any one of claims 1 to 5.

Images