Multi-vehicle millimeter wave cooperative sensing safe driving method, system and equipment

Abstract

The invention relates to the technical field of automatic driving, and discloses a multi-vehicle millimeter wave cooperative sensing safe driving method, system and device. The method comprises the following steps: acquiring point cloud data of a local vehicle and a cooperative vehicle, and selecting and separating the point cloud data based on a speed-assisted selection and lightweight view cone separation scheme to obtain point cloud data of a moving target; constructing a cross-space information transmission network; on the basis of a cross-space information transmission network, cross-view correlation and information updating are carried out on the point cloud features selected by the local vehicle and the cooperative vehicle, so that a common-view vehicle is determined; and the radar point cloud of the common-view vehicle is optimized based on an alignment optimization algorithm of background constraint so as to realize multi-vehicle cooperative perception. According to the invention, the clear point cloud data of the moving target is efficiently obtained based on the speed-assisted point cloud selection scheme, and the radar point cloud data of the moving vehicle is accurately filtered and separated in combination with the lightweight view cone separation scheme. Common-view target vehicle detection is performed by adopting a cross-space information transmission network, and unique and accurate identification of common-view vehicles in cross-view angles is realized. And an alignment optimization algorithm based on background constraint is adopted to improve the precision and improve the accuracy of space calibration. A transfer matrix between vehicle visual angles is calculated through radar point cloud of a common-view target, and accurate, real-time and self-adaptive multi-vehicle cooperative sensing and safe driving are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and in particular, to a safe driving method, system and device for multi-vehicle millimeter-wave collaborative perception. Background Art

[0002] Autonomous driving is considered a technology with broad prospects for changing daily traffic. To ensure the reliability and safety of the driving system, various sensors are deployed in autonomous driving, such as lidar, cameras and radars, to sense and understand the surrounding traffic conditions. However, the single-vehicle perception system has problems of obstacle occlusion and limited perception range, which seriously affect its understanding of the surrounding situation and safe driving planning. For this reason, the multi-vehicle collaborative perception technology has opened a new mode, that is, multiple interconnected autonomous vehicles share sensor data to break through the individual perception limitations and improve the safety of autonomous driving. As Figure 1 shown, at a complex intersection, due to vehicle occlusion, the local vehicle cannot sense the vehicle. In contrast, by sharing sensor information with the collaborative vehicle, the local vehicle can successfully sense and detect the occluded vehicle, and through spatial calibration alignment, accurately locate the vehicle spatial distribution.

[0003] Similar to the Simultaneous Localization and Mapping (SLAM) system, cross-view sensor data alignment is the key to achieving multi-vehicle collaborative perception and safe driving. Among them, the local vehicle needs to obtain the spatial transformation matrix between the local vehicle and the collaborative vehicle, so as to perform cross-view calibration and alignment on the shared sensor data, and then transform it to the local view to improve the vehicle perception performance. However, compared with the single-node SLAM, designing a robust and efficient cross-view alignment scheme is more challenging for the multi-vehicle collaborative system, which needs to meet three requirements at the same time: First, to meet the needs of various safe driving tasks such as path planning, lane change and overtaking, the cross-view alignment scheme should achieve a calibration accuracy of decimeter level; second, the system processing delay needs to be completed within dozens of milliseconds to avoid catastrophic accidents; finally, the dynamic environment and perspective differences have a significant impact on the data alignment performance, so the perception scheme needs to have good adaptability in these challenging situations.

[0004] Traditional multi-vehicle cross-view alignment solutions mainly focus on: 1) relying on the precise pose of the vehicle to calibrate and align sensor data. However, the positioning average error measured by the Global Navigation Satellite System (GPS) is several meters, which is difficult to meet the accuracy requirements for many driving tasks in actual traffic scenarios. 2) Extracting representative features from dense lidar point clouds for cross-view alignment. However, sharing high-density lidar points under limited bandwidth poses a great challenge to latency. In addition, lidar modules with dense point clouds are expensive and vulnerable to the interference of soot and haze. 3) Performing spatial calibration by sharing landmark key points extracted from visual images. However, the extraction method based on ground image landmarks is easily affected by environmental illumination, texture, and vehicle occlusion, resulting in incorrect detection and extraction of objects and their geometries, thereby reducing the accuracy of cross-view alignment. Therefore, how to design an accurate, real-time, and adaptive cross-view alignment solution to improve multi-vehicle collaborative perception performance is still a key problem to be solved in the safe driving system. Summary of the Invention

[0005] The main object of the present invention is to provide a safe driving method, system, and device for multi-vehicle millimeter-wave collaborative perception, aiming to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides a safe driving method for multi-vehicle millimeter-wave collaborative perception, including:

[0007] Obtaining the point cloud data of the local vehicle and the collaborative vehicle, and selecting and separating the point cloud data based on the scheme of speed-assisted selection and lightweight frustum separation to obtain the point cloud data of the moving target;

[0008] Constructing a cross-space information transfer network;

[0009] Based on the cross-space information transfer network, performing cross-view association and information update on the point cloud features selected by the local vehicle and the collaborative vehicle, so as to determine the co-view vehicles;

[0010] Optimizing the radar point cloud of the co-view vehicles based on the alignment optimization algorithm with background constraints to achieve multi-vehicle collaborative perception.

[0011] In some embodiments, the obtaining the point cloud data of the local vehicle and the collaborative vehicle, and selecting and separating the point cloud data based on the scheme of speed-assisted selection and lightweight frustum separation to obtain the point cloud data of the moving target includes:

[0012] Respectively obtaining the point cloud data of the local vehicle and the collaborative vehicle; wherein, the point cloud data includes image frames and radar point cloud frames;

[0013] Select point clouds based on a speed-assisted selection scheme to distinguish stationary objects from moving targets and identify the radar points of the moving targets;

[0014] A lightweight separation scheme based on frustum projection separates the points within the frustum of the region of interest according to the image frame and the radar point cloud frame;

[0015] Based on the points within the frustum of the region of interest, noise processing is performed to obtain the point cloud data of the same moving target, thereby improving the accuracy of point cloud separation.

[0016] In some embodiments, the speed-assisted selection scheme for selecting point clouds to distinguish stationary objects from moving targets and identify the radar points of the moving targets includes:

[0017] Establish a coordinate system based on the vehicle driving direction and determine the offset angle of the radar installation direction:

[0018] Based on the offset angle and the collected point cloud information, obtain the direction angles of all objects in the field of view and the corresponding radial velocities;

[0019] According to the coordinate relationship between the radial velocities of all objects and the corresponding target directions, fit a sine curve to determine the radar points of stationary objects;

[0020] According to the fitted sine curve, identify the radar point cloud that deviates from the sine curve as the radar point corresponding to the moving target.

[0021] In some embodiments, the lightweight separation scheme based on frustum projection separates the points within the frustum of the region of interest according to the image frame and the radar point cloud frame, including:

[0022] Establish a frustum propagation model;

[0023] Input the image frame into a target detection algorithm for preprocessing to obtain heatmap information, depth information, and bounding box information;

[0024] Based on the heatmap information, depth information, and bounding box information, determine the bounding box and estimated depth of the object, and determine the frustum of the region of interest according to the bounding box and estimated depth and expand the depth of the frustum of the region of interest;

[0025] Perform point cloud selection according to the radar point cloud frame, project the selected point cloud data into a bird's-eye view, and determine the points projected into the frustum of the region of interest.

[0026] In some embodiments, the noise processing based on the points within the frustum of the region of interest to obtain the point cloud data of the same moving target includes:

[0027] Collect the radar points of the identified moving targets and the points projected into the cone of the region of interest to obtain the initial point cloud data of the moving targets;

[0028] Use the position information and velocity information of the point cloud as input constraints, and design a filter based on spatial-Doppler two-dimensional density clustering to clean the initial point cloud data to filter out noise and obtain the point cloud data of the moving targets.

[0029] In some embodiments, the construction of the cross-space information transfer network includes:

[0030] Construct a graph structure based on the detection targets; wherein, the nodes in the graph structure correspond to the semantic features in the image, and the edges in the graph structure represent the connection relationships established using the spatial information of the millimeter-wave point cloud;

[0031] At the node end of the graph structure, extract the embedded features through an encoder based on a multi-layer perceptron;

[0032] Adopt two stages of local vehicle construction and cross-vehicle space construction to construct the edges of the graph structure.

[0033] In some embodiments, based on the cross-space information transfer network, perform cross-view association and information update on the point cloud features selected by the local vehicle and the cooperative vehicle, so as to determine the co-view vehicles, including:

[0034] In the cross-space information transfer network, perform edge embedded feature update and node embedded feature update in the local view and the cross-view respectively;

[0035] When performing edge embedded feature update, divide the edges into the target nodes connecting the local vehicle view and the target nodes connecting the cross-vehicle space, and perform edge embedded feature update to obtain the updated edge embedded feature set;

[0036] When performing node embedded feature update, iteratively connect the edge and node embedded features to obtain the updated node embedded features;

[0037] Construct the connections between all targets, and learn the associations of all targets by weighting the cross-view edges, so as to determine the co-view vehicles.

[0038] In some embodiments, the alignment optimization algorithm based on background constraints optimizes the radar point cloud of the co-view vehicles to achieve multi-vehicle cooperative perception, including:

[0039] Use the clustering method to filter out the abnormal points caused by noise and multipath reflection for the static background point cloud, and retain the background point cloud pairs containing only roadside and buildings;

[0040] Perform preliminary iteration according to the processed radar point cloud of the co-view vehicles to obtain the initial transformation and the initial rotation matrix;

[0041] The alignment optimization algorithm based on background constraints combines the point cloud pairs of co-visible vehicles and the radar point pairs of the background for continuous iteration to calculate the transformation matrix between two perspectives, so as to improve the registration accuracy and achieve multi-vehicle collaborative perception.

[0042] In addition, to achieve the above object, the present invention also provides a safe driving system for multi-vehicle millimeter-wave collaborative perception, including:

[0043] A point cloud separation and selection module, configured to obtain the point cloud data of the local vehicle and the collaborative vehicle, and select and separate the point cloud data based on the scheme of speed-assisted selection and lightweight frustum separation to obtain the point cloud data of moving targets;

[0044] A cross-space network construction module, configured to construct a cross-space information transfer network;

[0045] A co-visible vehicle detection module, configured to perform cross-perspective association and information update on the point cloud features selected by the local vehicle and the collaborative vehicle based on the cross-space information transfer network, so as to determine co-visible vehicles;

[0046] A background constraint alignment module, configured to optimize the radar point cloud of the co-visible vehicle based on the alignment optimization algorithm based on background constraints to achieve multi-vehicle collaborative perception.

[0047] In addition, to achieve the above object, the present invention also provides an electronic device, the electronic device includes: a memory, a processor, and a safe driving program for multi-vehicle millimeter-wave collaborative perception stored on the memory and executable on the processor, and the safe driving program for multi-vehicle millimeter-wave collaborative perception is configured to implement the safe driving method for multi-vehicle millimeter-wave collaborative perception as described above.

[0048] The present invention provides a safe driving method for multi-vehicle millimeter-wave collaborative perception, including: acquiring point cloud data of a local vehicle and collaborative vehicles, selecting and separating the point cloud data based on a speed-assisted selection and lightweight frustum separation scheme to obtain point cloud data of moving targets; constructing a cross-space information transfer network; based on the cross-space information transfer network, performing cross-perspective association and information update on the point cloud features selected by the local vehicle and collaborative vehicles to determine co-visible vehicles; optimizing the radar point cloud of the co-visible vehicles based on a background-constrained alignment optimization algorithm to achieve multi-vehicle collaborative perception. Compared with traditional multi-vehicle collaborative perception schemes, the point cloud selection scheme based on speed assistance in the present invention efficiently acquires clear point cloud data of moving targets, and combines with the lightweight frustum separation scheme to accurately filter and separate the radar point cloud data of moving vehicles. A cross-space information transfer network is used for detecting co-visible target vehicles, realizing unique and accurate identification of co-visible vehicles in different perspectives. In addition, a background-constrained alignment optimization algorithm is adopted to improve the accuracy, thereby enhancing the accuracy of spatial calibration. Based on this, a transfer matrix between vehicle perspectives is calculated through the radar point cloud of co-visible targets, so as to achieve precise, real-time and adaptive multi-vehicle collaborative perception and safe driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of multi-vehicle collaborative perception related to the solution of the embodiment of the present invention;

[0050] Figure 2 It is a schematic diagram of the structure of an electronic device in the hardware operating environment related to the solution of the embodiment of the present invention;

[0051] Figure 3 It is a schematic flowchart of an embodiment of the safe driving method for multi-vehicle millimeter-wave collaborative perception of the present invention;

[0052] Figure 4 It is a schematic diagram of the technical architecture of the safe driving method for multi-vehicle millimeter-wave collaborative perception related to the solution of the embodiment of the present invention;

[0053] Figure 5 It is a schematic flowchart of a lightweight separation scheme based on frustum projection related to the solution of the embodiment of the present invention;

[0054] Figure 6 It is a schematic diagram of the cross-space information transfer network related to the solution of the embodiment of the present invention;

[0055] Figure 7 It is a schematic diagram of edge embedding feature update and node embedding feature update related to the solution of the embodiment of the present invention;

[0056] Figure 8 It is a structural block diagram of an embodiment of the safe driving system for multi-vehicle millimeter-wave collaborative perception of the present invention.

[0057] The realization, functional features, and advantages of the present invention will be further described in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Embodiments

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0059] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0060] In addition, the descriptions involving "first", "second", etc. in the present invention are only for descriptive purposes, and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0061] Refer to Figure 2 , Figure 2 is a schematic structural diagram of an electronic device for the hardware operating environment involved in the embodiment solution of the present invention.

[0062] As Figure 2As shown in the figure, the electronic device may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed Random Access Memory (RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0063] Those skilled in the art can understand that Figure 2 the structure shown in the figure does not constitute a limitation on the electronic device, and it may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0064] As Figure 2 shown, in the memory 1005 as a storage medium, there may be included an operating system, a network communication module, a user interface module, and a safe driving program for multi-vehicle millimeter-wave collaborative perception.

[0065] In Figure 2 the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention may be arranged in the electronic device. The electronic device calls the safe driving program for multi-vehicle millimeter-wave collaborative perception stored in the memory 1005 through the processor 1001 and executes the safe driving method for multi-vehicle millimeter-wave collaborative perception provided by the embodiments of the present invention.

[0066] The present invention provides a safe driving method, system and device for multi-vehicle millimeter-wave collaborative perception.

[0067] The embodiments of the present invention provide a safe driving method for multi-vehicle millimeter-wave collaborative perception. Referring to Figure 3 Figure 3 is a schematic flowchart of an embodiment of the safe driving method for multi-vehicle millimeter-wave collaborative perception of the present invention.

[0068] As Figure 3 ​As shown, the safe driving method for multi-vehicle millimeter-wave collaborative perception includes:

[0069] Step S100: Obtain the point cloud data of the local vehicle and the collaborative vehicle, and select and separate the point cloud data based on the speed-assisted selection and lightweight frustum separation scheme to obtain the point cloud data of the moving target.

[0070] Step S200: Construct a cross-space information transfer network.

[0071] Step S300: Based on the cross-space information transfer network, perform cross-view association and information update according to the point cloud features selected by the local vehicle and the collaborative vehicle, so as to determine the co-visible vehicles.

[0072] Step S400: Optimize the radar point cloud of the co-visible vehicles based on the alignment optimization algorithm with background constraints to achieve multi-vehicle collaborative perception.

[0073] It should be noted that the execution subject in this embodiment can be an electronic device, which can be a computer device with data processing functions, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, a computer device is taken as an example for illustration.

[0074] It can be understood that this embodiment is illustrated by taking the millimeter-wave radar point cloud alignment in the autonomous driving scenario as an example. As Figure 4 shown, the process of the safe driving method for multi-vehicle millimeter-wave collaborative perception includes the following three key steps: radar selection and separation, co-visible target vehicle detection, and background constraint alignment. The following is a detailed description in combination with specific steps.

[0075] In one embodiment, obtaining the point cloud data of the local vehicle and the collaborative vehicle, and selecting and separating the point cloud data based on the speed-assisted selection and lightweight frustum separation scheme to obtain the point cloud data of the moving target includes: respectively obtaining the point cloud data of the local vehicle and the collaborative vehicle; wherein, the point cloud data includes image frames and radar point cloud frames; performing point cloud selection based on the speed-assisted selection scheme to distinguish stationary objects and moving targets, and identifying the radar points of the moving targets; based on the lightweight separation scheme of frustum projection, separating the points within the frustum of the region of interest according to the image frames and radar point cloud frames; based on the points within the frustum of the region of interest, performing noise processing to obtain the point cloud data of the same moving target, thereby improving the accuracy of point cloud separation.

[0076] Specifically, radar selection and separation: Due to the existence of specular reflection and interference, simply using all point cloud data for view alignment will introduce serious matching errors and complex calculation problems. Therefore, in this embodiment, a point cloud selection scheme based on velocity assistance is first designed to efficiently obtain clear point cloud data of moving targets. Subsequently, different from the radar-vision method that needs to be retrained through a large number of labeled data sets and fine-grained feature extraction, this embodiment combines a monocular camera and designs a lightweight separation scheme based on frustum projection to accurately filter and separate the radar point cloud data of moving vehicles.

[0077] In one embodiment, a point cloud selection is performed based on the velocity-assisted selection scheme to distinguish stationary objects and moving targets and identify the radar points of moving targets, including: establishing a coordinate system based on the vehicle driving direction and determining the offset angle of the radar installation direction; obtaining the direction angles of all objects in the field of view and their corresponding radial velocities based on the offset angle and the collected point cloud information; fitting a sine curve according to the coordinate relationship between the radial velocities of all objects and the corresponding target directions to determine the radar points of stationary objects; and identifying the radar point cloud deviating from the sine curve as the radar points corresponding to moving targets.

[0078] Specifically, due to the existence of specular reflection and interference, simply using all point cloud data for view alignment will introduce serious matching errors and complex calculation problems. Therefore, in this embodiment, a selection scheme based on velocity assistance is first designed to efficiently obtain clear point cloud data of moving targets. The core lies in that the radar point cloud data of stationary backgrounds and moving targets present significantly different patterns in radial velocity, and the point cloud data of moving targets can be identified through these patterns.

[0079] Exemplarily, assume that in the plane X-Y coordinate system, the local vehicle moves along the Y-axis direction, and there is an offset angle θ between the installation direction of the vehicle-mounted radar and the vehicle driving direction. Since the Doppler radar can only capture the radial velocity, the velocity of all stationary objects can be expressed as v t , but the direction is opposite to the radar module. At this time, the velocity of the stationary target can be projected onto the X-axis and Y-axis directions to obtain its velocity components as v x =-v t sin(θ) and v y =-v t cos(θ). At the same time, for any point i (whose azimuth angle is ω i ), its radial velocity can be obtained as:

[0080]

[0081] According to the above formula, all stationary radar points will fall on a sine curve. In contrast, for a moving vehicle with a velocity angle of Δα, its radial velocity is:

[0082]

[0083] By comparing the above two equations, the radar points from the moving vehicle will deviate from the sine curve due to the additional terms caused by its motion and become outliers. These point clouds are significantly different from stationary targets due to the change in radial velocity caused by their motion, thus enabling effective discrimination between moving targets and background stationary objects.

[0084] In one embodiment, based on a lightweight separation scheme of frustum projection, separating the points within the frustum of the region of interest according to the image frame and the radar point cloud frame includes: establishing a frustum propagation model; inputting the image frame into an object detection algorithm for preprocessing to obtain heatmap information, depth information, and bounding box information; determining the bounding box and estimated depth of the object based on the heatmap information, depth information, and bounding box information, determining the frustum of the region of interest according to the bounding box and estimated depth, and expanding the depth of the frustum of the region of interest; performing point cloud selection according to the radar point cloud frame, projecting the selected point cloud data onto a bird's-eye view, and determining the points projected into the frustum of the region of interest.

[0085] Specifically, first, this embodiment designs a selection scheme based on velocity assistance to efficiently obtain clear point cloud data of moving targets. Subsequently, different from the radar-vision fusion method that needs to be retrained through a large labeled dataset and fine-grained feature extraction, this embodiment combines a monocular camera and designs a lightweight separation scheme based on frustum projection to accurately filter and separate the radar point cloud data of moving vehicles. The basic idea is that both the radar and the camera present a frustum propagation mode, and the two can work together to determine the distribution of point cloud data.

[0086] It should be noted that in computer vision and 3D reconstruction, the frustum propagation model is a model that utilizes sensor data such as cameras and radars to understand the spatial relationships in a scene. In this model, a 2D heatmap is a data representation form used for visualizing and processing data from sensors. The 2D heatmap is the three-dimensional space information represented on a two-dimensional plane. Specifically, the 2D heatmap is a two-dimensional matrix, where each element represents a pixel, and the value of the pixel represents a certain metric at that position, such as depth, probability, or other information related to the scene. By projecting the point cloud data in the three-dimensional space onto a two-dimensional plane, a 2D heatmap is generated. This process usually involves projecting the point cloud data according to the frustum (i.e., the field of view) of the camera.

[0087] Exemplarily, such asFigure 5 As shown, first, a frustum propagation model is created, and its 2D heatmap is defined. Among them, W and H respectively represent the width and height of the image, K is the camera intrinsic matrix, and L = [X, Y, Z] represents the position in the frustum. Then, by combining the bounding box of the object and its estimated depth d = ||L||, a region of interest (RoI) frustum can be determined, indicating the possible distribution space of the point cloud data. In this embodiment, to reduce the impact of inaccurate depth estimation, the depth of the RoI frustum is extended by Δd to include more adjacent point cloud data. In addition, measurement errors in the height dimension may lead to incorrect separation of the point cloud data. Therefore, as Figure 5 shown, in this embodiment, the point cloud data is projected into a bird's eye view (BEV), and the points projected into the RoI frustum F i are determined. These points (the points projected into the region of interest frustum) can be expressed as Finally, a series of point cloud data is collected which corresponds to this moving target vehicle i, as Figure 5 shown. It represents the point cloud data set obtained by visual target separation.

[0088] In this embodiment, a scheme based on velocity-assisted selection and lightweight frustum separation is adopted to extract reliable and clear point cloud data for alignment, effectively reducing the impact of millimeter-wave point cloud sparsity and vulnerability to interference.

[0089] In one embodiment, based on the points within the region of interest frustum, noise processing is performed to obtain the point cloud data of the same moving target, including: collecting the radar points of the identified moving target and the points projected into the region of interest frustum to obtain the initial point cloud data of the moving target; using the position information and velocity information of the point cloud as input constraints, designing a filter based on space-Doppler two-dimensional density clustering to clean the initial point cloud data to filter out noise and obtain the point cloud data of the moving target.

[0090] In this embodiment, to solve the problems of data set acquisition annotation and pre-training, a lightweight space-Doppler filtering method is designed to filter out noise. The core idea is that when the radar point cloud data comes from the same target, the point cloud will gather together to form a cluster-like distribution, while the point cloud contaminated by noise may show a random distribution. In addition, the radar point cloud from the same vehicle will show similarity in speed and position, making the point cloud show a cluster-like pattern in both the speed and position dimensions. Therefore, as Figure 5 shown, in this embodiment, the position information and velocity information of the point cloud are used as input constraints, and a filter based on the density clustering algorithm is adopted To clean the point cloud data. In this way, noise points can be effectively filtered out and the point cloud from the same target can be retained, thereby improving the accuracy of point cloud separation.

[0091] Specifically, as Figure 4 shown, after point cloud selection and separation, the cooperative vehicle shares and transmits the image-radar frame (image frame, radar point cloud frame) to the local vehicle for perception fusion. Due to the sparsity of the point cloud data, the cooperative vehicle can not only transmit the point cloud data of moving targets, but also transmit the point cloud data of the static background to improve accuracy. For the image frame, since the original data will occupy a large amount of bandwidth, in this embodiment, compression is performed, and only the feature map of the detected vehicle is shared. By using this method, the data transmission volume can be effectively reduced while retaining key perception information for efficient cross-view alignment and calibration.

[0092] In one embodiment, a cross-space information transfer network is constructed, including: constructing a graph structure based on the detection target; wherein, the nodes in the graph structure correspond to the semantic features in the image, and the edges in the graph structure represent the connection relationships established using the spatial information of the millimeter-wave point cloud; at the node end of the graph structure, embedding features are extracted through an encoder based on a multi-layer perceptron; two stages of constructing the local vehicle and the cross-vehicle space are adopted to construct the edges of the graph structure.

[0093] In one embodiment, based on the cross-space information transfer network, cross-view association and information update are performed on the point cloud features selected by the local vehicle and the cooperative vehicle, so as to determine the co-visible vehicles, including: in the cross-space information transfer network, edge embedding feature update and node embedding feature update are respectively performed in the local view and the cross-view; when performing edge embedding feature update, the edges are divided into those connecting the target nodes in the local vehicle view and those connecting the target nodes in the cross-vehicle space, and edge embedding feature update is performed to obtain the updated edge embedding feature set; when performing node embedding feature update, the connected edges and node embedding features are iteratively connected to obtain the updated node embedding features; connections between all targets are constructed, and the associations of all targets are learned by weighting the cross-view edges, thereby determining the co-visible vehicles.

[0094] It can be understood that, as Figure 4As shown in the figure, co-view target vehicle detection: To achieve the unique and accurate identification of co-view vehicles across perspectives, in this embodiment, a cross-space information transfer network is designed to learn and understand the association characteristics between vehicles. Specifically, in this embodiment, a graph structure association based on the detection target is first constructed, where the nodes correspond to the semantic features in the image, and the edges represent the connection relationships established using the spatial information of the millimeter-wave point cloud. Further, to construct cross-perspective associations, a cross-space information transfer network is designed to update the nodes and edges in the graph, so as to understand the association characteristics between target vehicles. Finally, based on a predefined multi-label classifier, the association weights are measured and co-view vehicles are identified.

[0095] Specifically, to achieve the unique and accurate identification of co-view vehicles across perspectives, in this embodiment, a cross-space information transfer network is designed to construct and understand the association characteristics between vehicles. As Figure 6 shown, this design first constructs a graph structure G=(V, E) based on the detection target, where the nodes v i ∈V correspond to the semantic features in the image, and the edges e i,j ∈E represent the connection relationships established using the spatial information of the millimeter-wave point cloud. Specifically, at the node end, the embedded features are directly extracted through an encoder ε v based on a multi-layer perceptron (MLP), where the initial node embedded feature of node i is As for the construction of the edges, due to different reference systems, in this embodiment, the design of the edges is divided into two stages: local vehicle construction and cross-vehicle space construction.

[0096] Exemplarily, in the local vehicle construction stage, the nodes are connected through relevant distances and angles:

[0097]

[0098] where d i,j represents the distance between nodes i and j, d max is the maximum distance of all edges, θ i and θ j are the angles in the sensor perspective, respectively, and θ is the angle of the edge e i,j . Then, exemplarily, in the cross-vehicle space construction stage, the nodes are given an initial value of 0. Subsequently, the edges are input into an encoder ε e based on a multi-layer perceptron MLP to obtain the initial edge embedded feature During the embedding layer processing, the image features of each vehicle are encoded into 1×32 node embedded features, while the corresponding radar point cloud data is encoded into 1×16 edge embedded features. Finally, the connections between all targets are constructed, and the associations of all targets are learned by weighting the cross-perspective edges, so as to determine the co-view vehicles.

[0099] It should be noted that, different from the continuous frame system in a single node, the nodes and edges in the cooperative vehicle system cannot directly cross two independent spaces to update the information of nodes and edges. To solve this problem, as Figure 7 shown, this embodiment designs a cross-space information transfer network for information update. In this cross-space information transfer network, edge embedding feature update and node embedding feature update are respectively performed in the local perspective and the cross-perspective. In the edge embedding feature update stage, each edge e i,j ∈E connects two node embedding features, so they can be respectively represented as the source node embedding feature and the target node embedding feature. Therefore, the update of the edge embedding feature needs to consider the embedding features of the source node, the target node, and the edge simultaneously. Since the nodes between the local vehicle and the cooperative vehicle are in different spaces, the edges can be divided into two categories: one category represents connecting the target nodes in the local vehicle perspective, and the other category represents connecting the target nodes in the cross-vehicle space, and then perform edge embedding feature update on them.

[0100] Specifically, assume that the cross-space information transfer network performs N information transfer steps in the graph. Therefore, in the local perspective, the edge embedding feature update of a group of nodes in the nth iteration can be expressed as:

[0101]

[0102] For the cross-perspective edges, the edge embedding can be updated by performing similar operations, where the updated edge embedding feature in the nth iteration is represented as Therefore, by traversing all the edges in the graph, the updated edge embedding feature set is finally obtained In the node embedding feature update stage, each node v i ∈V is connected by a group of edges, so the update of the node embedding feature needs to iteratively connect the edge and the node embedding feature itself. It can be understood that, similar to the edge embedding feature update, the update operation can be performed in both the local perspective and the cross-perspective.

[0103] Specifically, first update the edge embedding feature in the local perspective The update formula can be expressed as:

[0104]

[0105] Among them, represents the previous node embedding feature, while is the current edge embedding feature. According to this update rule, the cross-perspective edge embedding feature Finally, the node embedding feature can be updated and connect the two spaces by combining these two types of edge embedding features. The update formula is:

[0106]

[0107] After N message passing iterations, all nodes in the graph have shared their information with each other. The ultimate goal is to find the co-visible targets from a series of edges connecting two independent perspective spaces and identify the co-visible targets. To this end, all the output edge embedding features are input into a classifier based on a multi-layer perceptron (MLP) which predicts the association probability of each edge e i,j ∈ E as In this classifier, by comparing with the threshold η, it can be determined whether the edge connects the same target from two different perspectives. The output of the classifier is:

[0108]

[0109] To train the model, the loss function is defined as weighted binary cross-entropy for the co-visible vehicle prediction in the last information passing iteration n. The loss function can be expressed as:

[0110]

[0111] where ω is a weighting factor used to compensate for the high imbalance between co-visible vehicle edges and other edges, and it is defined as:

[0112]

[0113] where 1(x) represents an indicator function that indicates whether a certain condition holds. When the condition x is satisfied, the return value is 1, otherwise the return value is 0. Therefore, this embodiment tends to assign larger weights to the edge classes that connect the same co-visible vehicles.

[0114] In one embodiment, an alignment optimization algorithm based on background constraints optimizes the radar point cloud of the co-visible vehicles to achieve multi-vehicle collaborative perception, including: filtering out the abnormal points caused by noise and multipath reflection from the static background point cloud by clustering, and retaining the background point cloud pairs containing only roadside and buildings; performing a preliminary iteration on the processed co-visible vehicle radar point cloud to obtain an initial transformation and an initial rotation matrix; respectively obtaining the background point clouds of the local vehicle and the collaborative vehicle and performing feature extraction to obtain the background radar point pairs; and performing continuous iterations on the point cloud pairs of the co-visible vehicles and the background radar point pairs by the alignment optimization algorithm based on background constraints to calculate the transformation matrix between the two perspectives, so as to improve the registration accuracy and achieve multi-vehicle collaborative perception.

[0115] Specifically, as Figure 4As shown, background constraint alignment: To avoid the inconsistency of the point cloud coverage space due to different perspectives, this embodiment further designs an alignment optimization algorithm based on background constraints to improve the accuracy. Specifically, since the point cloud data mapped by both sides of the road and other static objects has significant structural features, these features can be used to constrain the alignment process, thereby improving the accuracy of spatial calibration.

[0116] It should be noted that finally, to avoid the inconsistency of the point cloud coverage space due to different perspectives, this embodiment further designs an alignment optimization algorithm based on background constraints to improve the accuracy. The core lies in that the radar point clouds from the roadside and other static objects present unique structural features, which can be used to constrain the registration. However, typical road types, such as straight roads and intersections, have relatively ideal axial symmetry and rotational symmetry, which may lead to registration misguidance and thus bias towards the opposite direction or the rotationally symmetric direction. Therefore, directly combining the radar point pairs of the co-viewing vehicles and the background may have a negative impact on the registration. To solve this problem, this embodiment first uses clustering to filter out the abnormal points caused by noise and multipath reflection, and retains the point clusters of the roadside and buildings. Then, instead of directly performing the iterative closest point (ICP) processing by combining the radar point clouds, it first performs iteration only on the radar point pairs of the co-viewing vehicles to obtain the initial transformation T 0 . The iteration period is defined as min{L, N(D max )}, where L represents half of the total number of iterations of the radar point pairs of the co-viewing vehicles and N(D max ) is the number of iterations when the convergence distance threshold D max is reached. Since the spatial distribution of the co-viewing vehicles presents a good geometric structure, an approximate rotation matrix R 0 can be obtained, thereby avoiding the registration error caused by roadside symmetry. Finally, by combining the point pairs of the co-viewing vehicles and and and and the background point cloud continue to iterate. During this process, this embodiment designs the following optimization rules to further improve the registration accuracy:

[0117]

[0118] where E n and E g respectively represent the estimation errors caused by the co-viewing point cloud pairs and the background point cloud. and is the weight assigned to them. Specifically, the core idea is that if the vehicle detection obtains a high score through CenterNet (a deep learning-based object detection algorithm), a higher weight is assigned to the point cloud pairs of co-visible vehicles. Conversely, if only a small number of co-visible vehicles N are detected v , more reliance will be placed on the background radar point cloud, and a higher weight will be given to the background radar point cloud. In the above manner, this embodiment can accurately calculate the transfer matrix between two perspectives, achieving precise, real-time, and adaptive multi-vehicle collaborative perception and safe driving.

[0119] It can be understood that generally, spatial transformation can convert the point cloud data in one perspective to another perspective. The following steps can be used to calculate the transfer matrix between two perspectives. Select a reference perspective: First, determine a reference perspective, usually choosing one of the perspectives as a reference; Feature extraction: Extract features from the point cloud data of the two perspectives. These features can be the geometric features, color information of the point cloud, or high-level features extracted through a deep learning model; Alignment strategy: Determine the alignment strategy. Here, a feature-based registration algorithm can be used, including but not limited to the Iterative Closest Point (ICP) algorithm, or a deep learning-based registration method.

[0120] Exemplarily, Feature extraction: For the point cloud data of each perspective, extract key features; Initial registration: Use an initial registration algorithm to obtain the initial alignment between the point clouds of the two perspectives, which can be by selecting corresponding point pairs or using a feature-based registration method; Iterative optimization: Use an iterative algorithm to optimize the registration result; Output the final transfer matrix: Once the iterative process converges, the final transformation matrix T is the transfer matrix between the two perspectives.

[0121] This embodiment designs the first method based on millimeter-wave radar point cloud alignment, which can achieve precise and real-time multi-vehicle collaborative perception and safe driving using sparse point cloud data in challenging autonomous driving scenarios. In addition, this embodiment designs a scheme based on speed-assisted selection and lightweight frustum separation for extracting reliable and clear point cloud data for alignment, effectively reducing the impact of the sparsity and susceptibility to interference of millimeter-wave point clouds. Finally, this embodiment proposes a cross-space information transfer network, which can construct and understand the target association relationship, and designs a background-constrained alignment optimization method to address the sparsity problem, thereby achieving reduced latency while improving the alignment accuracy. Compared with traditional multi-vehicle collaborative perception schemes, the safe driving mechanism of multi-vehicle millimeter-wave collaborative perception proposed in this embodiment achieves precise, real-time, and adaptive multi-vehicle collaborative perception and safe driving.

[0122] The method described in this embodiment can accurately align and locate the target vehicles captured in images from different perspectives using a millimeter-wave radar. The core idea of the method described in this embodiment is that based on the fine-grained spatial information provided by the radar, such as position, speed, and angle, the local vehicle and cooperative vehicles can construct a unique cross-perspective association for all target vehicles under different perspectives. In this way, the local vehicle and cooperative vehicles can accurately capture and understand the connection topology between each target vehicle and adjacent vehicles in the environment, the local role of each target vehicle in the structure, and its global position in the entire perspective. Furthermore, the local vehicle and cooperative vehicles can uniquely identify each target and find the vehicles that are commonly visible. Finally, based on the millimeter-wave point cloud corresponding to the common vehicles of the local vehicle and cooperative vehicles across perspectives, the transformation matrix between the two perspectives can be accurately calculated, thereby achieving precise, real-time, and adaptive multi-vehicle cooperative perception.

[0123] This embodiment provides a safe driving method for multi-vehicle millimeter-wave cooperative perception, including: obtaining the point cloud data of the local vehicle and cooperative vehicles, selecting and separating the point cloud data based on a speed-assisted selection and lightweight frustum separation scheme to obtain the point cloud data of moving targets; constructing a cross-space information transfer network; based on the cross-space information transfer network, performing cross-perspective association and information update on the point cloud features selected by the local vehicle and cooperative vehicles to determine the co-visible vehicles; optimizing the radar point cloud of the co-visible vehicles based on an alignment optimization algorithm with background constraints to achieve multi-vehicle cooperative perception. Compared with traditional multi-vehicle cooperative perception schemes, the method described in this embodiment efficiently obtains clear point cloud data of moving targets based on a speed-assisted point cloud selection scheme, and combines a lightweight frustum separation scheme to accurately filter and separate the radar point cloud data of moving vehicles. A cross-space information transfer network is used for detecting co-visible target vehicles, realizing the unique and accurate identification of co-visible vehicles across perspectives. In addition, an alignment optimization algorithm based on background constraints is used to improve the accuracy, thereby enhancing the accuracy of spatial calibration. Based on this, the transformation matrix between vehicle perspectives is calculated through the radar point cloud of co-visible targets, thereby achieving precise, real-time, and adaptive multi-vehicle cooperative perception and safe driving.

[0124] In addition, an embodiment of the present invention also proposes a storage medium, on which a safe driving program for multi-vehicle millimeter-wave cooperative perception is stored. When the safe driving program for multi-vehicle millimeter-wave cooperative perception is executed by a processor, the steps of the safe driving method for multi-vehicle millimeter-wave cooperative perception described above are implemented.

[0125] Refer to Figure 8 , Figure 8 which is a structural block diagram of an embodiment of the safe driving system for multi-vehicle millimeter-wave cooperative perception of the present invention.

[0126] As Figure 8As shown in the figure, the multi-vehicle millimeter-wave collaborative perception-based safe driving system includes:

[0127] A point cloud separation and selection module 10, configured to obtain the point cloud data of the local vehicle and the collaborative vehicle, and perform selection and separation on the point cloud data based on a speed-assisted selection and lightweight frustum separation scheme to obtain the point cloud data of moving targets;

[0128] A cross-space network construction module 20, configured to construct a cross-space information transfer network;

[0129] A co-view vehicle detection module 30, configured to perform cross-view association and information update on the point cloud features selected by the local vehicle and the collaborative vehicle based on the cross-space information transfer network, so as to determine co-view vehicles;

[0130] A background constraint alignment module 40, configured to optimize the radar point cloud of the co-view vehicles based on a background constraint-based alignment optimization algorithm to achieve multi-vehicle collaborative perception.

[0131] This embodiment designs the first system based on millimeter-wave radar point cloud alignment, which can utilize sparse point cloud data to achieve accurate and real-time multi-vehicle collaborative perception and safe driving in challenging autonomous driving scenarios. In addition, this embodiment designs a speed-assisted selection and lightweight frustum separation scheme for extracting reliable and clear point cloud data for alignment, effectively reducing the impact of millimeter-wave point cloud sparsity and susceptibility to interference. Finally, this embodiment proposes a cross-space information transfer network, which can construct and understand target association relationships, and designs a background constraint-based alignment optimization method to address the sparsity problem, thereby improving the alignment accuracy while reducing latency. Compared with traditional multi-vehicle collaborative perception schemes, the safe driving mechanism of multi-vehicle millimeter-wave collaborative perception proposed in this embodiment achieves accurate, real-time, and adaptive multi-vehicle collaborative perception and safe driving.

[0132] This embodiment provides a multi-vehicle millimeter-wave collaborative perception-based safe driving system. The system in this embodiment efficiently obtains clear point cloud data of moving targets based on a speed-assisted point cloud selection scheme, combines a lightweight frustum separation scheme, and accurately filters and separates the radar point cloud data of moving vehicles. A cross-space information transfer network is used for co-view target vehicle detection to achieve unique and accurate identification of co-view vehicles in cross-views. In addition, a background constraint-based alignment optimization algorithm is used to improve the accuracy, thereby enhancing the accuracy of spatial calibration. Based on this, the transfer matrix between vehicle perspectives is calculated through the radar point cloud of co-view targets, so as to achieve accurate, real-time, and adaptive multi-vehicle collaborative perception and safe driving.

[0133] It should be noted that for the technical details not described in detail in the embodiments of the multi-vehicle millimeter-wave collaborative perception-based safe driving system, reference can be made to the safe driving method applied to the multi-vehicle millimeter-wave collaborative perception as described above in any embodiment of the present invention, which will not be elaborated here.

[0134] It should be understood that the above is only an example for illustration and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can set according to needs, and the present invention does not make any restrictions in this regard.

[0135] It should be noted that the above-described work process is only illustrative and does not limit the protection scope of the present invention. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and no restrictions are made here.

[0136] In addition, it should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or system including that element.

[0137] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0138] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0139] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A safe driving method for multi-vehicle millimeter wave collaborative sensing, characterized in that: include: Acquire point cloud data of the local vehicle and the cooperative vehicle, select and separate the point cloud data based on the speed-assisted selection and lightweight cone separation scheme, and obtain point cloud data of the moving target; Build a cross-space information transmission network; Based on the cross-space information transmission network, cross-view correlation and information update are performed on the point cloud features selected by the local vehicle and the cooperative vehicle, so as to determine the common view vehicle; An alignment optimization algorithm based on background constraints optimizes the radar point cloud of the common-view vehicle to achieve multi-vehicle collaborative perception.

2. The method according to claim 1, characterized in that The step of acquiring the point cloud data of the local vehicle and the cooperative vehicle, selecting and separating the point cloud data based on the speed-assisted selection and lightweight cone separation scheme, and obtaining the point cloud data of the moving target includes: Acquire point cloud data of the local vehicle and the cooperative vehicle respectively; wherein the point cloud data includes image frames and radar point cloud frames; Point cloud selection based on speed-assisted selection scheme to distinguish between stationary objects and moving targets and identify radar points of moving targets; A lightweight separation scheme based on cone projection is used to separate points within the cone of interest area according to the image frame and the radar point cloud frame; Based on the points within the visual cone of the region of interest, noise processing is performed to obtain point cloud data of the same moving target, thereby improving the accuracy of point cloud separation.

3. The method according to claim 2, characterized in that The speed-assisted selection scheme performs point cloud selection to distinguish between stationary objects and moving targets and identify radar points of moving targets, including: Establish a coordinate system based on the vehicle's driving direction and determine the offset angle of the radar installation direction; Obtaining direction angles and corresponding radial velocities of all objects in the field of view based on the offset angle and the collected point cloud information; According to the coordinate relationship between the radial velocity of all objects and the corresponding target direction, a sine curve is fitted to determine the radar point of the stationary object; According to the fitted sine curve, the radar point cloud deviating from the sine curve is identified as the radar point corresponding to the moving target.

4. The method according to claim 2, characterized in that The lightweight separation scheme based on cone projection separates points within the cone of interest area according to the image frame and the radar point cloud frame, including: Establish a cone propagation model; Input the image frame into the target detection algorithm for preprocessing to obtain thermal map information, depth information and bounding box information; Determine a bounding box and an estimated depth of the object based on the heat map information, the depth information and the bounding box information, determine a region of interest cone according to the bounding box and the estimated depth, and expand the depth of the region of interest cone; Point cloud selection is performed according to the radar point cloud frame, the selected point cloud data is projected into a bird's-eye view, and points projected into the visual cone of the area of ​​interest are determined.

5. The method according to claim 2, characterized in that The step of performing noise processing based on the points within the visual cone of the region of interest to obtain point cloud data of the same moving target includes: Collecting radar points of the identified moving target and points projected into the visual cone of the region of interest to obtain initial point cloud data of the moving target; Taking the position information and speed information of the point cloud as input constraints, a filter based on space-Doppler two-dimensional density clustering is designed to clean up the initial point cloud data to filter out noise and obtain point cloud data of the moving target.

6. The method according to claim 1, characterized in that The constructing of a cross-space information transmission network includes: Constructing a graph structure based on the detection target; wherein the nodes in the graph structure correspond to the semantic features in the image, and the edges in the graph structure represent the connection relationship established using the spatial information of the millimeter wave point cloud; At the node end of the graph structure, embedding features are extracted through an encoder based on a multi-layer perceptron; Two phases, local vehicle construction and cross-vehicle space construction, are adopted to construct the edges of the graph structure.

7. The method according to claim 6, characterized in that Based on the cross-space information transmission network, cross-view correlation and information update of point cloud features selected by the local vehicle and the cooperative vehicle are performed to determine the common view vehicle, including: In the cross-spatial information transfer network, edge embedding feature update and node embedding feature update are performed in local view and cross-view, respectively; When updating edge embedding features, the edges are divided into target nodes connected in the local vehicle perspective and target nodes connected across vehicle spaces, and edge embedding feature updates are performed to obtain an updated edge embedding feature set; When the node embedding feature is updated, iteratively connect the edge and node embedding features to obtain the updated node embedding feature; Connections between all objects are constructed and the associations of all objects are learned by weighting the edges across views to determine the common viewing vehicles.

8. The method according to claim 1, characterized in that The background constraint-based alignment optimization algorithm optimizes the radar point cloud of the common-view vehicle to achieve multi-vehicle collaborative perception, including: For the static background point cloud, the clustering method is used to filter out the abnormal points caused by noise and multipath reflection, and only the background point cloud pairs containing roadsides and buildings are retained; Performing preliminary iterations based on the processed common view vehicle radar point cloud to obtain an initial transformation and an initial rotation matrix; The background constraint-based alignment optimization algorithm combines the point cloud pairs of common-view vehicles and the radar point pairs of the background for continuous iteration and calculates the transfer matrix between the two perspectives to improve the registration accuracy and achieve multi-vehicle collaborative perception.

9. A multi-vehicle millimeter wave cooperative sensing safety driving system, characterized in that: include: A point cloud separation and selection module is used to obtain point cloud data of local vehicles and cooperative vehicles, select and separate the point cloud data based on speed-assisted selection and lightweight cone separation to obtain point cloud data of moving targets; A cross-space network construction module, used to build a cross-space information transmission network; A common view vehicle detection module, configured to perform cross-view correlation and information update on point cloud features selected by local vehicles and cooperative vehicles based on the cross-space information transmission network, thereby determining common view vehicles; The background constraint alignment module is used to optimize the radar point cloud of the common-view vehicle based on the background constraint alignment optimization algorithm to achieve multi-vehicle collaborative perception.

10. An electronic device, characterized in that: The electronic device includes: a memory, a processor, and a multi-vehicle millimeter-wave collaborative sensing safe driving program stored in the memory and executable on the processor, wherein the multi-vehicle millimeter-wave collaborative sensing safe driving program is configured to implement the multi-vehicle millimeter-wave collaborative sensing safe driving method as described in any one of claims 1 to 8.

Images