Target detection method for efficient vehicle-to-vehicle cooperation

Abstract

The invention relates to a target detection method for efficient vehicle-to-vehicle cooperation. The method comprises the following steps: collecting point cloud data of a vehicle and a networked vehicle in a scene; performing point-column fusion and feature extraction based on the point cloud data to obtain a pseudo image of the point cloud data; performing multi-scale feature extraction and dimension reduction processing on the basis of the pseudo image of the point cloud data to obtain lightweight sensing features; based on the perceptual features, generating a feature confidence map through depth supervision of feature tags, the feature confidence map being updated along with the conversion of a communication round; on the basis of the feature confidence map, collaborative sharing features are selected, and feature fusion between vehicles is carried out in combination with the sensing features to obtain fused sensing features; and performing target detection on the fused sensing features by using a pre-trained target detection model to obtain a target detection result. Compared with the prior art, the method has the advantages of improving the task sensing efficiency and the like.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving collaborative perception technology, and in particular to a target detection method for efficient vehicle-to-vehicle collaboration. Background Art

[0002] Multi-vehicle collaborative perception technology is an important research direction in the field of autonomous driving. This technology achieves comprehensive perception of the environment by sharing perception features between multiple vehicles. Collaborative perception technology fundamentally overcomes the inevitable limitations of single-vehicle perception, such as field of view occlusion and long-distance detection. Some current collaborative perception methods share all perception features with other vehicles, which not only brings information redundancy, but also increases the communication bandwidth. Therefore, how to efficiently and effectively share perception features between vehicles still faces technical challenges and limitations.

[0003] At present, multi-vehicle collaborative perception technology mainly relies on the fusion of on-board sensor data and information exchange between vehicles. Different vehicles share real-time data through the vehicle-to-everything (V2X) protocol, allowing each vehicle to use real-time perception data from other vehicles for perception compensation. Although there are some V2X-based collaborative perception systems, the large amount of information shared between vehicles leads to large bandwidth consumption and communication delay. Therefore, there are still some challenges in how to balance communication bandwidth consumption and perception accuracy, and how to effectively select features with rich target information for sharing.

[0004] At present, some multi-vehicle collaborative perception methods share complementary perception features between vehicles to achieve a balance between perception accuracy and communication bandwidth. However, these methods generate perception features through the target detection head and randomly select the largest k perception feature values ​​for sharing. When the k value is too large, it leads to redundant perception features. When the k value is too small, it leads to insufficient perception features. When facing the problem of target detection under vehicle-to-vehicle collaboration, it is necessary to optimize the perception features to obtain target detection efficiency. Summary of the invention

[0005] The purpose of the present invention is to provide a target detection method for efficient vehicle-to-vehicle collaboration that improves the efficiency of perception tasks.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A target detection method for efficient vehicle-to-vehicle collaboration includes the following steps:

[0008] Collect point cloud data of the scene where the vehicle and the connected vehicle are located;

[0009] Based on the point cloud data, point column fusion and feature extraction are performed to obtain a pseudo image of the point cloud data;

[0010] Based on the pseudo image of the point cloud data, multi-scale feature extraction and dimensionality reduction processing are performed to obtain lightweight perceptual features;

[0011] Based on the perceptual features, generating a feature confidence map through deep supervision of feature labels, wherein the feature confidence map is updated as the communication rounds change;

[0012] Based on the feature confidence map, selecting collaborative shared features, and combining the perception features to perform feature fusion between vehicles to obtain fused perception features;

[0013] A pre-trained target detection model is used to perform target detection on the fused perception features to obtain a target detection result.

[0014] Furthermore, the step of obtaining a pseudo image of point cloud data includes:

[0015] Based on the point cloud data, projection is performed with the own vehicle as the center to obtain projected point cloud data;

[0016] Discretize the projected point cloud data into a network evenly distributed on the xy plane, and create a point column for the projected point cloud data located in the same grid, finally generate P non-empty point columns, and further obtain a tensor with a characteristic volume of (D, P, N), where D is the point cloud dimension and N is the number of point clouds in each point column;

[0017] Based on the tensor, a PointNet network is used to extract features to obtain a pseudo image of the point cloud data.

[0018] Furthermore, the projected point cloud data is expressed as:

[0019]

[0020] In the formula, is the coordinate of the projected point cloud data, R and T represent the rotation matrix and translation variable of the point cloud data projected to the coordinate system centered on the vehicle. is the coordinate of the point cloud data before projection.

[0021] Furthermore, the step of obtaining lightweight perceptual features includes:

[0022] A feature pyramid network is used to extract multi-scale features from the pseudo image;

[0023] The multi-scale features are spliced ​​to obtain the spliced ​​feature F c ;

[0024] Based on the splicing feature F c , convolution is used to compress features and obtain lightweight perceptual features F.

[0025] Furthermore, the expression for deeply supervising the feature confidence map generation process through the feature label is:

[0026]

[0027] In the formula, is the deep supervision loss, is the cross entropy loss function, Ψ fuse It is a multi-head self-attention, which is used to fuse the feature confidence maps in multiple rounds of communication. GT i (k) is the true feature label of the i-th vehicle in the k-th round of communication, and the feature label is contained in the point cloud data.

[0028] Furthermore, the expression of the feature confidence map generated in each communication round is:

[0029]

[0030] In the formula, is the feature confidence map generated by the i-th vehicle in the k-th round of communication, Θ generator is the target detection head, F i (k) is the perceptual feature of the i-th vehicle in the K-th round of communication, H and W represent the length and width of the image, respectively.

[0031] Furthermore, the step of obtaining the fused perception features includes:

[0032] Get multiple 3D bounding box corner coordinates [x corner_i ,y corner_i ,z corner_i ], where i represents the number of corner points;

[0033] Convert multiple corner point coordinates into pseudo image pixel coordinates to obtain an AC graph;

[0034] Based on the feature confidence map and the communication map representation, collaborative shared features are selected, and combined with the perception features, feature fusion between vehicles is performed to obtain fused perception features.

[0035] Furthermore, the step of obtaining the communication diagram includes:

[0036] The coordinate system centered on the vehicle itself is used as a reference to obtain the maximum and minimum values ​​of multiple corner point coordinates on the xy plane, and obtain the coordinates [x min ,y min ,x max ,y max ];

[0037] The coordinates [x min ,y min ,x max ,y max ] is converted based on the pixel coordinates of the pseudo image to obtain the coordinates of the corresponding position on the pseudo image, wherein the conversion expression is:

[0038]

[0039] In the formula, [x min_pi ,y min_pi ,x max_pi ,y max_pi ] is the coordinate of the corresponding position on the pseudo image, r x , x y They represent the detection range of the laser radar in the x and y directions, respectively, and H and W represent the length and width of the pseudo image, respectively;

[0040] Given a tensor T with zero values ​​and the same size as the pseudo image, according to the coordinates [x min_pi ,y min_pi ,x max_pi ,y max_pi ] Set the coordinate range of the tensor T to x max_pi -x min_pi With y max_pi -y min_pi The value in the coordinate range is set to 1 to obtain an AC diagram, in which the process of setting the coordinate range value to 1 is expressed as:

[0041]

[0042] In the formula, The subscript HW represents a coordinate index.

[0043] Furthermore, the step of obtaining the fused perception features includes:

[0044] Based on the feature confidence map, communication map and perception features of each connected vehicle, a collaborative shared feature is selected, wherein the selection expression of the collaborative shared feature is:

[0045]

[0046] In the formula, represents the collaborative sharing feature sent by the jth vehicle to the ith vehicle in the kth round of communication, F i (k) is the perception feature of the i-th vehicle in the K-th round of communication, is the communication graph of the i-th vehicle in the K-th round of communication;

[0047] Use multi-head attention to transfer the shared features and the perception feature F of the vehicle itself i (k) Perform feature fusion to obtain the fused perceptual features, which are expressed as:

[0048]

[0049] In the formula, represents the fused perceptual features in the kth round of communication, MHSA is the multi-head self-attention operation, is the feature confidence map generated by the i-th vehicle in the k-th round of communication.

[0050] Furthermore, the loss function used by the target detection model is:

[0051]

[0052] Among them, Δb includes the following components:

[0053]

[0054] Δθ=sin(θ gt -θ a )

[0055] In the formula, is the total loss, is the category loss, α, β, γ are weights, is the positioning loss, p(x i ) is the label of category i, q(x i ) is the probability of being predicted as category i, is the depth supervision loss, x, y, z represent the center coordinates of the 3D bounding box, w, h, l represent the length, width, and height dimensions of the 3D bounding box, θ represents the direction angle of the 3D bounding box, Δb is the offset of the variable, gt is the true label, d is the diagonal distance between the length and width of the 3D bounding box, h represents the height of the prior 3D bounding box, and a is the predicted value.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] (1) The present invention performs multiple rounds of fusion and feature supervision on the process of generating feature confidence maps, which can better optimize the point-column fusion, feature extraction and pseudo-image multi-scale feature extraction processes, so that the generated feature confidence maps can better reflect the feature foreground and background. By selecting the area with the richest information in the perception feature data based on the pseudo-image pixel coordinates in the shared feature selection process, the information missing in the original perception feature can be supplemented, so that the amount of shared information is greatly reduced, and the efficiency of the perception task is improved.

[0058] (2) Through multiple rounds of fusion of feature confidence maps and deep supervision of labels, the method of the present invention can better resist the noise and interference generated by sensors and models, thereby improving the robustness of the overall detection process.

[0059] (3) The present invention selects shared features based on pseudo-image pixel coordinates. The shared features include targets that are occluded in the original perception features. The target foreground information that is missing from the own vehicle is selected for sharing, which greatly reduces the amount of shared information and saves communication costs. The sparse communication graph depends on a binary selection matrix. The binary matrix selects different perception areas according to the changing pseudo-image pixel coordinates, thereby dynamically allocating the communication budget of each networked vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0061] Figure 2 This is a visualization diagram of the target frame coordinate mapping of the present invention. DETAILED DESCRIPTION

[0062] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0063] This embodiment provides a target detection method for efficient vehicle-to-vehicle collaboration. Figure 1 As shown, the method comprises the following steps:

[0064] 101: Collect raw point cloud data.

[0065] The ego vehicle (ego) and the connected vehicle (cav) in the scene use the equipped lidar to collect raw point cloud data.

[0066] 102: Point cloud projection.

[0067] The original point cloud data coordinates collected are projected to the coordinates centered on the vehicle to obtain the projected point cloud data. Projected to the coordinates centered on the vehicle itself

[0068]

[0069] Where R and T represent the rotation matrix and translation variable of the original point cloud data projected into the coordinate system centered on the vehicle.

[0070] 103: Point cloud point column fusion and feature extraction.

[0071] The projected point cloud data is discretized into a network evenly distributed on the xy plane. A pillar is created for the point cloud data falling on the same grid. P non-empty pillars are generated for all projected point cloud data. Each pillar contains N point cloud data, and the dimension of each point cloud is D. A tensor with a feature volume of (D, P, N) is obtained. Then PointNet is used to extract features from the tensorized point cloud data.

[0072] 104:Pseudo image generation and multi-scale information extraction.

[0073] The result of feature extraction in step 103 is a pseudo image of the form (C, H, W), where C, H, W represent the number of channels, length, and width of the pseudo image, respectively. Then, a feature pyramid network is used to extract multi-scale features from the pseudo image: F1∈R 2C×H×W , F2∈R 2C×H×W F3∈R 2C×H×W , concatenate multi-scale features to get F c ∈R 6C×H×W , F c It contains features of three scales: large, medium, and small. Finally, convolution is used to compress the features to achieve data dimensionality reduction and obtain lightweight perceptual feature data F∈R C×H×W .

[0074] 105: Feature confidence map generation with label supervision.

[0075] Through deep supervision of feature labels, lightweight perceptual feature data is used to generate feature confidence maps, which are updated as the number of communication rounds changes. The specific steps include:

[0076] The vehicle itself and the connected vehicle use the detection head to generate feature confidence maps. The area with a larger score in the confidence map represents the target foreground, and the area with a smaller score represents the background. Specifically, given the feature map F of the Kth round of communication i (k) , the corresponding feature confidence map is:

[0077]

[0078] in is the feature confidence map generated by the i-th vehicle in the k-th round of communication, Θ generator It is the target detection head;

[0079] The feature confidence map generation process is deeply supervised by introducing feature labels:

[0080]

[0081] in is the deep supervision loss, is the cross entropy loss function, Ψ fuse It is a multi-head self-attention, which is used to fuse the feature confidence maps in multiple rounds of communication. GT i (k) is the true feature label of the i-th vehicle in the k-th round of communication.

[0082] Introducing deep supervision in feature confidence generation has the following two advantages: (1) It can better optimize the point column fusion, feature extraction and pseudo image multi-scale feature extraction process, so that the generated feature confidence map can better reflect the feature foreground and background; (2) Through multiple rounds of fusion of feature confidence maps and deep supervision of labels, it can better resist the noise and interference caused by sensors and models, thereby improving the robustness of the model.

[0083] 106: 3D Bounding Box coordinate extraction and conversion.

[0084] The eight 3D Bounding Box corner coordinates of the feature label are obtained, and the 3D Bounding Box corner coordinates are converted into pseudo image pixel coordinates. Based on the feature confidence map, the features of the corresponding pixel coordinate area are selected as collaborative shared features and sent to the own vehicle. The inter-vehicle feature fusion is performed to obtain the fused perception features. The specific steps include:

[0085] 1) Get the coordinates of the eight corner points [x, y, z, l, w, h, θ] according to the 3D Bounding Box label [x, y, z, l, w, h, θ] corner_i ,y corner_i ,z corner_i ], (i = 1-8);

[0086] 2) Taking the coordinate system centered on the vehicle as a reference, obtain the maximum and minimum values ​​of the eight corner points on the xy plane and obtain the coordinates [x min ,y min ,x max ,y max ];

[0087] 3) [x min ,y min ,x max ,y max ] is converted to pixel coordinates based on the pseudo image: Since the laser radar detection range is x:[-140,140], y:[-38,38], and the pseudo image pixel coordinate range is x:[0,176], y:[0,48], it is necessary to convert [x min ,y min ,x max ,y max ] to transform the coordinates [x min ,ymin ,x max ,y max ] are constrained to be within the pseudo image pixel coordinate range:

[0088]

[0089] Get the coordinates of the corresponding position of the target on the pseudo image [x min_pi ,y min_pi ,x max_pi ,y max_pi ], where r x , x y They represent the detection range of the laser radar in the x and y directions, respectively, and H and W represent the length and width of the pseudo image, respectively;

[0090] 4) According to [x min_pi ,y min_pi ,x max_pi ,y max_pi ] Generate an AC graph: Given a tensor T∈R with zero value and the same size as the pseudo image H×W , according to the coordinates [x min_pi ,y min_pi ,x max_pi ,y max_pi ], the coordinate range of the tensor T is in x max_pi -x min_pi With y max_pi -y min_pi Set the value in to 1:

[0091]

[0092] Get the communication diagram

[0093] 107: Coordinate-based shared feature selection.

[0094] Based on the feature confidence map generated by each connected vehicle, according to the communication diagram Indicates whether the corresponding area in the perception feature of the connected vehicle is selected to be sent to the own vehicle for inter-vehicle feature fusion. 1 means selection and 0 means no selection. Therefore, the selected collaborative sharing feature can be expressed as:

[0095]

[0096] in represents the collaborative sharing feature sent by the jth vehicle to the ith vehicle in the kth round of communication;

[0097] 108: Inter-vehicle perception feature fusion.

[0098] Fusion of shared features from connected vehicles using multi-head self-attention The original feature F of the vehicle i (k) :

[0099]

[0100] in represents the fused perceptual features in the kth round of communication, MHSA is multi-head self-attention;

[0101] The feature selection method based on pseudo image pixel coordinates has the following three advantages: (1) Shared features The original feature F is included i (k) The target is blocked in the image, and the area with the richest information in the perception feature data of the connected vehicle is selected through the pseudo image pixel coordinates, and the original feature F of the vehicle itself is i (k) (2) Since only the features within the coordinate range are shared, the amount of shared information is greatly reduced, saving communication costs; (3) Sparse communication graphs Depending on the binary selection matrix, the binary matrix selects different sensing areas according to the changing pseudo image pixel coordinates, thereby dynamically allocating the communication budget of each connected vehicle.

[0102] 109: Object detection.

[0103] Use the preset target detection model to fused the perception features Perform target detection to obtain target frame category, position and size information. The specific steps include:

[0104] Use the SSD target detection head in PointPillar to generate target box category, position, and size information;

[0105] The cross entropy loss function is used to calculate the target box category prediction loss:

[0106]

[0107] in is the category loss;

[0108] The SmoothL1 function is used to calculate the target box positioning loss:

[0109]

[0110] Among them, x, y, z represent the center coordinates of the 3D Bounding Box, w, h, l represent the length, width, and height dimensions of the 3D Bounding Box, θ represents the direction angle of the 3D Bounding Box, and Δb represents the offset of these 7 variables, which consists of the following parts:

[0111]

[0112] Δθ=sin(θ gt -θ a )

[0113] The total loss of the model is:

[0114]

[0115] Among them, α=γ=1.0 and γ=0.5.

[0116] The present invention is tested on the public OPV2V dataset. The OPV2V dataset is the first large-scale perception open dataset with V2V communication function. 73 different scenarios are collected through the OpenCDA and CARLA simulation frameworks, covering different challenging driving situations, such as ultra-long perception distance and severe occlusion. The scenario contains at least 2 networked vehicles (cav) and at most 7 networked vehicles. Each networked vehicle contains 4 cameras, 64-channel lidar and GPS / IMU. The OPV2V dataset contains 11464 frames of lidar point cloud and RGB images, of which the training set, validation set, and test set are 6764, 1981 / 2719 frames, respectively.

[0117] The present invention is compared and analyzed with 3D target detection methods: Early Fusion, Late Fusion, Intermediate Fusion, Fcooper, V2VNet, V2X-Vit, Cobevt, and Where2Comm. The verification indicators are 3D target detection accuracy AP@0.5 / 0.7, shared information rate (the proportion of shared features in feature maps), shared information volume, and GPU memory occupancy. Table 1 lists the data comparison results.

[0118] Table 1 Comparison of 3D object detection results on the OPV2V dataset

[0119]

[0120] Among them, "-" means that the method does not give specific data, and "ALL" means that the method uses the original point cloud data for fusion, and the amount of shared information is calculated by the formula:

[0121]

[0122] Calculated, represents the number of non-zero elements in the shared features, and D represents the channel dimension.

[0123] In order to verify the effectiveness of the method of the present invention, experiments were conducted on two NVIDIA RTX 3090s, the optimizer was Adam, the initial learning rate was 0.0005, the model was trained for 60 epochs, the first ten epochs were warmup, and the initial learning rate of warmup was 0.00005. In addition, the method of the present invention aims to solve the problem of efficient selection of shared features in vehicle-to-vehicle collaboration. Therefore, the present invention assumes that there is no posture error and asynchronous problem when all vehicles share features, that is, the vehicle itself can receive features from networked vehicles at the timestamp.

[0124] The target frame obtained by the present invention is as follows Figure 2 The present invention uses the mid-term fusion method Where2com as the baseline model, and retrains the models that need to be compared in the table to achieve fair comparison. It can be seen from the table that the detection accuracy of both the baseline model and the method of the present invention is lower than that of the early fusion methods Early Fusion and Fcooper, but the video memory occupancy is relatively large. Compared with other mid-term fusion methods: Intermediate Fusion, CoBEVT, V2VNet, and V2X-Vit, although the perception accuracy of the method of the present invention has declined, it saves the amount of shared information and video memory between vehicles. Compared with the baseline model Where2comm, the method of the present invention reduces the shared information rate from 49.8% to 0.8%, a decrease of 98.39%, and reduces the amount of shared information from 15.36 to 9.71, a decrease of 37.87%. The video memory occupancy during training is also reduced by 5.58G, while the accuracy of 0.5 / 0.7 is only reduced by 0.78 / 1.24. From the comparative data, it can be found that compared with other mid-term fusion methods and baseline models, the method of the present invention only shares the features within the coordinate range during training, which greatly reduces the amount of shared information, saves precious vehicle communication bandwidth, and only loses a very small amount of perception accuracy.

[0125] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0126] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0127] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0130] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0131] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A target detection method for efficient vehicle-to-vehicle collaboration, characterized in that: The following steps are involved: Collect point cloud data of the scene where the vehicle and the connected vehicle are located; Based on the point cloud data, point column fusion and feature extraction are performed to obtain a pseudo image of the point cloud data; Based on the pseudo image of the point cloud data, multi-scale feature extraction and dimensionality reduction processing are performed to obtain lightweight perceptual features; Based on the perceptual features, generating a feature confidence map through deep supervision of feature labels, wherein the feature confidence map is updated as the communication rounds change; Based on the feature confidence map, selecting collaborative shared features, and combining the perception features to perform feature fusion between vehicles to obtain fused perception features; A pre-trained target detection model is used to perform target detection on the fused perception features to obtain a target detection result.

2. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 1, characterized in that: The step of obtaining a pseudo image of point cloud data comprises: Based on the point cloud data, projection is performed with the own vehicle as the center to obtain projected point cloud data; Discretize the projected point cloud data into a network evenly distributed on the xy plane, and create a point column for the projected point cloud data located in the same grid, finally generate P non-empty point columns, and further obtain a tensor with a characteristic volume of (D, P, N), where D is the point cloud dimension and N is the number of point clouds in each point column; Based on the tensor, a PointNet network is used to extract features to obtain a pseudo image of the point cloud data.

3. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 2, characterized in that: The projected point cloud data is expressed as: In the formula, is the coordinate of the projected point cloud data, R and T represent the rotation matrix and translation variable of the point cloud data projected to the coordinate system centered on the vehicle. is the coordinate of the point cloud data before projection.

4. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 1, characterized in that: The step of obtaining lightweight perceptual features comprises: A feature pyramid network is used to extract multi-scale features from the pseudo image; The multi-scale features are spliced ​​to obtain the spliced ​​feature F c ; Based on the splicing feature F c , convolution is used to compress features and obtain lightweight perceptual features F.

5. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 1, characterized in that: The expression for deeply supervising the feature confidence map generation process through the feature label is: In the formula, is the deep supervision loss, is the cross entropy loss function, Ψ fuse It is a multi-head self-attention, which is used to fuse the feature confidence maps in multiple rounds of communication. is the true feature label of the i-th vehicle in the k-th round of communication, and the feature label is contained in the point cloud data.

6. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 1, characterized in that: The expression of the feature confidence map generated by each communication round is: In the formula, is the feature confidence map generated by the i-th vehicle in the k-th round of communication, Θ generator is the target detection head, F i (k) is the perceptual feature of the i-th vehicle in the K-th round of communication, H and W represent the length and width of the image, respectively.

7. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 1, characterized in that: The step of obtaining the fused perception features comprises: Get multiple 3D bounding box corner coordinates [x corner_i ,y corner_i ,z corner_i ], where i represents the number of corner points; Convert multiple corner point coordinates into pseudo image pixel coordinates to obtain an AC graph; Based on the feature confidence map and the communication map representation, collaborative shared features are selected, and combined with the perception features, feature fusion between vehicles is performed to obtain fused perception features.

8. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 7, characterized in that: The step of obtaining the communication diagram comprises: The coordinate system centered on the vehicle itself is used as a reference to obtain the maximum and minimum values ​​of multiple corner point coordinates on the xy plane, and obtain the coordinates [x min ,y min ,x max ,y max ]; The coordinates [x min ,y min ,x max ,y max ] is converted based on the pixel coordinates of the pseudo image to obtain the coordinates of the corresponding position on the pseudo image, wherein the conversion expression is: In the formula, [x min_pi ,y min_pi ,x max_pi ,y max_pi ] is the coordinate of the corresponding position on the pseudo image, r x , x y They represent the detection range of the laser radar in the x and y directions, respectively, and H and W represent the length and width of the pseudo image, respectively; Given a tensor T with zero values ​​and the same size as the pseudo image, according to the coordinates [x min_pi ,y min_pi ,x max_pi ,y max_pi ] Set the coordinate range of the tensor T to x max_pi -x min_pi With y max_pi -y min_pi The value in the coordinate range is set to 1 to obtain an AC diagram, in which the process of setting the coordinate range value to 1 is expressed as: In the formula, The subscript HW represents a coordinate index.

9. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 7, characterized in that: The step of obtaining the fused perception features comprises: Based on the feature confidence map, communication map and perception features of each connected vehicle, a collaborative shared feature is selected, wherein the selection expression of the collaborative shared feature is: In the formula, represents the collaborative sharing feature sent by the jth vehicle to the ith vehicle in the kth round of communication, F i (k) is the perception feature of the i-th vehicle in the K-th round of communication, is the communication graph of the i-th vehicle in the K-th round of communication; Use multi-head attention to transfer the shared features and the perception feature F of the vehicle itself i (k) Perform feature fusion to obtain the fused perceptual features, which are expressed as: In the formula, represents the fused perceptual features in the kth round of communication, MHSA is the multi-head self-attention operation, is the feature confidence map generated by the i-th vehicle in the k-th round of communication.

10. The target detection method for efficient vehicle-to-vehicle collaboration according to claim 1, characterized in that: The loss function used by the target detection model is: Among them, Δb includes the following components: Δθ=sin(θ gt -θ a ) In the formula, is the total loss, is the category loss, α, β, γ are weights, is the positioning loss, p(x i ) is the label of category i, q(x i ) is the probability of being predicted as category i, is the depth supervision loss, x, y, z represent the center coordinates of the 3D bounding box, w, h, l represent the length, width, and height dimensions of the 3D bounding box, θ represents the direction angle of the 3D bounding box, Δb is the offset of the variable, gt is the true label, d is the diagonal distance between the length and width of the 3D bounding box, h represents the height of the prior 3D bounding box, and a is the predicted value.

Images