A target detection method for efficient vehicle-vehicle cooperation

Abstract

The application relates to a target detection method for efficient vehicle-vehicle cooperation, comprising the following steps: collecting point cloud data under the scene of the ego vehicle and the connected vehicle; based on the point cloud data, point column fusion and feature extraction are carried out to obtain a pseudo image of the point cloud data; based on the pseudo image of the point cloud data, multi-scale feature extraction and dimension reduction processing are carried out to obtain light perception features; based on the perception features, a feature confidence map is generated through deep supervision of feature labels, wherein the feature confidence map is updated with the change of the communication round; based on the feature confidence map, collaborative sharing features are selected, and feature fusion between vehicles is carried out in combination with the perception features to obtain fused perception features; and a pre-trained target detection model is used to carry out target detection on the fused perception features to obtain a target detection result. Compared with the prior art, the application has the advantages of improving the efficiency of the perception task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving cooperative perception, and in particular to a target detection method for efficient vehicle-vehicle cooperation. BACKGROUND

[0002] Multi-vehicle cooperative perception technology is an important research direction in the field of automatic driving. This technology realizes comprehensive perception of the environment by sharing perception features among multiple vehicles. Cooperative perception technology fundamentally overcomes the inevitable limitations of single-vehicle perception, such as visual obstruction and long-distance detection. Some current cooperative perception methods share all perception features with other vehicles, which not only brings information redundancy, but also increases communication bandwidth. Therefore, there are still technical challenges and limitations in how to efficiently and effectively share perception features between vehicles.

[0003] Currently, multi-vehicle cooperative perception technology mainly relies on the fusion of vehicle-mounted sensor data and the exchange of information between vehicles. Real-time data sharing between different vehicles is achieved through vehicle-to-everything (V2X) protocols, enabling each vehicle to utilize real-time perception data from other vehicles for perception compensation. Although there are currently some cooperative perception systems based on V2X, the amount of information shared between vehicles is large, resulting in high bandwidth consumption and communication latency. Therefore, there are still challenges in balancing communication bandwidth consumption and perception accuracy, and effectively selecting features with rich target information for sharing.

[0004] Some current multi-vehicle cooperative perception methods share complementary perception features between vehicles to balance perception accuracy and communication bandwidth. However, these methods generate perception features through target detection heads and randomly select the top k perception feature values for sharing. When k is too large, it leads to perception feature redundancy, and when k is too small, it leads to insufficient perception features. In the face of target detection problems under vehicle-vehicle cooperation, it is necessary to optimize perception features to improve target detection efficiency. SUMMARY

[0005] The purpose of the present application is to provide a target detection method for efficient vehicle-vehicle cooperation that improves perception task efficiency.

[0006] The purpose of the present application can be achieved through the following technical solutions:

[0007] A target detection method for efficient vehicle-vehicle cooperation, comprising the following steps:

[0008] Collecting point cloud data under the scene of the own vehicle and connected vehicles;

[0009] Based on the point cloud data, performing point column fusion and feature extraction to obtain a pseudo image of the point cloud data;

[0010] Based on the pseudo image of the point cloud data, multi-scale feature extraction and dimension reduction processing are performed to obtain lightweight perception features;

[0011] Based on the perception features, a feature confidence map is generated through deep supervision of feature labels, wherein the feature confidence map is updated with changes in communication rounds;

[0012] Based on the feature confidence map, collaborative shared features are selected, and feature fusion between vehicles is performed in combination with the perception features to obtain fused perception features;

[0013] A pre-trained target detection model is used to perform target detection on the fused perception features to obtain a target detection result.

[0014] Further, the step of obtaining the pseudo image of the point cloud data comprises:

[0015] Based on the point cloud data, projection is performed with the ego vehicle as the center to obtain projected point cloud data;

[0016] The projected point cloud data is discretized into a network uniformly distributed on the x-y plane, and a point column is created for the projected point cloud data located in the same grid, finally generating P non-empty point columns, and further obtaining a tensor with a feature volume of (D, P, N), wherein D is the point cloud dimension, and N is the number of point clouds in each point column;

[0017] Based on the tensor, a PointNet network is used for feature extraction to obtain the pseudo image of the point cloud data.

[0018] Further, the projected point cloud data is represented as:

[0019]

[0020] In the formula, is the coordinate of the projected point cloud data, R and T represent the rotation matrix and translation variable of the point cloud data projected to the coordinate system with the ego vehicle as the center, is the coordinate of the point cloud data before projection.

[0021] Further, the step of obtaining the lightweight perception features comprises:

[0022] A feature pyramid network is used to extract multi-scale features from the pseudo image;

[0023] The multi-scale features are spliced to obtain spliced features F c ;

[0024] Based on the spliced features F c , convolution is used for feature compression to obtain lightweight perception features F.

[0025] Further, the expression of deep supervision of the feature confidence map generation process by the feature label is:

[0026]

[0027] In the formula, is a deep supervision loss, is a cross-entropy loss function, Ψ fuse is a multi-head self-attention used to fuse the feature confidence map in multi-round communication GT i (k) is the real feature label of the ith vehicle in the kth round of communication, and the feature label is contained in the point cloud data.

[0028] Further, the expression of the feature confidence map generated in each communication round is:

[0029]

[0030] In the formula, is the feature confidence map generated by the ith vehicle in the kth round of communication, Θ generator is a target detection head, F i (k) is the perception feature of the ith vehicle in the Kth round of communication, H and W represent the length and width of the image respectively.

[0031] Further, the step of obtaining the fused perception feature comprises:

[0032] obtaining a plurality of 3D bounding box corner point coordinates [x corner_i ,y corner_i ,z corner_i ] from the feature label, wherein i represents the number of corner points;

[0033] converting the plurality of corner point coordinates to pseudo-image pixel coordinates to obtain a communication graph;

[0034] selecting collaborative shared features based on the feature confidence map and communication graph representation, and combining the perception feature to perform feature fusion between vehicles to obtain the fused perception feature.

[0035] Further, the step of obtaining the communication graph comprises:

[0036] taking the coordinate system centered on the ego vehicle as a reference, obtaining the maximum and minimum values of a plurality of corner point coordinates on the x-y plane to obtain coordinates [x min ,y min ,x max ,y max ];

[0037] coordinates [x min ,y min ,x max ,y max The pixel coordinates of the pseudo-image are transformed to obtain the coordinates of the corresponding position on the pseudo-image, wherein the transformation expression is:

[0038]

[0039] In the formula, [x min_pi ,y min_pi ,x max_pi ,y max_pi [r] represents the coordinates of the corresponding position on the pseudo-image. x x y These represent the detection range of the lidar in the x and y directions, respectively, and H and W represent the length and width of the pseudo-image, respectively.

[0040] Given a tensor T with a value of zero and the same size as the pseudo-image, based on coordinates [x min_pi ,y min_pi ,x max_pi ,y max_pi The coordinate range of tensor T is in the x-axis. max_pi -x min_pi With y max_pi -y min_pi The value within the range is set to 1 to obtain the AC diagram. The process of setting the coordinate range value to 1 is represented as follows:

[0041]

[0042] In the formula, The subscript HW indicates the coordinate index.

[0043] Furthermore, the step of obtaining the fused perceptual features includes:

[0044] Based on the feature confidence map, communication map, and perception features of each connected vehicle, collaborative sharing features are selected, wherein the selection expression for the collaborative sharing features is:

[0045]

[0046] In the formula, F represents the collaborative sharing feature sent by vehicle j to vehicle i in the k-th round of communication. i (k) Let represent the perceptual features of the i-th vehicle during the K-th round of communication. The diagram shows the interaction between the i-th vehicle and the k-th vehicle in the k-th round.

[0047] Multi-head attention is used to integrate the shared features a perception feature F of the ego vehicle i (k) The feature fusion is performed to obtain a fused perception feature, denoted as:

[0048]

[0049] In the formula, denotes the fused perception feature in the kth round of communication, and MHSA is a multi-head self-attention operation, is a feature confidence map generated by the ith vehicle in the kth round of communication.

[0050] Further, the loss function adopted by the target detection model is:

[0051]

[0052] Wherein, Δb includes the following components:

[0053]

[0054] Δθ=sin(θ gt -θ a )

[0055] In the formula, is the total loss, is the category loss, and alpha, beta and gamma are weights, is the positioning loss, p(x i ) is the label of category i, and q(x i ) is the probability of being predicted as category i, is the deep supervision loss, x, y and z represent the center coordinates of the 3D bounding box, w, h and l represent the length, width and height size data of the 3D bounding box, theta represents the direction angle of the 3D bounding box, Delta b is the offset of the variable, gt is the true label, d is the diagonal distance of the length and width of the 3D bounding box, h represents the height of the prior 3D bounding box, and a is the predicted value.

[0056] Compared with the prior art, the present application has the following beneficial effects:

[0057] (1) The present application performs multi-round fusion and feature supervision on the generation process of the feature confidence map, which can better optimize the point column fusion, feature extraction and pseudo-image multi-scale feature extraction process, so that the generated feature confidence map can better reflect the feature foreground and background. By selecting the area with the most abundant information in the perception feature data based on the pseudo-image pixel coordinates in the shared feature selection process, the missing information in the original perception feature can be supplemented, so that the amount of shared information is greatly reduced, and the efficiency of the perception task is improved.

[0058] (2) The application can better resist noise and interference caused by sensors and models through multi-round fusion of feature confidence maps and deep supervision of labels, thereby improving the robustness of the overall detection process.

[0059] (3) The application selects shared features based on pseudo image pixel coordinates, and the shared features contain the occluded target in the original perception features, so that the target foreground information missing from the ego vehicle is selected for sharing, which greatly reduces the amount of shared information and saves communication costs; the sparse exchange graph depends on the binary selection matrix, and the binary matrix selects different perception areas according to the changing pseudo image pixel coordinates, thereby dynamically allocating the communication budget of each connected vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a method flowchart of the application;

[0061] Figure 2 is a target box coordinate mapping visualization diagram of the application. DETAILED DESCRIPTION

[0062] The application will be described in detail below in conjunction with the drawings and specific embodiments. The embodiments are implemented on the basis of the technical solutions of the application, and detailed implementation methods and specific operation processes are given, but the protection scope of the application is not limited to the following embodiments.

[0063] The embodiment provides a target detection method for efficient vehicle-vehicle cooperation, as shown in Figure 1 The method comprises the following steps:

[0064] 101: Collecting original point cloud data.

[0065] The ego vehicle and the connected vehicle (cav) in the scene use the equipped laser radar to collect original point cloud data.

[0066] 102: Point cloud projection.

[0067] The collected original point cloud data coordinates are projected to the coordinates centered on the ego vehicle to obtain the projected point cloud data, and the original point cloud data coordinates are projected to the coordinates centered on the ego vehicle

[0068]

[0069] Wherein R and T represent the rotation matrix and the translation variable of the original point cloud data projected to the coordinate system centered on the ego vehicle, respectively.

[0070] 103: Point cloud point column fusion and feature extraction.

[0071] The projected point cloud data is discretized into a network uniformly distributed on the x-y plane, and a point column is created for the point cloud data falling into the same grid. P non-empty point columns are generated for all projected point cloud data, each point column containing N point cloud data, each point cloud having a dimension of D, obtaining a tensor with a feature volume of (D, P, N); then the tensorized point cloud data is extracted using PointNet.

[0072] 104: Pseudo-image generation and multi-scale information extraction.

[0073] The result of step 103 feature extraction is a pseudo-image of the form (C, H, W), where C, H, and W represent the number of channels, length, and width of the pseudo-image, respectively. Then a feature pyramid network is used to extract multi-scale features from the pseudo-image: F1∈R 2C×H×W , F2∈R 2C×H×W F3∈R 2C×H×W , and the multi-scale features are concatenated to obtain F c ∈R 6C×H×W , F c contains features of large, medium, and small scales. Finally, feature compression is performed using convolution to reduce the dimension of the data, obtaining lightweight perception feature data F∈R C×H×W .

[0074] 105: Feature confidence map generation based on label supervision.

[0075] Through deep supervision of feature labels, a feature confidence map is generated using lightweight perception feature data. The feature confidence map is updated as the number of communication rounds changes. The specific steps include:

[0076] The ego vehicle and connected vehicles generate a feature confidence map using a detection head. The area with a larger score in the confidence map represents the target foreground, and the area with a smaller score represents the background. Specifically, given the feature map F i (k) , the corresponding feature confidence map is:

[0077]

[0078] where F is the feature confidence map generated by the i-th vehicle in the k-th round of communication, and Θ generator is the target detection head.

[0079] The feature confidence map generation process is deeply supervised by introducing feature labels:

[0080]

[0081] where L is the deep supervision loss, ​Ψ is the cross-entropy loss function fuse is the multi-head self-attention used to fuse the feature confidence maps in multi-round communication GT i (k) is the true feature label of the ith vehicle in the kth round of communication.

[0082] Introducing deep supervision in feature confidence generation has the following two advantages: (1) It can better optimize the point column fusion, feature extraction, and multi-scale feature extraction process of pseudo images, so that the generated feature confidence map can better reflect the feature foreground and background; (2) Through the multi-round fusion of feature confidence maps and the deep supervision of labels, it can better resist noise and interference caused by sensors and models, and improve the robustness of the model.

[0083] 106: 3D Bounding Box coordinate extraction and conversion.

[0084] Obtain the eight 3D Bounding Box corner point coordinates of the feature label, convert the 3D Bounding Box corner point coordinates into pseudo image pixel coordinates, select the features in the corresponding pixel coordinate region based on the feature confidence map as the collaborative shared features and send them to the ego vehicle for inter-vehicle feature fusion to obtain the fused perception features, which specifically includes the following steps:

[0085] 1) Obtain eight corner point coordinates [x corner_i ,y corner_i ,z corner_i ], (i = 1-8) from the 3D Bounding Box label [x, y, z, l, w, h, θ];

[0086] 2) Take the coordinate system centered on the ego vehicle as the reference to obtain the maximum and minimum values of the eight corner point coordinates in the x-y plane, and obtain the coordinates [x min ,y min ,x max ,y max ];

[0087] 3) Convert [x min ,y min ,x max ,y max ] to pseudo image-based pixel coordinates: Since the detection range of the laser radar is x: [-140, 140], y: [-38, 38], and the pixel coordinate range of the pseudo image is x: [0, 176], y: [0, 48], coordinate transformation is needed for [x min ,y min ,x max ,y max ] to coordinate [x min ,ymin max max Constrained in the pseudo-image pixel coordinate range:

[0088]

[0089] Get the coordinates of the corresponding position of the target on the pseudo-image [x min_pi min_pi max_pi max_pi ], where r x , x y respectively represent the range detected by the lidar in the x, y direction, H, W respectively represent the length and width of the pseudo-image;

[0090] 4) Generate an exchange map according to [x min_pi min_pi max_pi max_pi ]: Given a tensor T ∈ R H×W with a value of zero and a size consistent with the pseudo-image, according to the coordinates [x min_pi min_pi max_pi max_pi ], set the value of the coordinates in the tensor T within the range x max_pi -x min_pi and y max_pi -y min_pi to 1:

[0091]

[0092] Get the exchange map

[0093] 107: Coordinate-based shared feature selection.

[0094] Based on the feature confidence map generated by each connected vehicle, according to the exchange map characterize whether to select the corresponding region in the connected vehicle's perception feature to send to the ego vehicle for inter-vehicle feature fusion, 1 represents selection, 0 represents no selection, so the selected collaborative shared feature can be represented as:

[0095]

[0096] where represents the collaborative shared feature sent by the jth vehicle to the ith vehicle in the kth round of exchange;

[0097] 108: Inter-vehicle perception feature fusion.

[0098] Use multi-head self-attention to fuse shared features from connected vehicles​​​​​​​​​​​ With its own original characteristics F i (k) :

[0099]

[0100] in This represents the fused perceptual features in the k-th round of communication, where MHSA stands for Multi-Head Self-Attention.

[0101] The feature selection method based on pseudo-image pixel coordinates has the following three advantages: (1) Shared features It contains the original feature F i (k) For occluded targets, the region with the richest information in the networked vehicle perception feature data is selected using the pixel coordinates of the pseudo-image, and then the original features of the vehicle F are analyzed. i (k) (1) Supplementing missing information; (2) Since only features within the coordinate range are shared, the amount of shared information is greatly reduced, saving communication costs; (3) Sparse communication graph Depending on the binary selection matrix, which selects different sensing regions based on the changing pixel coordinates of the pseudo-image, the communication budget for each connected vehicle is dynamically allocated.

[0102] 109: Object detection.

[0103] The fused perceptual features are processed using a pre-defined target detection model. To perform object detection and obtain the object bounding box category, location, and size information, the specific steps include:

[0104] Use the SSD object detection head in PointPillar to generate object bounding box category, location, and size information;

[0105] The cross-entropy loss function is used to calculate the target box category prediction loss:

[0106]

[0107] in For category loss;

[0108] The SmoothL1 function is used to calculate the target bounding box localization loss:

[0109]

[0110] Where x, y, z represent the 3D Bounding Box center coordinates, w, h, l represent the 3D Bounding Box length, width, height size data, theta represents the 3D Bounding Box direction angle, and Delta b represents the offset of the seven variables, which consists of the following parts:

[0111]

[0112] Delta theta = sin(theta gt - theta a )

[0113] The total loss of the model is:

[0114]

[0115] Where alpha = gamma = 1.0, and gamma = 0.5.

[0116] The present application is tested on the disclosed OPV2V dataset, which is the first large-scale perception open dataset with V2V communication function, and 73 different scenes are collected through OpenCDA and CARLA simulation framework, covering different challenging driving conditions, such as super-long perception distance and severe occlusion, the scene contains at least 2 connected vehicles (cav) and at most 7 connected vehicles, each connected vehicle contains 4 cameras, 64 channels of laser radar and GPS / IMU, the OPV2V dataset contains 11464 frames of laser radar point cloud and RGB image, wherein the training set, the verification set and the test set are 6764, 1981 / 2719 frames respectively.

[0117] The present application is compared and analyzed with 3D target detection methods: Early Fusion, Late Fusion, Intermediate Fusion, Fcooper, V2VNet, V2X-Vit, Cobevt and Where2Comm, and the verification indexes are 3D target detection accuracy AP@0.5 / 0.7, shared information rate (proportion of shared features to feature maps), shared information amount and GPU memory occupation, and Table 1 lists the data comparison results.

[0118] Table 1 Comparison of 3D target detection results on OPV2V dataset

[0119]

[0120] Where "-" represents that the method does not give specific data, "ALL" represents that the method uses original point cloud data for fusion, and the shared information amount is calculated by the formula:

[0121]

[0122] The calculation is, represents the number of non-zero elements in the shared feature, and D represents the channel dimension.

[0123] In order to verify the effectiveness of the method of the application, the application is tested on two NVIDIA RTX 3090s, the optimizer is Adam, the initial learning rate is 0.0005, the model is trained for 60 epochs, and the first ten epochs are warmup, the initial learning rate of warmup is 0.00005, in addition, the method of the application aims to solve the problem of efficient selection of shared features in car-car cooperation, therefore, the application assumes that there is no pose error and asynchronous problem when all vehicles share features, that is, the ego vehicle can receive features from connected vehicles at the timestamp.

[0124] The target frame obtained by the application is as shown in Figure 2 The application takes the Where2com intermediate fusion method as the baseline model, and re-trains the models in the table that need to be compared to achieve fair comparison. From the table, it can be seen that whether it is the baseline model or the method of the application, the detection accuracy is lower than that of the early fusion method Early Fusion and Fcooper, but the video memory occupation is more, compared with other intermediate fusion methods: Intermediate Fusion, CoBEVT, V2VNet, V2X-Vit, although the perception accuracy of the method of the application decreases, it saves the amount of shared information between vehicles and video memory, compared with the baseline model Where2comm, the method of the application reduces the sharing information rate from 49.8% to 0.8%, reduces by 98.39%, reduces the amount of shared information from 15.36 to 9.71, reduces by 37.87%, and the video memory occupation during training is reduced by 5.58G, while the accuracy of 0.5 / 0.7 is only reduced by 0.78 / 1.24, from the comparison data, compared with other intermediate fusion methods and the baseline model, the method of the application only shares the features within the coordinate range during training, which greatly reduces the amount of shared information, saves the valuable vehicle communication bandwidth, and only loses a small amount of perception accuracy.

[0125] If the above functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or partially contribute to the prior art, or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0126] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0127] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0128] These computer program instructions can also be stored in a computer readable storage medium that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction devices that implement the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0129] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operations steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide the function of realizing the processes specified in the flowcharts Figure 1 one flowchart or multiple flowcharts and / or blocks Figure 1 one block or multiple blocks.

[0130] Although the preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims be construed to include all such modifications and variations as fall within the scope of the application.

[0131] Obviously, various modifications and changes are possible in the present application without departing from the spirit and scope of the application. It is to be understood that the application includes any such modifications and changes only insofar as they come within the scope of the appended claims and their equivalents.

Claims

1. A target detection method for efficient vehicle-to-vehicle cooperative operation, characterized in that, Includes the following steps: Collect point cloud data of its own vehicle and connected vehicles in the scene; Based on the point cloud data, point-column fusion and feature extraction are performed to obtain a pseudo-image of the point cloud data; Based on the pseudo-images of the point cloud data, multi-scale feature extraction and dimensionality reduction are performed to obtain lightweight perceptual features. Based on the perceived features, a feature confidence map is generated through deep supervision of feature labels, wherein the feature confidence map is updated as the communication rounds change; Based on the feature confidence map, collaborative shared features are selected, and feature fusion between vehicles is performed in combination with the perception features to obtain the fused perception features. A pre-trained target detection model is used to perform target detection on the fused perceptual features to obtain the target detection results; The steps for obtaining the fused perceptual features include: Obtain the coordinates of multiple 3D bounding box corner points [x] from the feature labels. corner_i ,y corner_i ,z corner_i ], where i represents the number of corner points; The coordinates of multiple corner points are converted to pixel coordinates of the pseudo-image to obtain the AC image; Based on the feature confidence map and communication map representation, collaborative shared features are selected and combined with the perception features to perform feature fusion between vehicles, resulting in fused perception features. The steps for obtaining the communication diagram include: Using a coordinate system centered on the vehicle itself as a reference, the maximum and minimum values ​​of multiple corner point coordinates on the xy plane are obtained to obtain the coordinates [x...]. min ,y min ,x max ,y max ]; coordinates [x min ,y min ,x max ,y max The pixel coordinates of the pseudo-image are transformed to obtain the coordinates of the corresponding position on the pseudo-image, wherein the transformation expression is: In the formula, [x min_pi ,y min_pi ,x max_pi ,y max_pi [r] represents the coordinates of the corresponding position on the pseudo-image. x x y These represent the detection range of the lidar in the x and y directions, respectively, and H and W represent the length and width of the pseudo-image, respectively. Given a tensor T with a value of zero and the same size as the pseudo-image, based on coordinates [x min_pi ,y min_pi ,x max_pi ,y max_pi The coordinate range of tensor T is in the x-axis. max_pi -x min_pi With y max_pi -y min_pi The value within the range is set to 1 to obtain the AC diagram. The process of setting the coordinate range value to 1 is represented as follows: In the formula, The subscript HW indicates the coordinate index; The steps for obtaining the fused perceptual features include: Based on the feature confidence map, communication map, and perception features of each connected vehicle, collaborative sharing features are selected, wherein the selection expression for the collaborative sharing features is: In the formula, This represents the collaborative sharing feature sent by vehicle j to vehicle i in the k-th round of communication. Let represent the perceptual features of the i-th vehicle during the K-th round of communication. The diagram shows the interaction between the i-th vehicle and the k-th vehicle in the k-th round. Multi-head attention is used to integrate the shared features Perceptual characteristics of its own vehicle Feature fusion is performed to obtain the fused perceptual features, represented as follows: In the formula, This represents the fused perceptual features after the k-th round of communication. MHSA is a multi-head self-attention operation. This is the feature confidence map generated by the i-th vehicle in the k-th round of communication.

2. The target detection method for efficient vehicle-to-vehicle cooperation according to claim 1, characterized in that, The step of obtaining a pseudo-image of point cloud data includes: Based on the point cloud data, a projection is performed with the vehicle itself as the center to obtain the projected point cloud data. The projected point cloud data is discretized into a network uniformly distributed on the xy plane, and a point column is created for the projected point cloud data located in the same grid. Finally, P non-empty point columns are generated, and a tensor with a feature volume of (D, P, N) is obtained, where D is the point cloud dimension and N is the number of points in each point column. Based on the tensor, a PointNet network is used for feature extraction to obtain a pseudo-image of the point cloud data.

3. The target detection method for efficient vehicle-to-vehicle cooperation according to claim 2, characterized in that, The projected point cloud data is represented as follows: In the formula, Let R represent the coordinates of the projected point cloud data, and let T represent the rotation matrix and translation variables of the point cloud data projected onto a coordinate system centered on the vehicle itself. The coordinates of the point cloud data before projection.

4. The target detection method for efficient vehicle-to-vehicle cooperation according to claim 1, characterized in that, The steps for obtaining lightweight perceptual features include: A feature pyramid network is used to extract multi-scale features from the pseudo-image; The multi-scale features are concatenated to obtain the concatenated feature F. c ; Based on the splicing feature F c We use convolution to compress features, resulting in lightweight perceptual features F.

5. The target detection method for efficient vehicle-to-vehicle cooperation according to claim 1, characterized in that, The expression for deep supervision of the feature confidence map generation process using the feature labels is: In the formula, To deeply monitor losses, Let Ψ be the cross-entropy loss function. fuse For multi-head self-attention, used to fuse feature confidence maps in multi-turn interactions. GT i (k) Let be the true feature label of the i-th vehicle in the k-th round of communication. The feature label is contained in the point cloud data.

6. The target detection method for efficient vehicle-to-vehicle cooperation according to claim 1, characterized in that, The expression for the feature confidence map generated in each of the aforementioned communication rounds is: In the formula, Let Θ be the feature confidence map generated by the i-th vehicle in the k-th round of communication. generator For target detection head, F i (k) Let H and W represent the perceptual features of the i-th vehicle during the K-th round of communication, where H and W represent the length and width of the image, respectively.

7. The target detection method for efficient vehicle-to-vehicle cooperation according to claim 1, characterized in that, The loss function used in the target detection model is: Δb includes the following components: Δθ=sin(θ gt -θ a ) In the formula, For the total loss, For class loss, α, β, and γ are weights. For localization loss, p(x) i ) represents the label of category i, q(x) i ) represents the probability of being predicted as category i. For deep supervision loss, x, y, z represent the center coordinates of the 3D bounding box, w, h, l represent the length, width, and height dimensions of the 3D bounding box, θ represents the orientation angle of the 3D bounding box, Δb is the offset of the variable, gt is the ground truth label, d is the diagonal distance between the length and width of the 3D bounding box, h represents the height of the prior 3D bounding box, and a is the predicted value.

Images