A point cloud semantic segmentation method based on vehicle-to-vehicle cooperation

By using a vehicle-to-vehicle collaborative point cloud semantic segmentation method, which utilizes the pose information of multiple vehicles and a self-attention mechanism, efficient point cloud semantic segmentation is achieved, solving the problem of low perception accuracy of single vehicles and improving the perception capability and accuracy of autonomous driving.

CN118628745BActive Publication Date: 2026-11-17BEIHANG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410826139.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-25
Publication Date
2026-11-17
Estimated Expiration
2044-06-25

AI Technical Summary

Technical Problem

Existing autonomous driving systems based on single-vehicle perception have low perception accuracy and small range when facing complex scenarios, and the cost of adding sensors is high, making it difficult to achieve high-precision semantic segmentation.

Method used

By using vehicle-to-vehicle collaboration, the coordinate transformation and bird's-eye view feature extraction of point cloud data are performed using the pose information of multiple vehicles. A collaborative point cloud semantic segmentation model is constructed, and the semantic category prediction of multi-vehicle point cloud data is achieved by using encoder feature extraction, self-attention mechanism feature fusion and decoder segmentation head.

Benefits of technology

It improves the accuracy and efficiency of point cloud semantic segmentation, reduces computational load and cost, enhances perception range and accuracy, and reduces the need for additional sensors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118628745B_ABST
    Figure CN118628745B_ABST
Patent Text Reader

Abstract

This invention relates to the field of point cloud segmentation technology for autonomous driving, specifically to a point cloud semantic segmentation method based on vehicle-to-vehicle collaboration. The method includes: multiple vehicles collecting point cloud data respectively; transforming the point cloud data of a second vehicle to the coordinate system of a first vehicle based on spatial pose information; performing bird's-eye view feature extraction processing on the coordinate-transformed point cloud data of the second and first vehicles to obtain bird's-eye view features of the multiple vehicles; constructing a collaborative point cloud semantic segmentation model, performing encoder feature extraction, self-attention mechanism feature fusion, and decoder segmentation head output on the bird's-eye view features of the multiple vehicles to obtain the semantic category of the point cloud data of the multiple vehicles from the bird's-eye view perspective, which serves as the collaborative point cloud semantic segmentation result. This invention can improve the perception accuracy and perception range of point cloud semantic segmentation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of point cloud segmentation technology for autonomous driving, and specifically to a point cloud semantic segmentation method based on vehicle-to-vehicle collaboration. Background Technology

[0002] While autonomous driving systems based on single-vehicle perception have made significant progress, they underperform in some challenging scenarios, such as blind spots at congested intersections and long-distance obstacle recognition at high speeds. These factors seriously threaten the safety of autonomous driving and have become key bottlenecks hindering vehicles from reaching Level 5 automation. Simply adding more high-fidelity sensors results in high costs and provides only limited improvement in perception accuracy.

[0003] Thanks to improved communication infrastructure and advancements in technologies like V2X, vehicles can reliably exchange messages, enabling collaboration. Sharing perception information across different vehicle locations significantly expands the perception range, while fusion of redundant information in overlapping multi-vehicle perception areas further enhances system accuracy. Furthermore, considering point cloud semantic segmentation as a fundamental task in autonomous driving environmental perception, providing rich geometric shape and size information along with semantic information, it can be used to construct high-precision semantic maps and lay the foundation for downstream modules such as control planning. Currently, vehicle-to-vehicle cooperative perception is in its exploratory stage, with most cooperative perception tasks focusing on object detection, and work on cooperative semantic segmentation models is relatively lacking. Summary of the Invention

[0004] In view of the above problems, the present invention provides a point cloud semantic segmentation method based on vehicle-to-vehicle cooperation, which solves the technical problems of low presence perception accuracy and small perception range in the prior art based on single-vehicle environmental perception.

[0005] This invention provides a point cloud semantic segmentation method based on vehicle-to-vehicle cooperation, comprising the following steps:

[0006] Step S1: Collect point cloud data from multiple vehicles respectively; based on pose information, transform the point cloud data of the second vehicle to the coordinate system of the first vehicle through coordinate transformation;

[0007] The first vehicle is one of the plurality of vehicles, and the second vehicle is the other vehicle among the plurality of vehicles besides the first vehicle;

[0008] Step S2: Perform bird's-eye view feature extraction processing on the point cloud data of the second vehicle and the point cloud data of the first vehicle after coordinate transformation to obtain the bird's-eye view features of the multiple vehicles.

[0009] Step S3: Construct a collaborative point cloud semantic segmentation model. Extract features from the encoder, fuse features using a self-attention mechanism, and output the segmentation head of the decoder for the bird's-eye view features of the multiple vehicles to obtain the semantic category of the point cloud data of the multiple vehicles from the bird's-eye view perspective, which is used as the collaborative point cloud semantic segmentation result.

[0010] Preferably, step S1 specifically includes:

[0011] Step S1-1: Based on the pose information of the multiple vehicles, determine the coordinate transformation matrix from the coordinate system of the multiple vehicles to the world coordinate system;

[0012] Step S1-2: Based on the coordinate transformation matrix from the coordinate system of the second vehicle to the world coordinate system and the coordinate transformation matrix from the coordinate system of the first vehicle to the world coordinate system, obtain the coordinate transformation matrix from the coordinate system of the second vehicle to the coordinate system of the first vehicle.

[0013] Steps S1-3: Based on the coordinate transformation matrix from the coordinate system of the second vehicle to the coordinate system of the first vehicle, transform the coordinates of each point cloud in the point cloud data of the second vehicle to the coordinate system of the first vehicle.

[0014] Preferably, step S1-1 specifically includes:

[0015] Based on the roll angle, pitch angle, yaw angle, and spatial coordinates in the world coordinate system from the vehicle's pose information, the coordinate transformation matrix R from the vehicle's coordinate system to the world coordinate system is established as follows:

[0016]

[0017] Where r, p, and h are the roll angle, pitch angle, and yaw angle in the vehicle's pose information, respectively. c y c , z c ) represents the spatial coordinates of the vehicle in the world coordinate system.

[0018] Preferably, step S1-2 specifically includes:

[0019] The coordinate transformation matrix M from the coordinate system of the second vehicle to that of the first vehicle is calculated as follows:

[0020] M = R c ×R e -1

[0021] Among them, R c The coordinate transformation matrix from the second vehicle's coordinate system to the world coordinate system, R e -1 It is the inverse of the coordinate transformation matrix from the coordinate system of the first vehicle to the world coordinate system.

[0022] Preferably, steps S1-3 specifically include coordinate transformation in the following manner:

[0023]

[0024] Where M ij (1≤i≤3,1≤j≤4) represents the elements in matrix M, i and j are the row and column indices of matrix M respectively, (x,y,z) represents the point cloud coordinates of the second vehicle before coordinate transformation, and (x′,y′,z′) represents the point cloud coordinates after coordinate transformation.

[0025] Preferably, step S2 specifically includes:

[0026] (1) Perform BEV projection on the point cloud coordinates in the point cloud data according to the following formula:

[0027]

[0028] Where x′ and y′ are the point cloud coordinates after coordinate transformation, (u, v) represent the pixel coordinates on the BEV image corresponding to the point cloud coordinates, W and H represent the width and height of the BEV image respectively, and mmin(·) and max(·) represent the minimum value operation and the maximum value operation respectively.

[0029] (2) The pixel coordinates of the BEV image are adjusted according to the height information z of the corresponding point cloud data. n The data is sorted, where n is the index of the point cloud data. The BEV images are then processed by pixel value normalization, edge pixel correction, and effective pixel filtering to obtain the BEV features of each vehicle. The BEV features include the three-dimensional coordinates, depth information, and reflectivity of the point cloud data.

[0030] Preferably, step S3 specifically includes:

[0031] Step S3-1: The bird's-eye view features of each vehicle are extracted by the encoder through the feature extractor to obtain the intermediate features of each vehicle.

[0032] Step S3-2: Through the intermediate feature fusion network, the intermediate features of each vehicle are fused using a self-attention mechanism to obtain the fused features;

[0033] Step S3-3: Classify the fused features using the decoder segmentation head to obtain the semantic categories of the point cloud data of the multiple vehicles.

[0034] Preferably, step S3-1 specifically includes: the feature extractor is a residual connection-based encoder, which includes three sequentially connected global context modules and four residual dilated convolution modules;

[0035] The input data of the global context module is processed through a 1×1 dilated convolutional block, a non-linear activation function, and two 3×3 dilated convolutional blocks, and then added to the output of the non-activation function to serve as the output of the global context module.

[0036] The input data of the residual dilated convolution module is processed by a 1×1 dilated convolution block, two 3×3 dilated convolution blocks, and a 2×2 dilated convolution block. The data is then concatenated with the outputs of the two 3×3 dilated convolution blocks, followed by a 1×1 convolution calculation. The result is then added to the output of the 1×1 dilated convolution block. Finally, the data is processed by average pooling and random discarding to obtain the output of the residual dilated convolution module.

[0037] Preferably, step S3-2 specifically includes: the intermediate feature fusion network comprises three sequentially connected feature fusion modules; the feature fusion module takes the intermediate features output by the feature extractor as input, and after shape transformation and transpose operations, obtains the Q, K, and V matrices, and then calculates the correlation matrix A. N×N And output matrix O:

[0038]

[0039] O = A N×N ·V

[0040] Where N represents the number of vehicles, softmax(·), dim = 0 indicates normalization according to the rows of the matrix, and d k is the dimension of the key matrix; Q, K, and V are the query matrix, key matrix, and value matrix, respectively; the output matrix O, after shape transformation and transpose, serves as the fused feature output by the intermediate feature fusion network.

[0041] Preferably, step S3-3 specifically includes: the decoder comprises three sequentially connected decoder modules; the input data of the decoder modules, after pixel shuffling and random discarding, is concatenated with the output of the residual dilated convolution module in the encoder via skip connections, and then after random discarding, two 3×3 dilated convolution blocks, and a 2×2 dilated convolution block, is concatenated with the output of the two 3×3 dilated convolution blocks as the output of the decoder modules; the segmentation head performs convolution calculation with a kernel size of 1×1 on the output of the decoder modules and performs random discarding to obtain the output logit of the segmentation head, and then calculates the pixel class classification result in the following manner:

[0042] Psm=softmax(logit),dim=1

[0043] Where logit represents the output of the segmentation head, softmax(·), dim=1 indicates normalization according to the channel dimension, and Psm is the semantic category of the point cloud.

[0044] Compared with the prior art, the present invention has at least the following beneficial effects:

[0045] (1) By unifying three-dimensional point cloud data from different perspectives into the same BEV perspective and using two-dimensional convolution to process image data, the present invention significantly reduces the computational load of the network and improves the processing efficiency compared with the traditional three-dimensional convolution method.

[0046] (2) This invention utilizes dilated convolutions with different dilation coefficients and convolution kernels of different sizes (1×1 and 3×3) to fuse features from large and small receptive fields, thereby obtaining more comprehensive contextual information and improving the extraction effect of semantic features. By employing a network based on a self-attention mechanism, intermediate features between different vehicles are fused, enhancing the expressive power of the feature map and improving the accuracy and effectiveness of point cloud semantic segmentation.

[0047] (3) This invention is aimed at autonomous vehicles. Most autonomous vehicles are equipped with the required radar and GPS equipment. Therefore, there is no need to add additional sensors, which reduces the processing load of the on-board processing equipment and lowers the cost. Attached Figure Description

[0048] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0049] Figure 1 This is a flowchart of the point cloud semantic segmentation method based on vehicle-to-vehicle collaboration disclosed in this invention.

[0050] Figure 2 This is a schematic diagram of the structure of the collaborative point cloud semantic segmentation model disclosed in this invention.

[0051] Figure 3 This is a schematic diagram of the structure of the global context module disclosed in this invention.

[0052] Figure 4 This is a schematic diagram of the residual dilated convolution module disclosed in this invention.

[0053] Figure 5 This is a schematic diagram of the intermediate feature fusion network disclosed in this invention.

[0054] Figure 6 This is a schematic diagram of the decoder and segmentation head disclosed in this invention. Detailed Implementation

[0055] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0056] To illustrate the effectiveness of the method proposed in this invention, the following detailed description of the above technical solution is provided through a specific embodiment, such as... Figure 1 As shown, a point cloud semantic segmentation method based on vehicle-to-vehicle cooperation is disclosed, and the specific implementation steps are as follows:

[0057] Step S1: Collect point cloud data from multiple vehicles respectively; based on spatial pose information, transform the point cloud data of the second vehicle to the coordinate system of the first vehicle through coordinate transformation;

[0058] The first vehicle is one of the plurality of vehicles, and the second vehicle is the other vehicle among the plurality of vehicles besides the first vehicle;

[0059] In this step, the spatial pose information of all vehicles is acquired. Based on this spatial pose information, a global coordinate transformation matrix from each vehicle's coordinate system to the world coordinate system is calculated. One vehicle is selected as the "own vehicle," and the point cloud data of other cooperating vehicles are uniformly transformed to the own vehicle's coordinate system according to the coordinate transformation relationship. Specific steps include:

[0060] (1) Select the vehicle with the smallest vehicle number as the primary vehicle, and the remaining vehicles as cooperative vehicles. Each cooperative vehicle should consist of at least two vehicles, but to conserve communication bandwidth, the number of cooperative vehicles should not exceed seven. Each vehicle uses sensors (such as lidar, cameras, radar, etc.) to capture 3D information about its surrounding environment. These sensors generate a large amount of point cloud data. A point cloud is a collection of numerous points, each representing a location in space, and typically contains other attribute information, such as reflection intensity or color. The collected point cloud data is converted to PCD format, and the x, y, and z coordinates of each point in the vehicle coordinate system are obtained. Each vehicle acquires its current spatial pose information, including its roll angle, pitch angle, yaw angle, and spatial coordinates in the world coordinate system.

[0061] (2) Based on the pose information of each vehicle at the same moment, determine the coordinate transformation matrix R from the vehicle coordinate system to the world coordinate system:

[0062]

[0063] Where r, p, and h represent the roll angle, pitch angle, and yaw angle in the vehicle attitude information, respectively (x c y c , z c ) represents the spatial coordinates of the vehicle in the world coordinate system. For the coordinate transformation matrix R, the elements in the first three columns of the first three rows are the elements involved in the rotation operation, and the elements in the fourth column of the first three rows are the elements involved in the translation operation.

[0064] (3) Based on the above steps, after obtaining the coordinate transformation matrix from the coordinate system of each vehicle to the world coordinate system, the coordinate transformation matrix M from the cooperating vehicle to the self-vehicle is calculated according to the following formula:

[0065] M = R c ×R e -1

[0066] Where R c The coordinate transformation matrix from the vehicle coordinate system to the world coordinate system for the cooperating vehicles, R e -1 M represents the inverse of the coordinate transformation matrix from the vehicle's coordinate system to the world coordinate system. Based on the coordinate transformation matrix M, the coordinates of the point cloud data collected by the cooperative vehicle are transformed in the following way:

[0067]

[0068] Where M i j (1≤i≤3,1≤j≤4) represents an element in matrix M, i and j are the row and column indices of matrix M, (x,y,z) represents the point cloud coordinates of the cooperative vehicle before coordinate transformation, and (x′,y′,z′) represents the point cloud coordinates after coordinate transformation.

[0069] The point cloud of the cooperative vehicle after coordinate transformation is uniformly transformed to the spatial coordinate system of the vehicle itself.

[0070] Step S2: Perform bird's-eye view feature extraction processing on the point cloud data of the second vehicle and the point cloud data of the first vehicle after coordinate transformation to obtain the bird's-eye view features of the multiple vehicles.

[0071] (1) For the point clouds of the cooperative vehicles and the autonomous vehicles after coordinate transformation, the three-dimensional point clouds are converted into two-dimensional pseudo-images by bird's-eye view (BEV) projection, and the point clouds are BEV projected according to the following formula:

[0072]

[0073] Where x′ and y′ are the point cloud coordinates after coordinate transformation, (u, v) represents the pixel coordinates on the BEV image corresponding to the point cloud coordinates, and W and H represent the width and height of the BEV image, respectively. mmin(·) and max(·) represent the minimum and maximum value operations, respectively.

[0074] (2) The calculated BEV image pixel coordinates are then adjusted according to the height information z of the corresponding point cloud data. n The data is sorted, where n is the index of the point cloud data. Additional processing is performed on the BEV images, including pixel value normalization, edge pixel correction, and filtering for valid pixels to remove noise and unnecessary information. Finally, the BEV features of each vehicle are obtained, which can be represented as tensors. It is said that tensor S n It includes the three-dimensional coordinates, depth information, and reflectivity of point cloud data.

[0075] Step S3: Construct a collaborative point cloud semantic segmentation model. Extract features from the encoder, fuse features using a self-attention mechanism, and output the segmentation head of the decoder for the bird's-eye view features of the multiple vehicles to obtain the semantic category of the point cloud data of the multiple vehicles from the bird's-eye view perspective, which is used as the collaborative point cloud semantic segmentation result.

[0076] In this step, a collaborative point cloud semantic segmentation model is constructed to process the BEV features of each vehicle and predict pixel-level point cloud classification results, which serve as the collaborative point cloud semantic segmentation results. The collaborative point cloud semantic segmentation model includes a feature extractor, an intermediate feature fusion network, a decoder, and a segmentation head. The collaborative point cloud semantic segmentation model achieves prediction through the following steps:

[0077] (1) The BEV features of each vehicle are extracted by a feature extractor to obtain the intermediate features of each vehicle. The feature extractor is specifically an encoder based on residual connections. The encoder includes three global context modules and four residual dilated convolution modules connected in sequence.

[0078] The structure of the global context module is as follows: Figure 3 As shown, the input data passes through a 1×1 dilated convolutional block with a dilation factor of 1, a non-linear activation function, a 3×3 dilated convolutional block with a dilation factor of 1, and a 3×3 dilated convolutional block with a dilation factor of 2. The output of this dilated convolutional block is then added to the output of the non-activation function to form the output of the global context module. The dilated convolutional block includes dilated convolution, a non-activation function, and a batch normalization layer. This approach ensures that the receptive field is increased without altering the image resolution, avoiding the loss of feature map spatial information. The outputs of the three global context modules are then concatenated, a process that fuses features from different receptive fields to obtain rich global context information.

[0079] The output of the global context module is used as the input to the residual dilated convolution module. The structure of the residual dilated convolution module is as follows: Figure 4 As shown, the input data is processed by an expanded convolutional block with a kernel size of 1×1, two expanded convolutional blocks with a kernel size of 3×3, and an expanded convolutional block with a kernel size of 2×2. This concatenation is then performed with the outputs of the two expanded convolutional blocks with a kernel size of 3×3. Next, a convolutional computation with a kernel size of 1×1 is performed, and the result is added to the output of the expanded convolutional block with a kernel size of 1×1, thus achieving residual connection. This allows for the extraction of feature information from different receptive fields, improving the gradient vanishing problem in deep neural networks and enhancing model performance and training efficiency. Finally, average pooling and random dropout are applied to obtain the output of the residual dilated convolutional module.

[0080] (2) An intermediate feature fusion network, employing a self-attention mechanism-based network, fuses intermediate features from other cooperating vehicles on the voluntary vehicle. The intermediate feature fusion network uses three sequentially connected feature fusion modules, as follows: Figure 5 As shown, the intermediate features encoded by the encoder are used as input, and after shape transformation and transpose operations, Q, K, and V matrices are obtained. The Q, K, and V matrices are the query matrix, key matrix, and value matrix, respectively, used to calculate the attention weights.

[0081] Based on the Q, K, and V matrices, a self-attention mechanism is used to calculate the correlation matrix A between the feature vectors of different vehicles at the same spatial location. N× N:

[0082]

[0083] Where N represents the number of vehicles, softmax(·), dim = 0 indicates normalization according to the rows of the matrix, and d k The dimension of the key matrix is ​​used to maintain matrix A. N×N The standard deviation of the standard deviation is then calculated. Then, based on the correlation matrix, the intermediate features of all cooperative vehicles are fused to obtain the output matrix O, which is calculated as follows:

[0084] O = A N×N ·V

[0085] The output matrix O is transformed and transposed, and then taken from the dimension where the car is located as the output of the intermediate feature fusion network.

[0086] (3) The decoder contains three decoder modules connected in sequence, such as Figure 6As shown, the input data of the decoder module, after pixel shuffling and random discarding, is concatenated with the output of the residual dilated convolution module in the encoder via skip connections. This method captures more semantic feature information and improves the model's prediction accuracy. Then, after random discarding, two dilated convolutional blocks with a kernel size of 3×3, and one dilated convolutional block with a kernel size of 2×2, the output is concatenated with the output of two dilated convolutional blocks with a kernel size of 3×3 to serve as the output of the decoder module.

[0087] (4) Figure 6 As shown, the segmentation head performs a 1×1 convolution on the output of the decoder module and randomly discards some data to obtain the segmentation head's output logit. Then, the pixel class classification result Ps is calculated using the following method. m :

[0088] Psm=softmax(logit),dim=1

[0089] Here, logit represents the output of the segmentation head, softmax(·), and dim=1 indicates normalization processing according to the channel dimension. Through the above processing, each point in the point cloud from the BEV perspective is predicted to have a semantic category.

[0090] This invention provides the aforementioned point cloud semantic segmentation method based on vehicle-to-vehicle cooperation. It selects one connected autonomous vehicle as the primary vehicle and the remaining vehicles as cooperative vehicles. A global coordinate transformation matrix is ​​obtained based on the pose information of each vehicle, thereby acquiring the mapping relationship between the cooperative vehicles and the primary vehicle's coordinate system. A BEV-based projection representation method is adopted to unify 3D point cloud data from different perspectives into the same BEV image viewpoint, facilitating subsequent 2D convolution processing of image data by the network and effectively reducing the computational load. The stitched original BEV features are used as input to the cooperative point cloud semantic segmentation model. Dilated convolutions with different dilation coefficients are used to extract semantic features of the receptive field, and 1×1 and 3×3 convolution kernels are used to fuse larger and smaller receptive fields, obtaining more comprehensive contextual information. The encoded intermediate features contain rich semantic features. A network based on a self-attention mechanism is used to fuse the intermediate features of different vehicles, obtaining a weighted fused feature. After the decoder restores the image resolution and the classifier performs convolution operations, the cooperative semantic segmentation result is obtained. This invention is aimed at autonomous vehicles. Most autonomous vehicles are already equipped with the radar and GPS devices required, so there is no need to add additional sensors. This reduces the processing load of onboard processing equipment and lowers costs.

[0091] While the specific embodiments of the present invention depict actions or steps in a particular order, this should be understood as requiring such actions or steps to be performed in the shown specific order or sequential order, or requiring all illustrated actions or steps to be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations. The above descriptions are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention.

[0092] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A point cloud semantic segmentation method based on vehicle-to-vehicle cooperation, characterized in that, Includes the following steps: Step S1: Collect point cloud data from multiple vehicles respectively; based on pose information, transform the point cloud data of the second vehicle to the coordinate system of the first vehicle through coordinate transformation; The first vehicle is one of the plurality of vehicles, and the second vehicle is the other vehicle among the plurality of vehicles besides the first vehicle; Step S2: Extract bird's-eye view features from the point cloud data of the second vehicle and the point cloud data of the first vehicle after coordinate transformation to obtain the bird's-eye view features of the multiple vehicles. Step S3: Construct a collaborative point cloud semantic segmentation model. Extract features from the encoder, fuse features using a self-attention mechanism, and output the segmentation head of the decoder for the bird's-eye view features of the multiple vehicles to obtain the semantic category of the point cloud data of the multiple vehicles from the bird's-eye view perspective, which is used as the collaborative point cloud semantic segmentation result. Step S3 specifically includes: Step S3-1: The bird's-eye view features of each vehicle are extracted by the encoder through the feature extractor to obtain the intermediate features of each vehicle. Step S3-2: Through the intermediate feature fusion network, the intermediate features of each vehicle are fused using a self-attention mechanism to obtain the fused features; Step S3-3: Classify the fused features using the decoder segmentation head to obtain the semantic categories of the point cloud data of the multiple vehicles; Step S3-1 specifically includes: The feature extractor is specifically a residual-connected encoder, which includes three sequentially connected global context modules and four residual dilated convolutional modules; The input data of the global context module is processed through a 1×1 dilated convolutional block, a non-linear activation function, and two 3×3 dilated convolutional blocks, and then added to the output of the non-linear activation function to serve as the output of the global context module. The input data of the residual dilated convolution module is processed by a 1×1 dilated convolution block, two 3×3 dilated convolution blocks, and a 2×2 dilated convolution block. The data is then concatenated with the outputs of the two 3×3 dilated convolution blocks, followed by a 1×1 convolution calculation. The result is then added to the output of the 1×1 dilated convolution block. Finally, the data is processed by average pooling and random discarding to obtain the output of the residual dilated convolution module.

2. The point cloud semantic segmentation method based on vehicle-to-vehicle cooperation according to claim 1, characterized in that, Step S1 specifically includes: Step S1-1: Based on the pose information of the multiple vehicles, determine the coordinate transformation matrix from the coordinate system to the world coordinate system for each of the multiple vehicles. Step S1-2: Based on the coordinate transformation matrix from the coordinate system of the second vehicle to the world coordinate system and the coordinate transformation matrix from the coordinate system of the first vehicle to the world coordinate system, obtain the coordinate transformation matrix from the coordinate system of the second vehicle to the coordinate system of the first vehicle. Steps S1-3: Based on the coordinate transformation matrix from the coordinate system of the second vehicle to the coordinate system of the first vehicle, transform the coordinates of each point in the point cloud data of the second vehicle to the coordinate system of the first vehicle.

3. The point cloud semantic segmentation method based on vehicle-to-vehicle cooperation according to claim 2, characterized in that, Step S1-1 specifically includes: Based on the roll angle, pitch angle, yaw angle, and their spatial coordinates in the world coordinate system from the vehicle's pose information, a coordinate transformation matrix from the vehicle's coordinate system to the world coordinate system is established using the following method. : in These are the roll angle, pitch angle, and yaw angle from the vehicle's pose information. , , ) represents the spatial coordinates of the vehicle in the world coordinate system.

4. The point cloud semantic segmentation method based on vehicle-to-vehicle cooperation according to claim 3, characterized in that, Step S1-2 specifically includes: The coordinate transformation matrix from the coordinate system of the second vehicle to that of the first vehicle is calculated as follows: : in, The coordinate transformation matrix from the second vehicle's coordinate system to the world coordinate system. It is the inverse of the coordinate transformation matrix from the coordinate system of the first vehicle to the world coordinate system.

5. The point cloud semantic segmentation method based on vehicle-to-vehicle cooperation according to claim 4, characterized in that, Steps S1-3 specifically include performing coordinate transformation in the following manner: in ( , ) represents a matrix The elements in Each is a matrix row and column indexes, ( , , () represents the point cloud coordinates of the second vehicle before coordinate transformation. , , ) represents the point cloud coordinates after the corresponding coordinate transformation.

6. The point cloud semantic segmentation method based on vehicle-to-vehicle cooperation according to claim 5, characterized in that, Step S2 specifically includes: (1) The point cloud coordinates in the point cloud data are projected into BEV according to the following formula: in, , The coordinates of the point cloud after coordinate transformation. This represents the pixel coordinates on the BEV image corresponding to the point cloud coordinates, where W and H represent the width and height of the BEV image, respectively. , These represent the minimum value operation and the maximum value operation, respectively. (2) The pixel coordinates of the BEV image are adjusted according to the height information of the corresponding point cloud data. The data is sorted, where n is the index of the point cloud data. The BEV images are then processed by pixel value normalization, edge pixel correction, and effective pixel filtering to obtain the BEV features of each vehicle. The BEV features include the three-dimensional coordinates, depth information, and reflectivity of the point cloud data.

7. The point cloud semantic segmentation method based on vehicle-to-vehicle cooperation according to claim 6, characterized in that, Step S3-2 specifically includes: The intermediate feature fusion network comprises three sequentially connected feature fusion modules; each feature fusion module takes the intermediate features output by the feature extractor as input, and performs operations such as shape transformation and transposition to obtain the intermediate features. Q , K , V Matrix, then calculate the incidence matrix. and output matrix : in, Indicates the number of vehicles. This indicates that the matrix is ​​normalized according to its rows. Let be the dimension of the key matrix; Q , K , V These are the query matrix, key matrix, and value matrix, respectively. Output matrix The fused features, after shape transformation and transposition, are used as intermediate features to fuse the output of the fusion network.

8. The point cloud semantic segmentation method based on vehicle-to-vehicle cooperation according to claim 7, characterized in that, Step S3-3 specifically includes: The decoder contains three decoder modules connected in sequence. The input data of the decoder modules is shuffled and randomly discarded by pixels, and then concatenated with the output of the residual dilated convolution module in the encoder through a skip connection. After random discarding, two 3×3 dilated convolution blocks and a 2×2 dilated convolution block, it is concatenated with the output of the two 3×3 dilated convolution blocks as the output of the decoder module. The segmentation head performs a 1×1 convolution on the output of the decoder module and then randomly discards some data to obtain the output of the segmentation head. Then, the pixel class classification result is calculated in the following way: in Represents the output of the segmentation head. This indicates that normalization is performed according to the channel dimension. The semantic category of point clouds.