A sparse point cloud data completion method and related equipment for autonomous driving

Abstract

The present invention provides a sparse point cloud data completion method and related equipment for autonomous driving. A first-category feature attention unit is introduced into the encoding module of the point cloud completion model. Through the joint modeling of category information and point cloud geometric features, the completion network can more accurately identify the structural features of objects of different categories during the point cloud completion process. The second-category feature attention unit in the generation module is used to dynamically adjust the features of the sparse point cloud to ensure that the model can adapt to different environments and improve the generalization ability in multiple scenarios. The upsampling unit, the third-category feature attention unit, and the sixth multi-layer perceptron in the generation module are used to gradually optimize the sparse point cloud completion process from coarse to fine, thereby improving the consistency of the completed point cloud and further enhancing the environmental perception ability of the autonomous driving system.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud completion, and in particular to a sparse point cloud data completion method for autonomous driving and related equipment. Background Art

[0002] With the rapid development of autonomous driving technology, the demand for three-dimensional environmental perception is increasing. As an important data form for three-dimensional environmental perception, power data can provide rich spatial and geometric information, which has led to the widespread application of three-dimensional point cloud data in autonomous driving. However, due to the limitations of sensors such as LiDAR or environmental occlusion, point cloud data is often sparse and incomplete, which directly affects the environmental perception and object recognition capabilities of autonomous vehicles. Therefore, there have been many explorations in the existing technology for efficient sparse point cloud data completion, which mainly include geometric interpolation-based completion methods and rule-based completion methods.

[0003] In recent years, deep learning methods have shown great potential in point cloud data completion tasks. In particular, methods based on models such as generative adversarial networks and variational autoencoders can learn the global and local features of point clouds to a certain extent, achieving high-quality completion. However, relying solely on sparse point clouds for completion still has the following problems:

[0004] 1. Insufficient global structural information makes it difficult to restore the complete environment: Since sparse point clouds have limited local information, relying solely on existing point clouds for completion in the absence of sufficient contextual information may result in incomplete point cloud structures or morphological distortion. In autonomous driving scenarios, distant vehicles or obstacles may only have a small amount of point data. Direct completion may lead to inaccurate shape reconstruction, affecting the accuracy of target detection and path planning.

[0005] 2. Limited expression of local geometric features affects completion accuracy: Point cloud data is inherently unstructured. Traditional completion methods (such as interpolation) struggle to accurately capture the geometric features of complex shapes. In sparse point clouds, local geometric features may not be sufficient to accurately infer surface details, resulting in deformation or even spurious features in the completed point cloud, which in turn affects the stability of target detection and tracking tasks.

[0006] 3. Blurred obstacle boundaries: Autonomous driving systems require high-precision environmental information for path planning and obstacle avoidance. The defects of sparse point clouds can cause blurred obstacle boundaries, making it difficult for vehicles to accurately determine the drivable area. Summary of the Invention

[0007] The present invention provides a sparse point cloud data completion method and related equipment for autonomous driving, the purpose of which is to enhance the environmental perception capability of the autonomous driving system by improving the consistency of the completed point cloud.

[0008] To achieve the above objectives, the present invention provides a sparse point cloud data completion method for autonomous driving, comprising:

[0009] Step 1: Obtain sparse point cloud for training in autonomous driving scenarios;

[0010] Step 2: Build a point cloud completion model, which includes an encoding module and a generation module;

[0011] Step 3: Use the training sparse point cloud to train the point cloud completion model, and use the point cloud completion loss and category prediction loss to optimize the parameters of the trained point cloud completion model to obtain the trained point cloud completion model;

[0012] Step 4: Input the sparse point cloud data of the target autonomous driving vehicle into the trained point cloud completion model for completion to obtain a dense point cloud of the target autonomous driving vehicle;

[0013] The encoding module includes a splicing unit, a first multi-layer perceptron, a first category feature attention unit for fusing category attention and feature attention, and a maximum pooling unit connected in sequence;

[0014] The generation module includes a second multi-layer perceptron, a second category feature attention unit for fusing category attention and feature attention, a third multi-layer perceptron, a farthest point sampling unit, a fourth multi-layer perceptron, a fifth multi-layer perceptron, a splicing unit, an upsampling unit, a third category feature attention unit for fusing category attention and feature attention, and a sixth multi-layer perceptron.

[0015] Furthermore, before using the training sparse point cloud to train the point cloud completion model, it also includes:

[0016] Calculate the point cloud centroid of the sparse point cloud used for training;

[0017] Translate all training sparse point clouds to a new coordinate system with the point cloud centroid as the origin;

[0018] Calculate the maximum distance from each point cloud in the training sparse point cloud to the point cloud centroid, and normalize all training sparse point clouds based on the maximum distance to obtain a standardized training sparse point cloud;

[0019] Use one-hot encoding to convert the category label of each point cloud in the training sparse point cloud into a category embedding vector to obtain a category embedding vector matrix;

[0020] The standardized training sparse point cloud is denoised by a local neighborhood analysis method to obtain a denoised training sparse point cloud.

[0021] More specifically, the generation module includes:

[0022] The global features are processed in sequence through the second multi-layer perceptron, the second category feature attention unit, the third multi-layer perceptron, and the farthest point sampling unit to generate a seed point cloud;

[0023] Processing the seed point cloud through the fourth multi-layer perceptron to obtain a processed seed point cloud;

[0024] The global features are processed by a fifth multi-layer perceptron to obtain processed global features;

[0025] The processed seed point cloud and the processed global features are spliced together by a splicing unit to obtain a seed point cloud feature matrix;

[0026] The upsampling unit, the third category feature attention unit, and the sixth multi-layer perceptron are used to perform cyclic optimization processing on the seed point cloud feature matrix to generate a dense point cloud.

[0027] Specifically, both the first-category feature attention unit and the second-category feature attention unit include:

[0028] Category attention layer, feature attention layer, fusion layer, residual connection layer, normalization layer, feedforward network layer;

[0029] The input ends of the category attention layer and the feature attention layer are both the input ends of the first category feature attention unit and the second category feature attention unit;

[0030] The output of the category attention layer and the output of the feature attention layer are connected to the input of the fusion layer;

[0031] The output of the fusion layer and the input of the feature attention layer are connected to the input of the residual connection layer, the output of the residual connection layer is connected to the input of the normalization layer, and the output of the normalization layer is connected to the input of the feedforward network layer;

[0032] The output end of the feedforward network layer is the output end of the first category feature attention unit and the second category feature attention unit.

[0033] More specifically, the category attention layer is used to:

[0034] After performing the inner product of the input category embedding vector matrix and performing the Softmax operation, we can get the category attention weight matrix. The calculation expression is:

[0035]

[0036] in, represents the category attention weight matrix, represents the input category embedding vector matrix, Represents the transposed category embedding vector matrix.

[0037] More specifically, the feature attention layer is used to:

[0038] Use linear transformation to transform the input point feature matrix to obtain the query matrix and key matrix;

[0039]

[0040]

[0041]

[0042] The feature attention weight matrix is obtained by transforming the query matrix and the key matrix. The expression is:

[0043]

[0044] in, represents the feature attention weight matrix, represents the query matrix, represents the bond matrix, represents the value matrix, represents the dimension of the point feature matrix, represents the query weight matrix, represents the key weight matrix, represents the value weight matrix, Represents the input point feature matrix.

[0045] Furthermore, the point cloud completion loss is used to measure the geometric difference between the generated point cloud and the real point cloud, and the calculation expression is:

[0046]

[0047] Category prediction loss is used to measure the accuracy of the model's prediction of point cloud categories. The calculation expression is:

[0048]

[0049] in, Represents the point cloud completion loss value, Indicates the number of point clouds, Represents the first Points, Represents the first Points, Represents the category prediction loss value, The model predicts the points belong to their true category probability.

[0050] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, a sparse point cloud data completion method for autonomous driving is implemented.

[0051] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, a sparse point cloud data completion method for autonomous driving is implemented.

[0052] The above solution of the present invention has the following beneficial effects:

[0053] The present invention obtains sparse point clouds for training in an autonomous driving scenario; constructs a point cloud completion model including an encoding module and a generation module; uses the sparse point clouds for training to train the point cloud completion model, and uses the point cloud completion loss and the category prediction loss to optimize the parameters of the trained point cloud completion model to obtain a trained point cloud completion model; inputs the sparse point cloud data of the target autonomous driving vehicle into the trained point cloud completion model for completion to obtain a dense point cloud of the target autonomous driving vehicle; the encoding module includes a splicing unit, a first multi-layer perceptron, a first category feature attention unit, and a maximum pooling unit connected in sequence; the generation module includes a second multi-layer perceptron, a second category feature attention unit for fusing category attention and feature attention, a third multi-layer perceptron, a farthest point sampling unit, a fourth multi-layer perceptron, a fifth multi-layer perceptron, a splicing unit, and a plurality of other units. Unit, upsampling unit, third category feature attention unit for fusing category attention and feature attention, sixth multi-layer perceptron; compared with the prior art, the present invention introduces the first category feature attention unit in the encoding module of the point cloud completion model, and through the joint modeling of category information and point cloud geometric features, the completion network can more accurately identify the structural features of objects of different categories in the process of completing the point cloud, and dynamically adjust the features of the sparse point cloud through the second category feature attention unit in the generation module to ensure that the model can adapt to different environments and improve the generalization ability in multiple scenarios, and through the upsampling unit, the third category feature attention unit and the sixth multi-layer perceptron in the generation module, the completion process of the sparse point cloud is gradually optimized from coarse to fine, thereby improving the consistency of the completed point cloud, and then improving the environmental perception ability of the autonomous driving system.

[0054] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0056] Figure 2Schematic diagram of the structure of the point cloud completion model in an embodiment of the present invention;

[0057] Figure 3 Schematic diagram of the structure of the category feature attention unit in an embodiment of the present invention;

[0058] Figure 4 Schematic diagram of the structure of the terminal device in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] To make the technical problems, technical solutions, and advantages to be solved by the present invention more clear, the following is a detailed description with reference to the accompanying drawings and specific embodiments. It is obvious that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0060] In the description of the present invention, it should be noted that the terms "first", "second" and "third" are only used for descriptive purposes and should not be understood as indicating or implying relative importance.

[0061] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to a locking connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0062] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0063] In response to existing problems, the present invention provides a sparse point cloud data completion method and related equipment for autonomous driving.

[0064] like Figure 1 、 Figure 2 As shown, an embodiment of the present invention provides a sparse point cloud data completion method for autonomous driving, comprising:

[0065] Step 1: Obtain sparse point cloud for training in autonomous driving scenarios;

[0066] Step 2: Build a point cloud completion model, which includes an encoding module and a generation module;

[0067] Step 3: Use the training sparse point cloud to train the point cloud completion model, and use the point cloud completion loss and category prediction loss to optimize the parameters of the trained point cloud completion model to obtain the trained point cloud completion model;

[0068] Step 4: Input the sparse point cloud data of the target autonomous driving vehicle into the trained point cloud completion model for completion to obtain a dense point cloud of the target autonomous driving vehicle.

[0069] To improve the consistency and quality of model input, embodiments of the present invention require preprocessing of the acquired training sparse point cloud to ensure normalization of input data and effective utilization of category information. Therefore, before training the point cloud completion model using the training sparse point cloud, the following steps are also included:

[0070] Calculate the point cloud centroid of the sparse point cloud used for training. The calculation formula is:

[0071]

[0072] in, Represents the sparse point cloud used for training The point cloud centroid, represents the number of sparse point clouds used for training, Represents the sparse point cloud used for training The point clouds;

[0073] All training sparse point clouds are translated to a new coordinate system with the point cloud centroid as the origin. The translation formula is:

[0074]

[0075] in, Represents the first point in the sparse point cloud used for training in the new coordinate system point clouds;

[0076] Calculate the maximum distance from each point cloud to the point cloud centroid in the training sparse point cloud. The calculation expression is:

[0077]

[0078] in, Indicates the maximum distance, Represents the first point in the sparse point cloud used for training in the new coordinate system The distance from a point cloud to the point cloud centroid, , Represents the first point in the sparse point cloud used for training in the new coordinate system The coordinates of the point cloud;

[0079] Based on the maximum distance, all training sparse point clouds are normalized, and all training sparse point clouds are scaled to the unit range to obtain the standardized training sparse point clouds. The normalization expression is:

[0080]

[0081] in, Represents the sparse point cloud for training after normalization The point clouds, ;

[0082] Use one-hot encoding to convert the category label of each point cloud in the training sparse point cloud into a category embedding vector to obtain the category embedding vector matrix ;

[0083] The standardized training sparse point cloud is denoised by a local neighborhood analysis method to obtain a denoised training sparse point cloud.

[0084] Since removing outliers and noise points is an important step in data preprocessing, especially when processing point cloud data, in this embodiment of the present invention, the sparse point cloud for training after normalization is denoised using a local neighborhood analysis method. The specific process is as follows:

[0085] For each point cloud in the standardized training sparse point cloud, the calculation expression is:

[0086]

[0087] in, Represents the sparse point cloud for training after normalization The Point cloud and The distance between point clouds, Represents the sparse point cloud for training after normalization The The coordinates of the point cloud;

[0088] From the sparse point cloud after normalization Select the distance The k nearest point clouds form a k-nearest neighbor point set ;

[0089] Use principal component analysis to calculate the normalized sparse point cloud for training The Point Cloud The local normal vector of ,get:

[0090]

[0091] Calculate the normalized sparse point cloud for training The Point Cloud Angular deviation from its neighboring points:

[0092]

[0093] in, represents the angle between two points, Represents the norm of the local normal vector.

[0094] If the training is done with sparse point clouds after normalization The Point Cloud If the angle deviation of a point is significantly greater than the average deviation of its neighboring points, it is considered an outlier and is removed.

[0095] In an embodiment of the present invention, the main task of the encoding module in the point cloud completion model is to extract global features from the input sparse point cloud and enhance the understanding of the point cloud context in combination with category information.

[0096] Specifically, the encoding module includes a splicing unit, a first multi-layer perceptron, a first category feature attention unit for fusing category attention and feature attention, and a maximum pooling unit connected in sequence;

[0097] The first multi-layer perceptron performs preliminary feature extraction on the input sparse point cloud to obtain the point feature matrix, which is expressed as:

[0098]

[0099] in, represents the point feature matrix, represents multi-layer perceptual processing, represents the field of real numbers, Represents the feature dimension. The input layer of the first multi-layer perceptron receives the preprocessed sparse point cloud. The hidden layer of the first multi-layer perceptron uses multiple fully connected layers for feature extraction, and then outputs the point feature matrix through the output layer ;

[0100] The point feature matrix and the category embedding vector matrix are input into the first category feature attention unit for category attention and feature attention fusion to obtain the updated point feature matrix, which is expressed as:

[0101]

[0102] in, represents the updated point feature matrix, represents the fusion operation, Represents the category embedding vector matrix;

[0103] The updated point feature matrix is input into the maximum pooling unit for maximum pooling operation to generate global features. The global features contain the local geometric structure and category distribution information of the point cloud, providing context-aware shape encoding for subsequent generation tasks. The expression is:

[0104]

[0105] in, Represents global features, Represents the maximum pooling operation.

[0106] In an embodiment of the present invention, the generation module in the point cloud completion model gradually generates a high-quality dense point cloud based on the global features extracted by the encoding module.

[0107] Specifically, the generation module includes a second multi-layer perceptron, a second category feature attention unit for fusing category attention and feature attention, a third multi-layer perceptron, a farthest point sampling unit, a fourth multi-layer perceptron, a fifth multi-layer perceptron, a splicing unit, an upsampling unit, a third category feature attention unit for fusing category attention and feature attention, and a sixth multi-layer perceptron;

[0108] The global features are processed in sequence through the second multi-layer perceptron, the second category feature attention unit, the third multi-layer perceptron, and the farthest point sampling unit to generate a seed point cloud;

[0109] Processing the seed point cloud through the fourth multi-layer perceptron to obtain a processed seed point cloud;

[0110] The global features are processed by a fifth multi-layer perceptron to obtain processed global features;

[0111] The processed seed point cloud and the processed global features are spliced together by a splicing unit to obtain a seed point cloud feature matrix;

[0112] The upsampling unit, the third category feature attention unit, and the sixth multi-layer perceptron are used to perform cyclic optimization processing on the seed point cloud feature matrix to generate a dense point cloud.

[0113] In the embodiment of the present invention, the generation module first converts the global features into Mapped to the initial point feature, the expression is:

[0114]

[0115] in, represents the initial point feature, Represents the mapping operation of the second multi-layer perceptron;

[0116] Then the category attention and feature attention are fused through the second category feature attention unit:

[0117]

[0118] in, is the feature matrix after being enhanced by the second category feature attention unit;

[0119] The features are then mapped to three-dimensional space through the third multi-layer perceptron to generate the initial seed point cloud:

[0120]

[0121] in, represents the initial seed point cloud, Represents the mapping operation of the third multi-layer perceptron;

[0122] The spatial distribution of the initial seed point cloud is optimized by the farthest point sampling unit to obtain the seed point cloud:

[0123]

[0124] in, represents the seed point cloud, Indicates the farthest point sampling process;

[0125] The seed point cloud and the global features are processed by the fourth and fifth multi-layer perceptrons and then spliced to generate the seed point cloud feature matrix:

[0126]

[0127]

[0128]

[0129] in, and They are used to convert seed point clouds and global features into matching feature matrices, Represents feature cascade operation, used for merging and ;

[0130] The upsampling unit is used to extract the features of the spliced feature matrix to obtain the features of the seed point cloud feature matrix. :

[0131]

[0132] in, is a linear transformation matrix used to expand the feature dimension, is the number of points after upsampling;

[0133] The upsampled features are then dynamically adjusted through the third category feature attention unit:

[0134]

[0135] The adjusted features are processed by the sixth multi-layer perceptron Inverse mapping to three-dimensional space to generate the initial dense point cloud :

[0136]

[0137] Repeat the above upsampling, dynamic adjustment and inverse mapping steps L times for the initial dense point cloud, gradually refine the point cloud resolution, and finally output a complete and high-density dense point cloud :

[0138]

[0139] in, Indicates the number of upsampling layers. The progressive generation method can avoid the blurring or overfitting problems caused by single upsampling to a certain extent.

[0140] According to the above process, the output of the generator is a complete dense point cloud, which not only retains the global shape information but also has rich local details and category perception capabilities.

[0141] Most preferably, the first category feature attention unit and the second category feature attention unit are designed to enhance the model's ability to understand point cloud data by dynamically weighing category information and local geometric structure information, such as Figure 3 As shown, all include:

[0142] Category attention layer, feature attention layer, fusion layer, residual connection layer, normalization layer, feedforward network layer;

[0143] The input ends of the category attention layer and the feature attention layer are both the input ends of the first category feature attention unit and the second category feature attention unit;

[0144] The output of the category attention layer and the output of the feature attention layer are connected to the input of the fusion layer;

[0145] The output of the fusion layer and the input of the feature attention layer are connected to the input of the residual connection layer, the output of the residual connection layer is connected to the input of the normalization layer, and the output of the normalization layer is connected to the input of the feedforward network layer;

[0146] The output end of the feedforward network layer is the output end of the first category feature attention unit and the second category feature attention unit.

[0147] In the embodiment of the present invention, the category feature attention unit can receive two inputs: a point feature matrix and a category embedding vector matrix. The point feature matrix is obtained by feature extraction of preprocessed point cloud data, and the category embedding vector matrix is obtained by feature extraction of preprocessed point cloud data. It is obtained after one-hot encoding.

[0148] Specifically, the category attention layer is used to:

[0149] After performing the inner product of the input category embedding vector matrix and performing the Softmax operation, we can get the category attention weight matrix. The calculation expression is:

[0150]

[0151] in, represents the category attention weight matrix, represents the input category embedding vector matrix, Represents the transposed category embedding vector matrix.

[0152] Specifically, the feature attention layer is used to:

[0153] Use linear transformation to transform the input point feature matrix to obtain the query matrix and key matrix;

[0154]

[0155]

[0156]

[0157] The feature attention weight matrix is obtained by transforming the query matrix and the key matrix. The expression is:

[0158]

[0159] in, represents the feature attention weight matrix, represents the query matrix, represents the bond matrix, represents the value matrix, represents the dimension of the point feature matrix, represents the query weight matrix, represents the key weight matrix, represents the value weight matrix, represents the input point feature matrix, The purpose of the operation is to prevent numerical overflow.

[0160] Specifically, the fusion layer is used to weightedly fuse the category attention weight matrix and the feature attention weight matrix to generate a comprehensive attention weight matrix, which is expressed as:

[0161]

[0162] in, represents the comprehensive attention weight matrix, Represents a hyperparameter used to weigh the contribution of categories and features;

[0163] The fusion layer is also used to fuse the value matrix using the comprehensive attention weight matrix to generate the updated point feature matrix, which is expressed as:

[0164]

[0165] in, represents the updated point feature matrix, Represents a matrix of values.

[0166] Specifically, the residual connection layer is used to perform residual connection on the updated point feature matrix and the point feature matrix, and the normalization layer is used to normalize the point feature matrix after residual connection to enhance the stability of the model. The expression is:

[0167]

[0168] in, represents the normalized point feature matrix, Represents a normalization operation.

[0169] Specifically, the feedforward network layer is used to enhance the feature representation capability, and the expression is:

[0170]

[0171] in, Represents the output of the category feature attention unit, Indicates an enhancement operation, represents the activation function, 、 represents the weight matrix of the feedforward network, 、 Represents the bias vector.

[0172] The goal of optimizing the parameters of the trained point cloud completion model using point cloud completion loss and category prediction loss in an embodiment of the present invention is to improve the completion quality and classification accuracy of the model by minimizing the difference between the generated point cloud and the real point cloud while combining category prediction information.

[0173] Specifically, the point cloud completion loss is used to measure the geometric difference between the generated point cloud and the real point cloud, and the calculation expression is:

[0174]

[0175] Category prediction loss is used to measure the accuracy of the model's prediction of point cloud categories. The calculation expression is:

[0176]

[0177] in, Represents the point cloud completion loss value, Indicates the number of point clouds, Represents the first Points, Represents the first Points, Represents the category prediction loss value, The model predicts the points belong to their true category probability.

[0178] The comprehensive loss function is a weighted combination of the above two losses to form the final optimization goal, ensuring that the model maintains the accuracy of category prediction while completing the point cloud. The expression is:

[0179]

[0180] in, represents the comprehensive loss function, represents an adjustable hyperparameter, and .

[0181] The embodiment of the present invention obtains a sparse point cloud for training in an autonomous driving scenario; constructs a point cloud completion model including an encoding module and a generation module; uses the sparse point cloud for training to train the point cloud completion model, and uses the point cloud completion loss and the category prediction loss to optimize the parameters of the trained point cloud completion model to obtain a trained point cloud completion model; inputs the sparse point cloud data of the target autonomous driving vehicle into the trained point cloud completion model for completion to obtain a dense point cloud of the target autonomous driving vehicle; the encoding module includes a splicing unit, a first multi-layer perceptron, a first category feature attention unit, and a maximum pooling unit connected in sequence; the generation module includes a second multi-layer perceptron, a second category feature attention unit for fusing category attention and feature attention, a third multi-layer perceptron, a farthest point sampling unit, a fourth multi-layer perceptron, a fifth multi-layer perceptron, a splicing unit, and a fusion unit. Unit, upsampling unit, third category feature attention unit for fusing category attention and feature attention, sixth multi-layer perceptron; compared with the prior art, the embodiment of the present invention introduces the first category feature attention unit in the encoding module of the point cloud completion model, and through the joint modeling of category information and point cloud geometric features, the completion network can more accurately identify the structural features of objects of different categories in the process of completing the point cloud, and dynamically adjust the features of the sparse point cloud through the second category feature attention unit in the generation module to ensure that the model can adapt to different environments and improve the generalization ability in multiple scenarios, and through the upsampling unit, the third category feature attention unit and the sixth multi-layer perceptron in the generation module, the completion process of the sparse point cloud is gradually optimized from coarse to fine, thereby improving the consistency of the completed point cloud, and then improving the environmental perception ability of the autonomous driving system.

[0182] The embodiment of the present invention further provides a terminal device, such as Figure 4 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 4 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the above-mentioned sparse point cloud data completion method for autonomous driving is implemented.

[0183] The terminal device D10 can be a computing device such as a desktop computer, a notebook, a PDA, a server, a server cluster, or a cloud server. The terminal device may include, but is not limited to, a processor D100 and a memory D101. Those skilled in the art will understand that Figure 4 This is merely an example of the terminal device D10 and does not constitute a limitation on the terminal device D10 . The terminal device D10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device D10 may also include input and output devices, network access devices, etc.

[0184] The processor D100 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0185] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device D10. Furthermore, the memory D101 may include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory D101 may also be used to temporarily store data that has been output or is about to be output.

[0186] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0188] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, a sparse point cloud data completion method for autonomous driving is implemented.

[0189] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a construction device / terminal device, a recording medium, computer memory, read-only memory (ROM), random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Examples include a USB flash drive, a removable hard drive, a magnetic disk, or an optical disk.

[0190] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A sparse point cloud data completion method for autonomous driving, characterized by: include: Step 1: Obtain sparse point cloud for training in autonomous driving scenarios; Step 2: construct a point cloud completion model, which includes an encoding module and a generation module; Step 3: training the point cloud completion model using the training sparse point cloud, and optimizing the parameters of the trained point cloud completion model using the point cloud completion loss and the category prediction loss to obtain a trained point cloud completion model; Step 4: Input the sparse point cloud data of the target autonomous driving vehicle into the trained point cloud completion model for completion to obtain a dense point cloud of the target autonomous driving vehicle; The encoding module includes a splicing unit, a first multi-layer perceptron, a first category feature attention unit for fusing category attention and feature attention, and a maximum pooling unit connected in sequence; The generation module includes a second multi-layer perceptron, a second category feature attention unit for fusing category attention and feature attention, a third multi-layer perceptron, a farthest point sampling unit, a fourth multi-layer perceptron, a fifth multi-layer perceptron, a splicing unit, an upsampling unit, a third category feature attention unit for fusing category attention and feature attention, and a sixth multi-layer perceptron; The first category feature attention unit and the second category feature attention unit both include: Category attention layer, feature attention layer, fusion layer, residual connection layer, normalization layer, feedforward network layer; The input end of the category attention layer and the input end of the feature attention layer are both the input ends of the first category feature attention unit and the second category feature attention unit; The output end of the category attention layer and the output end of the feature attention layer are both connected to the input end of the fusion layer; The output end of the fusion layer and the input end of the feature attention layer are both connected to the input end of the residual connection layer, the output end of the residual connection layer is connected to the input end of the normalization layer, and the output end of the normalization layer is connected to the input end of the feedforward network layer; The output end of the feedforward network layer is the output end of the first category feature attention unit and the second category feature attention unit.

2. The sparse point cloud data completion method for autonomous driving according to claim 1, characterized in that: Before training the point cloud completion model using the training sparse point cloud, the method further includes: Calculating the point cloud centroid of the training sparse point cloud; Translate all training sparse point clouds to a new coordinate system with the centroid of the point cloud as the origin; Calculating the maximum distance between each point cloud in the training sparse point cloud and the centroid of the point cloud, and normalizing all the training sparse point clouds based on the maximum distance to obtain a normalized training sparse point cloud; Use one-hot encoding to convert the category label of each point cloud in the training sparse point cloud into a category embedding vector to obtain a category embedding vector matrix; The standardized training sparse point cloud is denoised by a local neighborhood analysis method to obtain a denoised training sparse point cloud.

3. The sparse point cloud data completion method for autonomous driving according to claim 2, characterized in that: The encoding module includes: Performing preliminary feature extraction on the input sparse point cloud through the first multi-layer perceptron to obtain a point feature matrix; Inputting the point feature matrix and the category embedding vector matrix into the first category feature attention unit to perform category attention and feature attention fusion to obtain an updated point feature matrix; The updated point feature matrix is input into the maximum pooling unit to perform a maximum pooling operation to generate global features.

4. The sparse point cloud data completion method for autonomous driving according to claim 3, characterized in that: The generation module includes: Processing the global features sequentially through the second multi-layer perceptron, the second category feature attention unit, the third multi-layer perceptron, and the farthest point sampling unit to generate a seed point cloud; Processing the seed point cloud by the fourth multi-layer perceptron to obtain a processed seed point cloud; Processing the global feature by the fifth multi-layer perceptron to obtain a processed global feature; The processed seed point cloud and the processed global features are spliced by the splicing unit to obtain a seed point cloud feature matrix; The upsampling unit, the third category feature attention unit, and the sixth multi-layer perceptron are used to perform cyclic optimization processing on the seed point cloud feature matrix to generate a dense point cloud.

5. The sparse point cloud data completion method for autonomous driving according to claim 1, characterized in that: The category attention layer is used to: After performing the inner product of the input category embedding vector matrix and performing the Softmax operation, we can get the category attention weight matrix. The calculation expression is: in, represents the category attention weight matrix, represents the input category embedding vector matrix, Represents the transposed category embedding vector matrix.

6. The sparse point cloud data completion method for autonomous driving according to claim 1, characterized in that: The feature attention layer is used to: Use linear transformation to transform the input point feature matrix to obtain the query matrix and key matrix; The query matrix and the key matrix are used to transform the feature attention weight matrix, which is expressed as: in, represents the feature attention weight matrix, represents the query matrix, represents the bond matrix, represents the value matrix, represents the dimension of the point feature matrix, represents the query weight matrix, represents the key weight matrix, represents the value weight matrix, Represents the input point feature matrix.

7. The sparse point cloud data completion method for autonomous driving according to claim 1, characterized in that: The point cloud completion loss is used to measure the geometric difference between the generated point cloud and the real point cloud. The calculation expression is: The category prediction loss is used to measure the accuracy of the model's prediction of point cloud categories, and the calculation expression is: in, Represents the point cloud completion loss value, Indicates the number of point clouds, Represents the first Points, Represents the first Points, Represents the category prediction loss value, The model predicts the points belong to their true category probability.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the sparse point cloud data completion method for autonomous driving is implemented as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the sparse point cloud data completion method for autonomous driving is implemented as described in any one of claims 1 to 7.

Images