A method, apparatus, device and medium for point cloud completion

Abstract

This invention provides a method, apparatus, device, and readable medium for point cloud completion. The method includes: calculating a first high-dimensional semantic feature of point cloud data collected by a sensor, and calculating a global semantic feature of the first high-dimensional semantic feature; calculating the features of point clusters after clustering the point cloud data, and calculating a second high-dimensional semantic feature of the point cluster features; concatenating the second high-dimensional semantic feature with the global semantic feature to obtain a fused feature map; calculating the high-dimensional semantic feature of the fused feature map, and decoding the high-dimensional semantic feature to obtain the completed point cloud. By using the solution of this invention, the computational efficiency of the model can be significantly improved, the memory space requirements of the computing device can be reduced, the number of model parameters and computational load can be reduced, and the accuracy and robustness of point cloud completion can be improved.

Description

Technical Field

[0001] This invention relates to the field of point cloud completion, and more specifically to a method, apparatus, device, and readable medium for point cloud completion. Background Technology

[0002] In the fields of vehicle-to-everything (V2X) and autonomous driving, point cloud data collected by sensors such as LiDAR and millimeter-wave radar are commonly used for 3D object detection. However, these sensors are often unable to capture complete data about the target object due to factors such as occlusion by external obstacles, self-occlusion by the target object, and blind spots during data acquisition. Furthermore, as the distance between the acquisition device and the target object increases, the point cloud data acquired by these devices becomes sparser. Performing 3D object detection on this incomplete and sparse point cloud data significantly reduces the model's accuracy and recall, which seriously affects the safety of autonomous driving and V2X vehicles. Existing point cloud completion technologies suffer from poor accuracy, high memory requirements, and slow computation speeds. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a method, apparatus, device and readable medium for point cloud completion. By using the technical solution of this invention, the model calculation efficiency can be greatly improved, the memory space requirements of the computing device can be reduced, the number of model parameters and the amount of calculation can be reduced, and the accuracy and robustness of point cloud completion can be improved.

[0004] To achieve the above objectives, one aspect of the present invention provides a point cloud completion method, comprising the following steps:

[0005] Calculate the first high-dimensional semantic features of the point cloud data collected by the sensor, and calculate the global semantic features of the first high-dimensional semantic features;

[0006] Calculate the features of point clusters after clustering point cloud data, and calculate the second high-dimensional semantic features of the point clusters.

[0007] The second high-dimensional semantic features are concatenated with the global semantic features to obtain a fused feature map;

[0008] Calculate the third high-dimensional semantic feature of the fused feature map, and decode the third high-dimensional semantic feature to obtain the completed point cloud.

[0009] According to one embodiment of the present invention, calculating the first high-dimensional semantic feature of the point cloud data acquired by the sensor includes:

[0010] Point cloud data is input into a shared multilayer perceptron for computation to obtain the first high-dimensional semantic features.

[0011] According to one embodiment of the present invention, calculating the global semantic features of the first high-dimensional semantic feature includes:

[0012] Max pooling is performed on the first high-dimensional semantic features to obtain the global semantic features.

[0013] According to one embodiment of the present invention, calculating the features of point clusters after clustering point cloud data includes:

[0014] Clustering is performed on the point cloud data to obtain K point clusters, where K is greater than 1 and less than N / 2.

[0015] According to one embodiment of the present invention, calculating the features of point clusters after clustering point cloud data includes:

[0016] Calculate the average vector of the feature vectors in each of the K point clusters, and use the average vector of the feature vectors in each point cluster as the feature of each of the K point clusters.

[0017] According to one embodiment of the present invention, the second high-dimensional semantic feature for calculating the features of a point cluster includes:

[0018] The features of K point clusters are input into the first multilayer perceptron for computation to obtain the second high-dimensional semantic features.

[0019] According to one embodiment of the present invention, concatenating the second high-dimensional semantic features with the global semantic features to obtain a fused feature map includes:

[0020] The first global semantic feature is obtained by repeating the global semantic feature K times.

[0021] The second high-dimensional semantic feature is concatenated with the first global semantic feature to obtain the fused feature;

[0022] The fused features are input into the second multilayer perceptron for computation to obtain the fused feature map.

[0023] According to one embodiment of the present invention, calculating the third high-dimensional semantic feature of the fused feature map includes:

[0024] The third high-dimensional semantic feature is obtained by max pooling the fused feature map.

[0025] According to one embodiment of the present invention, decoding the third high-dimensional semantic features to obtain the completed point cloud includes:

[0026] The third high-dimensional semantic feature is input into a multilayer perceptron for decoding and computation to obtain the completed points.

[0027] Another aspect of the present invention provides a point cloud completion apparatus, the apparatus comprising:

[0028] The first computing module is configured to calculate the first high-dimensional semantic features of the point cloud data collected by the sensor, and to calculate the global semantic features of the first high-dimensional semantic features.

[0029] The second calculation module is configured to calculate the features of point clusters after clustering point cloud data, and to calculate the second high-dimensional semantic features of the point cluster features.

[0030] The splicing module is configured to splice the second high-dimensional semantic features with the global semantic features to obtain a fused feature map;

[0031] The third calculation module is configured to calculate the high-dimensional semantic features of the fused feature map, and decode the high-dimensional semantic features to obtain the completed point cloud.

[0032] Another aspect of the embodiments of the present invention also provides a computer device, the computer device comprising:

[0033] At least one processor; and

[0034] The memory stores computer instructions that can be executed by a processor, which, when executed by the processor, implement the steps of any of the methods described above.

[0035] In another aspect, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0036] The present invention has the following beneficial technical effects: The point cloud completion method provided in the embodiments of the present invention calculates the first high-dimensional semantic features of the point cloud data collected by the sensor, and calculates the global semantic features of the first high-dimensional semantic features; calculates the features of the point clusters after the point cloud data is clustered, and calculates the second high-dimensional semantic features of the point cluster features; concatenates the second high-dimensional semantic features with the global semantic features to obtain a fused feature map; calculates the high-dimensional semantic features of the fused feature map, and decodes the high-dimensional semantic features to obtain the completed point cloud. This technical solution can significantly improve the model calculation efficiency, reduce the memory space requirements of the computing device, reduce the number of model parameters and the amount of calculation, and improve the accuracy and robustness of point cloud completion. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic flowchart of a point cloud completion method according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of a point cloud completion process according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of an encode module according to an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of a point cloud completion device according to an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of a computer device according to an embodiment of the present invention;

[0043] Figure 6 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0045] Based on the above objectives, a first aspect of the embodiments of the present invention provides an embodiment of a point cloud completion method. Figure 1 The diagram shown is a schematic flowchart of the method.

[0046] like Figure 1 As shown, the method may include the following steps:

[0047] S1 calculates the first high-dimensional semantic feature of the point cloud data collected by the sensor, and then calculates the global semantic feature of the first high-dimensional semantic feature. Point cloud data of vehicles, objects, or environments can be collected by sensors such as radar and millimeter-wave radar. The collected N*3 point cloud data is input into a shared multilayer perceptron for calculation, which yields N*512 first high-dimensional semantic features, where N is the number of point clouds collected by the sensor, and 3 represents the coordinates (x, y, z) of a point in the point cloud. The shared multilayer perceptron consists of six multilayer perceptrons with three hidden layers each, such as... Figure 2 As shown, the feature vector lengths of the six multilayer perceptrons are 256, 512, 1024, 2048, 1024, and 512, respectively. Other vector lengths can also be used as needed. Then, the first high-dimensional semantic features of N*512 are max-pooled to obtain global semantic features of 1*512. This calculation process is performed in the auxiliary branch, as shown... Figure 3As shown, the branch containing the auxiliary head serves as an auxiliary branch, used to supervise the learning of the intermediate semantic feature layer, improve the richness and representational ability of the global semantic information obtained by the intermediate semantic feature layer, improve the accuracy of point cloud completion, and reduce the error rate of point cloud completion.

[0048] S2 calculates the features of point clusters after clustering point cloud data, and then calculates the second high-dimensional semantic features of the point clusters. For example... Figure 3 As shown, the N*3 point cloud data is clustered to obtain key point feature data for K point clusters, where N is the number of point clouds acquired by the sensor, 3 represents the coordinates (x, y, z) of a point in the point cloud, and K is greater than 1 and less than N / 2. The value range of K can be set according to requirements. In some embodiments, the average vector of the feature vectors in each of the K point clusters can also be calculated, and the average vector of the feature vectors in each point cluster can be used as the feature of each of the K point clusters. This can improve the model calculation speed and reduce the memory requirements of the hardware device. For example, if there are 100 points in the point cloud data, after clustering, 5 point clusters are obtained, that is, each point cluster has 20 points. The average vector of the feature vectors of the 20 points in the first point cluster is calculated, and this average vector is used to represent the feature of the first point cluster, and so on. Then, the key point data of the K point clusters are input into a three-layer first multilayer perceptron for calculation, which can obtain the second high-dimensional semantic features of K*512.

[0049] S3 concatenates the second high-dimensional semantic feature with the global semantic feature to obtain a fused feature map. Repeating the above 1*512 global semantic feature K times yields a K*512 global semantic feature. Concatenating the K*512 second high-dimensional semantic feature with the K*512 global semantic feature yields a K*1024 fused feature. Then, the K*1024 fused feature is input into a three-layer second multilayer perceptron for calculation, resulting in a K*2048 fused feature map.

[0050] S4 calculates the high-dimensional semantic features of the fused feature map, and decodes the high-dimensional semantic features to obtain the completed point cloud. The K*2048 fused feature map obtained above is then subjected to max pooling to obtain a 1*2048 high-dimensional semantic feature. This 1*2048 high-dimensional semantic feature is then input into a multilayer perceptron containing five hidden layers for decoding to obtain the completed point cloud.

[0051] This invention completes incomplete and sparse point cloud data acquired by LiDAR and millimeter-wave radar. The point cloud data acquired by the point cloud acquisition device is input into the main branch and the auxiliary branch for calculation. The auxiliary branch uses all point cloud data information to extract global semantic features. The main branch preprocesses the input point cloud and uses the key points obtained after preprocessing to replace the original point cloud for calculation. The prediction result of the main branch is output as the final result of point cloud completion. This invention solves the problems of existing point cloud completion models, such as many model parameters, slow calculation speed, high memory space requirements of computing devices, and poor completion effect.

[0052] By using the technical solution of the present invention, the model calculation efficiency can be greatly improved, the memory space requirements of computing devices can be reduced, the number of model parameters and the amount of calculation can be reduced, and the accuracy and robustness of point cloud completion can be improved.

[0053] In a preferred embodiment of the present invention, the first high-dimensional semantic feature of the point cloud data collected by the sensor includes:

[0054] The N*3 point cloud data is input into a shared multilayer perceptron for computation to obtain the first high-dimensional semantic features of N*512, where N is the number of point clouds collected by the sensor.

[0055] In a preferred embodiment of the present invention, the shared multilayer perceptron includes six multilayer perceptrons, each containing three hidden layers. The feature vector lengths of the six multilayer perceptrons are 256, 512, 1024, 2048, 1024, and 512, respectively. The collected N*3 point cloud data is input into the encoding module in the auxiliary branch for feature extraction, where N is the number of missing or sparse point clouds collected by the point cloud acquisition device, and 3 represents the coordinates (x, y, z) of a point in the point cloud. The internal structure of the encoding module is as follows... Figure 2 As shown, the input point cloud is sequentially processed by six shared multilayer perceptrons (MLPs) containing three hidden layers. The MLPs share weight parameters, which can improve computational efficiency and reduce the number of model parameters. The feature map size output by the encoder module after processing by the MLPs with shared weight parameters is F = N*V, where V represents the length of the vector, with values ​​of {256, 512, 1024, 2048, 1024, 512}. That is, the input N*3 point cloud data is first upgraded and then reduced in dimensionality to obtain N*512 high-dimensional semantic features. The vector length can be selected according to the computational needs.

[0056] In a preferred embodiment of the present invention, the global semantic features for calculating the first high-dimensional semantic feature include:

[0057] Max pooling is performed on the first high-dimensional semantic feature of N*512 to obtain a global semantic feature of 1*512. Point-wise max pooling is then performed on the N*512 high-dimensional semantic feature output by the encoder module to obtain a global semantic feature of 1*512 containing information from all points.

[0058] In a preferred embodiment of the present invention, the key point data for calculating point cloud data includes:

[0059] Clustering is performed on N*3 point cloud data to obtain keypoint data for K point clusters, where N is the number of point clouds collected by the sensor, and K is greater than 1 and less than N / 2. The collected N*3 point cloud data is input into the main branch, and clustering is performed on the N*3 point cloud data to obtain keypoints for K point clusters. The value of K is set to the range (1, N / 2), or it can be set according to the actual situation.

[0060] In a preferred embodiment of the present invention, it further includes:

[0061] The average vector of feature vectors in each of the K point clusters is calculated, and this average vector is used as the feature of each of the K point clusters. The average vector of feature vectors in each of the K point clusters is then used as the feature of the key point, representing the feature of that point cluster. This improves the model's computation speed and reduces the memory requirements of hardware devices. For example, if there are 100 points in the point cloud data, after clustering, 5 point clusters are obtained, with 20 points in each cluster. The average vector of feature vectors of the 20 points in the first cluster is calculated, and this average vector represents the feature of the first cluster. This process continues, and ultimately, the feature vectors of the 100 points in the point cloud data are represented by the feature vectors of the 5 key points.

[0062] In a preferred embodiment of the present invention, calculating the second high-dimensional semantic features of the key point data includes:

[0063] Keypoint data from K point clusters are input into a three-layer first multilayer perceptron for computation to obtain a second high-dimensional semantic feature of K*512. The feature values ​​of the K keypoints are then input into a three-layer MLP for computation to obtain the second high-dimensional semantic feature of K*512.

[0064] In a preferred embodiment of the present invention, concatenating the second high-dimensional semantic features with the global semantic features to obtain a fused feature map includes:

[0065] Repeat the 1*512 global semantic features K times to obtain K*512 global semantic features;

[0066] The second high-dimensional semantic feature of K*512 is concatenated with the global semantic feature of K*512 to obtain the fused feature of K*1024;

[0067] The K*1024 fused features are input into a three-layer second multilayer perceptron for computation to obtain a K*2048 fused feature map. The 1*512 global semantic features obtained from the auxiliary branch are repeated K times to obtain K*512 global semantic features. Then, the second high-dimensional semantic features of K*512 are concatenated with the K*512 global semantic features to obtain K*1024 fused features. These K*1024 fused features are then input into a three-layer second multilayer perceptron for computation to obtain a K*2048 fused feature map. The parameters of the first and second multilayer perceptrons can be the same or different; this invention does not impose any restrictions.

[0068] In a preferred embodiment of the present invention, calculating the high-dimensional semantic features of the fused feature map includes:

[0069] Max pooling is performed on the K*2048 fused feature map to obtain a 1*2048 high-dimensional semantic feature. Max pooling of the aforementioned K*2048 fused feature map yields a 1*2048 high-dimensional semantic feature that includes both local and global point cloud information.

[0070] In a preferred embodiment of the present invention, decoding high-dimensional semantic features to obtain the completed point cloud includes:

[0071] The 1*2048 high-dimensional semantic features are input into a multilayer perceptron containing five hidden layers for decoding and computation to obtain the completed point cloud. The aforementioned 1*2048 high-dimensional semantic features are input into a multilayer perceptron containing five hidden layers for decoding and computation to obtain the predicted completed point cloud. This invention does not impose a limit on the number of neurons in each layer of the multilayer perceptron containing five hidden layers.

[0072] In a preferred embodiment of the present invention, it further includes:

[0073] The global semantic features of 1*512 are input into an MLP containing three hidden layers for decoding to obtain a point cloud prediction result of 2048*3.

[0074] In a preferred embodiment of the present invention, it further includes:

[0075] The predicted point set (2048*3) is compared with the ground truth point set (ground true), i.e., the collected point cloud data, to calculate EMD (Earth Mover's Distance). The function expression for EMD is: Where L EMD Let φ represent the distance between predicted points and ground truth points, P represent the set of point clouds predicted by the auxiliary branch, and Q represent the set of ground truth point clouds. φ: P→Q indicates bidirectional computation between the two datasets, the predicted point cloud set of the auxiliary branch and the ground truth point cloud set. This supervised computation method for extracting global features from point clouds can improve the representational ability of global semantic features in the auxiliary branch, thereby improving the accuracy of point cloud completion results.

[0076] In a preferred embodiment of the present invention, it further includes:

[0077] The difference between the completed point cloud obtained from the main branch and the actual point cloud is calculated to evaluate the loss. The loss function Chamfer Distance (CD) is calculated as follows:

[0078]

[0079] S1 is the completed point cloud dataset, S2 is the real target point cloud dataset, and s′1 and s′2 represent the points in the point cloud, respectively.

[0080] In a preferred embodiment of the present invention, it further includes:

[0081] The total loss function for model optimization is expressed as follows: Where α and β satisfy α+β=1 and 0.5<α<1, where α is the weight of the main branch and β is the weight of the auxiliary branch. The weight of the main branch must be greater than the weight of the auxiliary branch. The weight parameters can be updated according to the calculated total loss function.

[0082] By using the technical solution of the present invention, the model calculation efficiency can be greatly improved, the memory space requirements of computing devices can be reduced, the number of model parameters and the amount of calculation can be reduced, and the accuracy and robustness of point cloud completion can be improved.

[0083] It should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The embodiments of the computer program described above can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0084] Furthermore, the method disclosed in the embodiments of the present invention can also be implemented as a computer program executed by a CPU, which may be stored in a computer-readable storage medium. When the computer program is executed by the CPU, it performs the functions defined in the method disclosed in the embodiments of the present invention.

[0085] Based on the above objectives, a second aspect of the embodiments of the present invention provides a point cloud completion apparatus, such as... Figure 4 As shown, the device 200 includes:

[0086] The first computing module is configured to calculate the first high-dimensional semantic features of the point cloud data collected by the sensor, and to calculate the global semantic features of the first high-dimensional semantic features.

[0087] The second calculation module is configured to calculate the features of point clusters after clustering point cloud data, and to calculate the second high-dimensional semantic features of the point cluster features.

[0088] The splicing module is configured to splice the second high-dimensional semantic features with the global semantic features to obtain a fused feature map;

[0089] The third calculation module is configured to calculate the high-dimensional semantic features of the fused feature map, and decode the high-dimensional semantic features to obtain the completed point cloud.

[0090] In a preferred embodiment of the present invention, the first computing module is further configured as follows:

[0091] Point cloud data is input into a shared multilayer perceptron for computation to obtain the first high-dimensional semantic features.

[0092] In a preferred embodiment of the present invention, the first computing module is further configured as follows:

[0093] Max pooling is performed on the first high-dimensional semantic features to obtain the global semantic features.

[0094] In a preferred embodiment of the present invention, the second computing module is further configured as follows:

[0095] Clustering is performed on the point cloud data to obtain K point clusters, where K is greater than 1 and less than N / 2.

[0096] In a preferred embodiment of the present invention, the second computing module is further configured as follows:

[0097] Calculate the average vector of the feature vectors in each of the K point clusters, and use the average vector of the feature vectors in each point cluster as the feature of each of the K point clusters.

[0098] In a preferred embodiment of the present invention, the second computing module is further configured as follows:

[0099] The features of K point clusters are input into the first multilayer perceptron for computation to obtain the second high-dimensional semantic features.

[0100] In a preferred embodiment of the present invention, the splicing module is further configured as follows:

[0101] The first global semantic feature is obtained by repeating the global semantic feature K times.

[0102] The second high-dimensional semantic feature is concatenated with the first global semantic feature to obtain the fused feature;

[0103] The fused features are input into the second multilayer perceptron for computation to obtain the fused feature map.

[0104] In a preferred embodiment of the present invention, the third calculation module is further configured as follows:

[0105] The third high-dimensional semantic feature is obtained by max pooling the fused feature map.

[0106] In a preferred embodiment of the present invention, the third calculation module is further configured as follows:

[0107] The third high-dimensional semantic feature is input into a multilayer perceptron for decoding and computation to obtain the completed point cloud.

[0108] In view of the above objectives, a third aspect of the present invention provides a computer device. Figure 5 The diagram shown is a schematic representation of an embodiment of the computer device provided by the present invention. Figure 5 As shown, embodiments of the present invention include the following apparatus: at least one processor 21; and a memory 22, the memory 22 storing computer instructions 23 executable on the processor, which, when executed by the processor, implement any of the above methods.

[0109] In view of the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium. Figure 6 The diagram shown is a schematic representation of an embodiment of the computer-readable storage medium provided by the present invention. Figure 6 As shown, the computer-readable storage medium 31 stores a computer program 32 that, when executed by a processor, performs any of the methods described above.

[0110] Furthermore, the method disclosed in the embodiments of the present invention can also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. When the computer program is executed by the processor, it performs the functions defined in the method disclosed in the embodiments of the present invention.

[0111] Furthermore, the above-described method steps and system units can also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to perform the functions of the above-described steps or units.

[0112] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0113] In one or more exemplary designs, functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, functionality may be stored as one or more instructions or code on or transmitted via a computer-readable medium. Computer-readable media include computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one location to another. Storage media may be any available medium accessible to a general-purpose or special-purpose computer. By way of example, and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other medium that may be used to carry or store the required program code in the form of instructions or data structures and is accessible to a general-purpose or special-purpose computer or a general-purpose or special-purpose processor. Furthermore, any connection may be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the aforementioned coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are all included in the definition of media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0114] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0115] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0116] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0117] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0118] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for point cloud completion, characterized in that, Includes the following steps: The method calculates the first high-dimensional semantic features of the point cloud data collected by the sensor, and calculates the global semantic features of the first high-dimensional semantic features; wherein, the calculation of the first high-dimensional semantic features of the point cloud data collected by the sensor includes: inputting the point cloud data into a shared multilayer perceptron for calculation to obtain the first high-dimensional semantic features; the calculation of the global semantic features of the first high-dimensional semantic features includes: performing max pooling calculation on the first high-dimensional semantic features to obtain the global semantic features; The features of point clusters after clustering point cloud data are calculated, and the second high-dimensional semantic features of the point cluster features are calculated. The calculation of the features of point clusters after clustering point cloud data includes: performing clustering calculations on the point cloud data to obtain K point clusters; calculating the average vector of the feature vectors in each of the K point clusters; and using the average vector of the feature vectors in each point cluster as the feature of each of the K point clusters, where K is greater than 1 and less than N / 2. The calculation of the second high-dimensional semantic features of the point cluster features includes: inputting the features of the K point clusters into a first multilayer perceptron for calculation to obtain the second high-dimensional semantic features. The second high-dimensional semantic features are concatenated with the global semantic features to obtain a fused feature map; Calculate the third high-dimensional semantic feature of the fused feature map, and decode the third high-dimensional semantic feature to obtain the completed point cloud; wherein, the calculation of the third high-dimensional semantic feature of the fused feature map includes: performing max pooling calculation on the fused feature map to obtain the third high-dimensional semantic feature.

2. The method of claim 1, wherein, The fused feature map is obtained by concatenating the second high-dimensional semantic features with the global semantic features, including: The first global semantic feature is obtained by repeating the global semantic feature K times. The second high-dimensional semantic feature is concatenated with the first global semantic feature to obtain the fused feature; The fused features are input into the second multilayer perceptron for computation to obtain the fused feature map.

3. The method of claim 1, wherein, Decoding the third high-dimensional semantic features yields the completed point cloud, which includes: The third high-dimensional semantic feature is input into a multilayer perceptron for decoding and computation to obtain the completed point cloud.

4. An apparatus for point cloud completion, the apparatus comprising: The device includes: A first computing module is configured to calculate a first high-dimensional semantic feature of point cloud data collected by a sensor, and to calculate a global semantic feature of the first high-dimensional semantic feature; wherein, calculating the first high-dimensional semantic feature of the point cloud data collected by the sensor includes: inputting the point cloud data into a shared multilayer perceptron for calculation to obtain the first high-dimensional semantic feature; calculating the global semantic feature of the first high-dimensional semantic feature includes: performing max pooling calculation on the first high-dimensional semantic feature to obtain the global semantic feature; The second calculation module is configured to calculate the features of point clusters after clustering point cloud data, and to calculate the second high-dimensional semantic features of the point cluster features. The calculation of the features of the point clusters after clustering includes: performing clustering calculations on the point cloud data to obtain K point clusters, calculating the average vector of the feature vectors in each of the K point clusters, and using the average vector of the feature vectors in each point cluster as the feature of each of the K point clusters, where K is greater than 1 and less than N / 2. The calculation of the second high-dimensional semantic features of the point cluster features includes: inputting the features of the K point clusters into a first multilayer perceptron for calculation to obtain the second high-dimensional semantic features. The splicing module is configured to splice the second high-dimensional semantic features with the global semantic features to obtain a fused feature map; The third calculation module is configured to calculate the third high-dimensional semantic feature of the fused feature map, and to decode the third high-dimensional semantic feature to obtain the completed point cloud; wherein, the calculation of the third high-dimensional semantic feature of the fused feature map includes: performing max pooling calculation on the fused feature map to obtain the third high-dimensional semantic feature.

5. A computer device, characterized by include: At least one processor; as well as A memory storing computer instructions executable on the processor, which, when executed by the processor, implement the steps of the method according to any one of claims 1-3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-3.

Images