A method, system, device and storage medium for dual-link point cloud completion

Abstract

The application provides a kind of double-link point cloud completion method, system, equipment and storage medium, method includes: the original point cloud data is respectively input into first feature extraction branch and second feature extraction branch, and first feature vector and second feature vector are respectively obtained by different network architecture;The first feature vector and the second feature vector are fused to obtain global high-dimensional semantic feature;And the global high-dimensional semantic feature is decoded to generate the target point cloud set of completion.This application uses two branches to simultaneously extract different dimensional features of point cloud, obtain more, richer high-dimensional semantic features and local features containing global point cloud information, which can improve point cloud completion accuracy and point cloud density.

Description

Technical Field

[0001] This invention relates to the fields of vehicle-road cooperation and autonomous driving, and more specifically, to a method, system, device, and storage medium for dual-link point cloud completion. Background Technology

[0002] Point cloud data, a common data format acquired by LiDAR sensors (Light Detection and Ranging) scanners, is widely used in autonomous driving, vehicle-to-everything (V2X) communication, and other fields. However, point cloud data acquisition is affected by sensor resolution limitations, specular reflection, and occlusion, leading to incomplete data. Furthermore, the dense near-field and sparse far-field characteristics of point cloud data significantly hinder its application in downstream tasks such as 3D object detection and 3D shape classification. Therefore, it is essential to complete the missing and sparse point cloud data acquired by LiDAR sensors. Current common point cloud completion techniques can only be performed on a single computing device, resulting in low model training efficiency and high memory requirements. Summary of the Invention

[0003] In view of this, the purpose of this invention is to propose a method, system, electronic device, and computer-readable storage medium for dual-link point cloud completion. This invention adopts an encoder-decoder approach to complete missing point clouds. Different feature extraction branches of the encoder module can be allocated to different computing devices for computation. This not only improves the computational efficiency of model training and inference but also reduces the memory space requirements of a single computing device. The model network structure proposed in this invention can also reduce the error rate of point cloud completion and improve the accuracy of feature point cloud completion.

[0004] Based on the above objectives, one aspect of the present invention provides a method for dual-link point cloud completion, comprising the following steps: inputting the original point cloud data into a first feature extraction branch and a second feature extraction branch respectively, and obtaining a first feature vector and a second feature vector through different network architectures respectively; fusing the first feature vector and the second feature vector to obtain global high-dimensional semantic features; and decoding the global high-dimensional semantic features to generate a completed prediction point cloud set.

[0005] In some implementations, obtaining the first feature vector and the second feature vector through different network architectures includes: using a multilayer perceptron in the first feature extraction branch to extract semantic information from the original point cloud data at different levels to obtain a first high-dimensional semantic feature, and obtaining the first feature vector by performing point-by-point max pooling on the first high-dimensional semantic feature; and using a multilayer perceptron in the second feature extraction branch to perform multiple calculations on the original point cloud data to obtain a second high-dimensional semantic feature, and obtaining the second feature vector by performing local max pooling and convolution on the second high-dimensional semantic feature.

[0006] In some implementations, the step of using a multilayer perceptron to extract semantic information from the original point cloud data at different levels in the first feature extraction branch to obtain a first high-dimensional semantic feature includes: using a multilayer perceptron to extract features from the original point cloud data to obtain a first basic semantic feature, and obtaining a second basic semantic feature based on the first basic semantic feature and the original point cloud data; and using a multilayer perceptron to extract higher-level semantic features from the second basic semantic feature to obtain the first high-dimensional semantic feature.

[0007] In some implementations, obtaining the second basic semantic feature based on the first basic semantic feature and the original point cloud data includes: concatenating the first basic semantic feature and the point cloud coordinates of the original point cloud data to obtain the second basic semantic feature.

[0008] In some implementations, the step of using a multilayer perceptron to extract higher-level semantic features from the second basic semantic features to obtain the first high-dimensional semantic features includes: using a multilayer perceptron to perform more convolution operations on the second basic semantic features to obtain the first high-dimensional semantic features.

[0009] In some implementations, the step of using a multilayer perceptron to perform multiple calculations on the original point cloud data in the second feature extraction branch to obtain a second high-dimensional semantic feature includes: performing multiple multilayer perceptron calculations on the original point cloud data using a multilayer perceptron, and fusing the features of the original point cloud data during each multilayer perceptron calculation.

[0010] In some implementations, fusing the features of the original point cloud data during each multilayer perceptron calculation includes: stitching the feature map obtained from each multilayer perceptron calculation with the original point cloud data along the same dimension, and using the stitched point cloud feature map as the initial data for the next multilayer perceptron calculation.

[0011] In some implementations, obtaining the second feature vector by performing local max pooling and convolution calculation on the second high-dimensional semantic features includes: performing local max pooling calculation on the second high-dimensional semantic features to obtain a feature map with a predetermined dimension multiplied by a predetermined length, and performing convolution calculation on the feature map using a one-dimensional linear convolution kernel to obtain the second feature vector.

[0012] In some implementations, decoding the global high-dimensional semantic features to generate a complete prediction point cloud includes: using a multilayer perceptron to perform decoding operations on the global high-dimensional semantic features to generate a complete point cloud result of a preset size.

[0013] In some implementations, the method further includes: evaluating the point cloud completion result using a real target point cloud dataset and a loss function to determine whether the point cloud completion result meets the requirements.

[0014] In some implementations, the step of evaluating the point cloud completion result using a real target point cloud dataset and a loss function to determine whether the point cloud completion result meets the requirements includes: calculating the sum of a first minimum distance from any point in the real target point cloud dataset to the point cloud completion result, and calculating the sum of a second minimum distance from any point in the point cloud completion result to the real target point cloud dataset; and determining whether the sum of the first minimum distance and the second minimum distance is less than a threshold.

[0015] In another aspect of this invention, a dual-link point cloud completion system is provided, comprising: an input module configured to input raw point cloud data into a first feature extraction branch and a second feature extraction branch respectively, and obtain a first feature vector and a second feature vector respectively through different network architectures; a fusion module configured to fuse the first feature vector and the second feature vector to obtain global high-dimensional semantic features; and a decoding module configured to decode the global high-dimensional semantic features to generate a completed prediction point cloud set.

[0016] In another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory storing computer instructions executable on the processor, the instructions, when executed by the processor, implementing the steps of the method described above.

[0017] In another aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method steps.

[0018] The present invention has the following beneficial technical effects:

[0019] (1) The two feature extraction branches used can simultaneously extract features from the point cloud in different dimensions, and obtain more and richer high-dimensional semantic features and local features containing global point cloud information, which can improve the accuracy of point cloud completion and the density of point cloud.

[0020] (2) The two feature extraction branches can be assigned to different machines for simultaneous computation, which can reduce the model’s memory requirements on computing devices and improve the efficiency of model training and inference. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0022] Figure 1 A schematic diagram illustrating an embodiment of the dual-link point cloud completion method provided by the present invention;

[0023] Figure 2 A flowchart illustrating an embodiment of the dual-link point cloud completion method provided by the present invention;

[0024] Figure 3 A schematic diagram of an embodiment of the dual-link point cloud completion system provided by the present invention;

[0025] Figure 4 A schematic diagram of the hardware structure of an embodiment of the dual-link point cloud completion electronic device provided by the present invention;

[0026] Figure 5 This is a schematic diagram of an embodiment of the computer storage medium for dual-link point cloud completion provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0028] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0029] In a first aspect, an embodiment of a dual-link point cloud completion method is proposed. Figure 1The diagram shown is an embodiment of the dual-link point cloud completion method provided by the present invention. Figure 1 As shown, the embodiments of the present invention include the following steps:

[0030] S1. Input the original point cloud data into the first feature extraction branch and the second feature extraction branch respectively, and obtain the first feature vector and the second feature vector through different network architectures respectively;

[0031] S2. Fuse the first feature vector and the second feature vector to obtain global high-dimensional semantic features; and

[0032] S3. Decode the global high-dimensional semantic features to generate a complete set of prediction points.

[0033] This invention provides a method to complete partially missing object point cloud data acquired by a LiDAR sensor. Specifically, it uses an encoder-decode method to complete the input missing point cloud. The complete point cloud completion process and encoder-decode structure are as follows: Figure 2 As shown, combined with Figure 2 The embodiments of the present invention will be described.

[0034] The original point cloud data is input into the first feature extraction branch and the second feature extraction branch respectively, and the first feature vector and the second feature vector are obtained through different network architectures respectively; the first feature vector and the second feature vector are fused to obtain global high-dimensional semantic features; and the global high-dimensional semantic features are decoded to generate a complete target point cloud set.

[0035] like Figure 2 As shown, during the encoding calculation process, the point cloud data collected by the LiDAR is first input into the first feature extraction branch ( Figure 2 The upper part of the encoder and the second feature extraction branch ( Figure 2 Feature extraction is performed on the lower half of the encoder, and then the addition function is used to fuse the calculation results of the two branches to obtain global high-dimensional semantic feature information. The global high-dimensional semantic features are then input into the decode module to generate a complete target point cloud.

[0036] In some implementations, obtaining the first feature vector and the second feature vector through different network architectures includes: in the first feature extraction branch, using a multilayer perceptron to extract semantic information from the original point cloud data at different levels to obtain a first high-dimensional semantic feature, and then obtaining the first feature vector by performing point-by-point max pooling on the first high-dimensional semantic feature; and in the second feature extraction branch, using a multilayer perceptron to perform multiple calculations on the original point cloud data to obtain a second high-dimensional semantic feature, and then obtaining the second feature vector by performing local max pooling and convolution on the second high-dimensional semantic feature. The feature extraction method of this embodiment can retain the key features of the original point cloud data to the greatest extent, improving the accuracy of point cloud completion.

[0037] In some implementations, the step of using a multilayer perceptron to extract semantic information from the original point cloud data at different levels in the first feature extraction branch to obtain a first high-dimensional semantic feature includes: using a multilayer perceptron to extract features from the original point cloud data to obtain a first basic semantic feature, and obtaining a second basic semantic feature based on the first basic semantic feature and the original point cloud data; and using a multilayer perceptron to extract higher-level semantic features from the second basic semantic feature to obtain the first high-dimensional semantic feature.

[0038] like Figure 2 As shown, in this embodiment of the invention, the first feature extraction branch uses an X-layer Multilayer Perception (MLP) to extract features from the input incomplete point cloud (i.e., the original point cloud data) to obtain M*128 semantic features (i.e., the first basic semantic features). To reduce the number of MLP parameters and increase model capacity, in this embodiment of the invention, X is set to {3, 4, 5} and the MLP does not share parameters. Then, based on the M*128 semantic features and the original point cloud data, the second basic semantic features are obtained, and then higher-level semantic feature extraction is performed on the second basic semantic features.

[0039] In some implementations, obtaining the second basic semantic feature based on the first basic semantic feature and the original point cloud data includes: concatenating the first basic semantic feature and the point cloud coordinates of the original point cloud data to obtain the second basic semantic feature.

[0040] For example, in an embodiment of the present invention, the semantic features of M*128 can be concatenated with the point cloud coordinates of the original point M*3 to obtain a new feature of M*131 (that is, the second basic semantic feature). The term "concatenation" in the present invention is a conventional term in the art, which means that the semantic features of M*128 and the point cloud coordinates of the original point M*3 are connected by feature vectors along the M dimension.

[0041] In some implementations, the step of using a multilayer perceptron to extract higher-level semantic features from the second basic semantic features to obtain the first high-dimensional semantic features includes: using a multilayer perceptron to perform more convolution operations on the second basic semantic features to obtain the first high-dimensional semantic features.

[0042] The new features are then processed using an X-layer multilayer perceptron to extract higher-level semantic information, resulting in a high-dimensional semantic feature of M*512 (i.e., the first high-dimensional semantic feature). In this embodiment, the more convolution operations are performed, the higher the dimension of the point cloud's semantic features and the more abstract the point cloud features expressed. Therefore, in this embodiment, the second basic semantic feature is subjected to more convolution operations to obtain the first high-dimensional semantic feature. Then, point-wise max pooling is performed on this high-dimensional semantic feature to obtain a 1*512 high-dimensional semantic feature vector containing global point cloud information (i.e., the first feature vector).

[0043] In some implementations, the step of using a multilayer perceptron to perform multiple calculations on the original point cloud data in the second feature extraction branch to obtain a second high-dimensional semantic feature includes: performing multiple multilayer perceptron calculations on the original point cloud data using a multilayer perceptron, and fusing the features of the original point cloud data during each multilayer perceptron calculation.

[0044] like Figure 2 As shown, in this embodiment of the invention, the second feature extraction branch performs five MLP calculations on the input incomplete point cloud using a three-layer MLP, obtaining a corresponding feature map after each calculation. The size of the feature map is V. i =M*(2 j -3), where the value range of i is {1, 2, 3, 4, 5}, and the corresponding value range of j is {7, 8, 9, 10, 11}.

[0045] In some implementations, fusing the features of the original point cloud data during each multilayer perceptron calculation includes: stitching the feature map obtained from each multilayer perceptron calculation with the original point cloud data along the same dimension, and using the stitched point cloud feature map as the initial data for the next multilayer perceptron calculation.

[0046] In each MLP computation, the original point cloud data features are fused. Specifically, the feature map obtained after each MLP computation is concatenated with the original point cloud data along the M-dimensional axis. The resulting concatenated point cloud feature map is used as the initial data for the next computation. For example, V1 and the original point cloud data can be concatenated to obtain the first point cloud feature map, and the data obtained from the MLP operation on the first point cloud feature map can be used as V2 and concatenated with the original point cloud data. After the final MLP computation, the high-dimensional semantic information of the point cloud (i.e., the second high-dimensional semantic feature) of M*2048 is obtained.

[0047] In some embodiments, obtaining the second feature vector by performing local max pooling and convolution calculations on the second high-dimensional semantic features includes: performing local max pooling calculations on the second high-dimensional semantic features to obtain a feature map of a predetermined dimension multiplied by a predetermined length, and then performing convolution calculations on the feature map using a one-dimensional linear convolution kernel to obtain the second feature vector. This embodiment of the invention uses one-dimensional linear convolution calculations for adaptive learning of different point cloud features and extraction of global high-dimensional semantic features from different dimensions, enabling the global high-dimensional semantic feature vector to possess richer local features and global information, thereby improving the accuracy of point cloud completion.

[0048] Local max pooling is used to calculate the feature map of M*512 from the high-dimensional semantic information feature layer of M*2048. The calculation method of local max pooling is to calculate the farthest distance of all input points to obtain 1 / 4 of the key points. Max pooling is then performed on the four nearest neighbors of each key point, and the features of one key point are used to replace the features of the four neighboring points. This can significantly reduce the amount of data and the computational cost of model training and inference, thereby improving computational efficiency, while maintaining almost no loss in model performance. Then, a 1*M one-dimensional linear convolution kernel is used to convolve the M*512 semantic features to obtain a 1*512 high-dimensional semantic feature vector containing global point cloud information. The importance of each key point can be learned autonomously during model training. In this embodiment of the invention, the deep semantic features use 1 / 4 of the local key point features to replace the features of all points, rather than reducing the number of input point clouds. This can retain as much original point cloud data information as possible while greatly reducing the number of model parameters and computational cost, thereby improving the robustness of the model.

[0049] In some implementations, decoding the global high-dimensional semantic features to generate a complete prediction point cloud includes: using a multilayer perceptron to perform decoding operations on the global high-dimensional semantic features to generate a complete point cloud result of a preset size.

[0050] The high-dimensional semantic feature vectors of the global point cloud information obtained from the first and second feature extraction branches in the encoding module are added together to obtain a new global high-dimensional semantic feature vector with richer information. This also increases the weights of the keypoints in the original point cloud. Finally, this result is input into the decoding module and decoded using a multilayer perceptron with L layers to generate an H*3 point cloud completion result for output. Considering the efficiency of model training and inference, the parameter L is set to {3, 4, 5}, and H can be set independently according to the performance of the hardware device, such as 2048 or 4096. The point cloud completion network structure proposed in this embodiment of the invention uses two branches in the encoding process that can simultaneously extract features from the point cloud in different dimensions, obtaining more and richer high-dimensional semantic features and local features containing global point cloud information. This can improve the accuracy and density of point cloud completion.

[0051] In some implementations, the method further includes: evaluating the point cloud completion result using a real target point cloud dataset and a loss function to determine whether the point cloud completion result meets the requirements.

[0052] In some implementations, the step of evaluating the point cloud completion result using a real target point cloud dataset and a loss function to determine whether the point cloud completion result meets the requirements includes: calculating the sum of a first minimum distance from any point in the real target point cloud dataset to the point cloud completion result, and calculating the sum of a second minimum distance from any point in the point cloud completion result to the real target point cloud dataset; and determining whether the sum of the first minimum distance and the second minimum distance is less than a threshold.

[0053] Since this scheme uses a supervised approach for model training, and point cloud data is unordered and permutation invariant, the number of points in the completed point cloud may differ from the actual number of points. Therefore, this embodiment of the invention selects the commonly used Chamfer Distance (CD) loss function as the loss function for this embodiment. The formula for calculating Chamfer Distance is as follows:

[0054]

[0055] Where S1 is the completed point cloud dataset (i.e., the point cloud completion result obtained in this embodiment of the invention), and S2 is the real target point cloud dataset. x represents any point in S1, and y represents any point in S2. The first term in the above formula represents the sum of the minimum distances from any point x in S1 to S2, and the second term represents the sum of the minimum distances from any point y in S2 to S1. If the sum of the first and second terms is less than a preset threshold, it indicates that the completion effect meets the requirements.

[0056] The point cloud completion network structure proposed in this embodiment of the invention uses two branches during the encoding process to simultaneously extract features from the point cloud in different dimensions, obtaining more and richer high-dimensional semantic features and local features containing global point cloud information. This can improve the accuracy and density of point cloud completion. Furthermore, during the encoding process, this embodiment of the invention can distribute the two feature extraction branches to different machines for simultaneous computation, which can reduce the model's memory requirements and improve model training speed and inference efficiency. Finally, by using local keypoint features instead of all point features during the encoding process, this embodiment of the invention can reduce the number of model parameters without significantly sacrificing accuracy, thus improving the efficiency of point cloud completion computation and increasing the robustness of the model.

[0057] It should be noted that the steps in each embodiment of the above dual-link point cloud completion method can be interleaved, substituted, added, or deleted. Therefore, these reasonable permutations and combinations of the dual-link point cloud completion method should also fall within the protection scope of this invention, and the protection scope of this invention should not be limited to the embodiments.

[0058] Based on the above objectives, a second aspect of this invention proposes a dual-link point cloud completion system. For example... Figure 3 As shown, system 200 includes the following modules: an input module configured to input raw point cloud data into a first feature extraction branch and a second feature extraction branch respectively, and obtain a first feature vector and a second feature vector respectively through different network architectures; a fusion module configured to fuse the first feature vector and the second feature vector to obtain global high-dimensional semantic features; and a decoding module configured to decode the global high-dimensional semantic features to generate a complete prediction point cloud set.

[0059] In some implementations, the input module is further configured to: use a multilayer perceptron in the first feature extraction branch to extract semantic information from the original point cloud data at different levels to obtain a first high-dimensional semantic feature, and obtain a first feature vector by performing point-by-point max pooling on the first high-dimensional semantic feature; and use a multilayer perceptron in the second feature extraction branch to perform multiple calculations on the original point cloud data to obtain a second high-dimensional semantic feature, and obtain a second feature vector by performing local max pooling and convolution on the second high-dimensional semantic feature.

[0060] In some embodiments, the input module is further configured to: use a multilayer perceptron to extract features from the original point cloud data to obtain a first basic semantic feature, and obtain a second basic semantic feature based on the first basic semantic feature and the original point cloud data; and use a multilayer perceptron to extract higher-level semantic features from the second basic semantic feature to obtain the first high-dimensional semantic feature.

[0061] In some implementations, the input module is further configured to: concatenate the first basic semantic feature and the point cloud coordinates of the original point cloud data to obtain the second basic semantic feature.

[0062] In some implementations, the input module is further configured to: perform more convolution operations on the second basic semantic features using a multilayer perceptron to obtain the first high-dimensional semantic features.

[0063] In some implementations, the input module is further configured to: perform multiple multilayer perceptron calculations on the original point cloud data using a multilayer perceptron, and fuse features of the original point cloud data during each multilayer perceptron calculation.

[0064] In some implementations, the input module is further configured to: concatenate the feature map obtained from each multilayer perceptron calculation with the original point cloud data along the same dimension, and use the concatenated point cloud feature map as the initial data for the next multilayer perceptron calculation.

[0065] In some implementations, the input module is further configured to: perform local max pooling on the second high-dimensional semantic features to obtain a feature map with a predetermined dimension multiplied by a predetermined length, and perform convolution on the feature map using a one-dimensional linear convolution kernel to obtain the second feature vector.

[0066] In some embodiments, the decoding module is further configured to: use a multilayer perceptron to perform decoding operations on the global high-dimensional semantic features to generate a completed point cloud result of a preset size.

[0067] In some implementations, the system further includes an evaluation calculation module configured to: evaluate the point cloud completion result using a real target point cloud dataset and a loss function to determine whether the point cloud completion result meets the requirements.

[0068] In some implementations, the evaluation calculation module is further configured to: calculate the sum of a first minimum distance from any point in the real target point cloud dataset to the point cloud completion result, and calculate the sum of a second minimum distance from any point in the point cloud completion result to the real target point cloud dataset; and determine whether the sum of the first minimum distance and the second minimum distance is less than a threshold.

[0069] Based on the above objectives, a third aspect of the present invention provides an electronic device, comprising: at least one processor; and a memory storing computer instructions executable by the processor to perform the following steps: S1, inputting raw point cloud data into a first feature extraction branch and a second feature extraction branch respectively, and obtaining a first feature vector and a second feature vector through different network architectures respectively; S2, fusing the first feature vector and the second feature vector to obtain a global high-dimensional semantic feature; and S3, decoding the global high-dimensional semantic feature to generate a complete predicted point cloud set.

[0070] In some implementations, obtaining the first feature vector and the second feature vector through different network architectures includes: using a multilayer perceptron in the first feature extraction branch to extract semantic information from the original point cloud data at different levels to obtain a first high-dimensional semantic feature, and obtaining the first feature vector by performing point-by-point max pooling on the first high-dimensional semantic feature; and using a multilayer perceptron in the second feature extraction branch to perform multiple calculations on the original point cloud data to obtain a second high-dimensional semantic feature, and obtaining the second feature vector by performing local max pooling and convolution on the second high-dimensional semantic feature.

[0071] In some implementations, the step of using a multilayer perceptron to extract semantic information from the original point cloud data at different levels in the first feature extraction branch to obtain a first high-dimensional semantic feature includes: using a multilayer perceptron to extract features from the original point cloud data to obtain a first basic semantic feature, and obtaining a second basic semantic feature based on the first basic semantic feature and the original point cloud data; and using a multilayer perceptron to extract higher-level semantic features from the second basic semantic feature to obtain the first high-dimensional semantic feature.

[0072] In some implementations, obtaining the second basic semantic feature based on the first basic semantic feature and the original point cloud data includes: concatenating the first basic semantic feature and the point cloud coordinates of the original point cloud data to obtain the second basic semantic feature.

[0073] In some implementations, the step of using a multilayer perceptron to extract higher-level semantic features from the second basic semantic features to obtain the first high-dimensional semantic features includes: using a multilayer perceptron to perform more convolution operations on the second basic semantic features to obtain the first high-dimensional semantic features.

[0074] In some implementations, the step of using a multilayer perceptron to perform multiple calculations on the original point cloud data in the second feature extraction branch to obtain a second high-dimensional semantic feature includes: performing multiple multilayer perceptron calculations on the original point cloud data using a multilayer perceptron, and fusing the features of the original point cloud data during each multilayer perceptron calculation.

[0075] In some implementations, fusing the features of the original point cloud data during each multilayer perceptron calculation includes: concatenating the feature map obtained from each multilayer perceptron calculation with the original point cloud data along the same dimensions, and using the concatenated data as the initial data for the next multilayer perceptron calculation.

[0076] In some implementations, obtaining the second feature vector by performing local max pooling and convolution calculation on the second high-dimensional semantic features includes: performing local max pooling calculation on the second high-dimensional semantic features to obtain a feature map with a predetermined dimension multiplied by a predetermined length, and performing convolution calculation on the feature map using a one-dimensional linear convolution kernel to obtain the second feature vector.

[0077] In some implementations, decoding the global high-dimensional semantic features to generate a complete prediction point cloud includes: using a multilayer perceptron to perform decoding operations on the global high-dimensional semantic features to generate a complete point cloud result of a preset size.

[0078] In some implementations, the steps further include: evaluating the point cloud completion result using a real target point cloud dataset and a loss function to determine whether the point cloud completion result meets the requirements.

[0079] In some implementations, the step of evaluating the point cloud completion result using a real target point cloud dataset and a loss function to determine whether the point cloud completion result meets the requirements includes: calculating the sum of a first minimum distance from any point in the real target point cloud dataset to the point cloud completion result, and calculating the sum of a second minimum distance from any point in the point cloud completion result to the real target point cloud dataset; and determining whether the sum of the first minimum distance and the second minimum distance is less than a threshold.

[0080] like Figure 4 The diagram shown is a hardware structure schematic of an embodiment of the dual-link point cloud completion electronic device provided by the present invention.

[0081] For example Figure 4 Taking the device shown as an example, the device includes a processor 301 and a memory 302.

[0082] Processor 301 and memory 302 can be connected via a bus or other means. Figure 4Taking the example of a connection between China and Israel via a bus.

[0083] The memory 302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the dual-link point cloud completion method in this embodiment. The processor 301 executes various server functions and data processing by running the non-volatile software programs, instructions, and modules stored in the memory 302, thereby realizing the dual-link point cloud completion method.

[0084] Memory 302 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created using the dual-link point cloud completion method, etc. Furthermore, memory 302 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 302 may optionally include memory remotely located relative to processor 301, and these remote memories can be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0085] The computer instructions 303 corresponding to one or more dual-link point cloud completion methods are stored in the memory 302. When executed by the processor 301, the dual-link point cloud completion method in any of the above method embodiments is executed.

[0086] Any embodiment of the electronic device that performs the above-described dual-link point cloud completion method can achieve the same or similar effects as any of the aforementioned method embodiments.

[0087] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs a method for dual-link point cloud completion.

[0088] like Figure 5 The diagram shown is a schematic representation of an embodiment of the computer storage medium for dual-link point cloud completion provided by the present invention. Figure 5 Taking the computer storage medium shown as an example, the computer-readable storage medium 401 stores a computer program 402 that, when executed by a processor, performs the above method.

[0089] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for the dual-link point cloud completion method can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0090] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0091] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0092] The embodiment numbers disclosed in the above embodiments of the present invention are merely for description and do not represent the superiority or inferiority of the embodiments.

[0093] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0094] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for dual-link point cloud completion, characterized in that, Includes the following steps: The original point cloud data is input into the first feature extraction branch and the second feature extraction branch respectively, and the first feature vector and the second feature vector are obtained through different network architectures respectively. The step of obtaining the first feature vector and the second feature vector through different network architectures includes: using a multilayer perceptron in the second feature extraction branch to perform multiple calculations on the original point cloud data to obtain a second high-dimensional semantic feature, and then performing local max pooling and convolution calculations on the second high-dimensional semantic feature to obtain the second feature vector. The step of using a multilayer perceptron in the second feature extraction branch to perform multiple calculations on the original point cloud data to obtain the second high-dimensional semantic feature includes: using a multilayer perceptron to perform multiple multilayer perceptron calculations on the original point cloud data, and fusing the features of the original point cloud data during each multilayer perceptron calculation. The local max pooling calculation method involves calculating the farthest distance of all input points to obtain 1 / 4 keypoints, and then performing max pooling calculations on the four nearest neighbors of each keypoint. The first feature vector and the second feature vector are fused to obtain global high-dimensional semantic features; and The global high-dimensional semantic features are decoded to generate a complete set of prediction points.

2. The method for dual-link point cloud completion according to claim 1, characterized in that, The process of obtaining the first feature vector and the second feature vector through different network architectures also includes: In the first feature extraction branch, a multilayer perceptron is used to extract semantic information from the original point cloud data at different levels to obtain a first high-dimensional semantic feature, and the first feature vector is obtained by performing point-by-point maximum pooling on the first high-dimensional semantic feature.

3. The method for dual-link point cloud completion according to claim 2, characterized in that, The step of using a multilayer perceptron in the first feature extraction branch to extract semantic information from the original point cloud data at different levels to obtain the first high-dimensional semantic features includes: A first basic semantic feature is obtained by using a multilayer perceptron to extract features from the original point cloud data, and a second basic semantic feature is obtained based on the first basic semantic feature and the original point cloud data; and The first high-dimensional semantic feature is obtained by using a multilayer perceptron to extract higher-level semantic features from the second basic semantic feature.

4. The method for dual-link point cloud completion according to claim 3, characterized in that, The process of obtaining the second basic semantic feature based on the first basic semantic feature and the original point cloud data includes: The first basic semantic feature and the point cloud coordinates of the original point cloud data are concatenated to obtain the second basic semantic feature.

5. The method for dual-link point cloud completion according to claim 3, characterized in that, The step of using a multilayer perceptron to extract higher-level semantic features from the second basic semantic features to obtain the first high-dimensional semantic features includes: The first high-dimensional semantic feature is obtained by performing more convolution operations on the second basic semantic feature using a multilayer perceptron.

6. The method for dual-link point cloud completion according to claim 1, characterized in that, The features of fusing the original point cloud data during each multilayer perceptron computation include: The feature map obtained from each multilayer perceptron calculation is concatenated with the original point cloud data along the same dimensions, and the concatenated point cloud feature map is used as the initial data for the next multilayer perceptron calculation.

7. The method for dual-link point cloud completion according to claim 6, characterized in that, The step of obtaining the second feature vector by performing local max pooling and convolution calculations on the second high-dimensional semantic features includes: The second high-dimensional semantic feature is subjected to local max pooling to obtain a feature map with a predetermined dimension multiplied by a predetermined length, and the feature map is convolved with a one-dimensional linear convolution kernel to obtain the second feature vector.

8. The method for dual-link point cloud completion according to claim 1, characterized in that, The step of decoding the global high-dimensional semantic features to generate a complete set of predicted points includes: The global high-dimensional semantic features are decoded using a multilayer perceptron to generate a point cloud completion result of a preset size.

9. The method for dual-link point cloud completion according to claim 8, characterized in that, The method also includes: The point cloud completion results are evaluated and calculated using a real target point cloud dataset and a loss function to determine whether the point cloud completion results meet the requirements.

10. The method for dual-link point cloud completion according to claim 9, characterized in that, The step of evaluating the point cloud completion result using a real target point cloud dataset and a loss function to determine whether the point cloud completion result meets the requirements includes: Calculate the sum of the first minimum distances from any point in the real target point cloud dataset to the point cloud completion result, and calculate the sum of the second minimum distances from any point in the point cloud completion result to the real target point cloud dataset; and Determine whether the sum of the first minimum distance and the sum of the second minimum distance is less than a threshold.

11. A dual-link point cloud completion system, characterized in that, include: An input module is configured to input raw point cloud data into a first feature extraction branch and a second feature extraction branch, respectively, and obtain a first feature vector and a second feature vector through different network architectures. The step of obtaining the first and second feature vectors through different network architectures includes: using a multilayer perceptron (MLP) in the second feature extraction branch to perform multiple calculations on the raw point cloud data to obtain a second high-dimensional semantic feature, and then performing local max pooling and convolution calculations on the second high-dimensional semantic feature to obtain the second feature vector. The step of using a MLP in the second feature extraction branch to perform multiple calculations on the raw point cloud data to obtain the second high-dimensional semantic feature includes: performing multiple MLP calculations on the raw point cloud data using a MLP, and fusing the features of the raw point cloud data during each MLP calculation. The local max pooling calculation method involves calculating the farthest distance of all input points to obtain 1 / 4 of a keypoint, and then performing max pooling calculations on the four nearest neighbors of each keypoint. The fusion module is configured to fuse the first feature vector and the second feature vector to obtain global high-dimensional semantic features; and The decoding module is configured to decode the global high-dimensional semantic features to generate a complete set of prediction points.

12. An electronic device, characterized in that, include: At least one processor; as well as A memory storing computer instructions executable on the processor, which, when executed by the processor, implement the steps of the method according to any one of claims 1-10.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-10.

Images