Point cloud data completion method and device based on CP decomposition and neural network

Abstract

The embodiments of the present disclosure provide a method and device for completing point cloud data based on CP decomposition and neural networks; they are applied to the field of image visual processing. The method includes mapping points in the point cloud data to be completed into a voxel grid to obtain tensor data of the voxel grid; preprocessing the tensor data, inputting the preprocessed tensor data into a trained neural network model, so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix, and performs CP inverse decomposition on the missing tensor data based on the generated factor matrix, and outputs reconstructed tensor data; mapping the reconstructed tensor data into three-dimensional space to obtain completed point cloud data. In this way, data integrity and completion accuracy are improved, computing resources are reduced, and the limitations of existing point cloud completion methods are effectively addressed.

Description

Technical Field

[0001] The present disclosure relates to the field of image vision processing, and in particular to a point cloud data completion method and device based on CP decomposition and neural network. Background Art

[0002] 3D point cloud data, as a discrete representation of spatial information, is widely used in fields such as autonomous driving, computer vision, medical imaging, digital manufacturing, and geographic information systems. In autonomous driving, point cloud data is used for environmental perception, obstacle detection, and high-precision map construction. In medical image analysis, 3D point clouds can assist in organ modeling, lesion detection, and surgical planning. In manufacturing and cultural heritage preservation, point cloud data is widely used for high-precision 3D reconstruction and reverse engineering. However, point cloud data acquisition typically relies on sensing devices such as LiDAR, structured light cameras, or stereo vision systems. These devices can be affected by factors such as occlusion, illumination variations, sensor noise, and perspective deviation in complex environments, resulting in missing, sparse, or partially incomplete point cloud data. Missing point cloud data not only affects the integrity of the target object but also reduces the accuracy of subsequent 3D reconstruction, object detection, and semantic segmentation tasks. For example, in autonomous driving scenarios, incomplete point cloud data can lead to incorrect obstacle recognition, compromising vehicle safety decisions. In medical imaging, missing point cloud information can hinder the precise detection of lesions and reduce diagnostic accuracy. In this context, how to efficiently and accurately complete the missing point cloud data and restore its geometric structure and detail information has become a key issue in current research.

[0003] At present, point cloud completion methods mainly use technologies such as geometric modeling, deep learning, and multi-view fusion. Among them, geometric modeling methods usually rely on prior shapes and have limited adaptability to complex structures; deep learning methods such as 3D convolutional networks and graph neural networks can learn completion patterns from data, but they have high computational complexity and are sensitive to data scale, making them difficult to apply to large-scale point cloud completion tasks; multi-view fusion methods require additional alignment steps, which increases computational overhead and may introduce error propagation. Although these methods have made some progress in solving the point cloud completion problem, they still have the following key defects: (1) High dependence on data quality. Most existing methods rely on image depth information, multi-view registration or external shape prior information, which makes them have high requirements on the quality of input data. For example, image-based completion methods perform poorly on low-quality or noisy image data and cannot effectively process high-noise or low-resolution point cloud data; (2) Insufficient recovery of local details. When processing point cloud data, many methods recover point clouds through coarse-grained completion or multi-scale feature processing. However, these methods often ignore the local geometric details of the point cloud, resulting in a lack of precision in the completion results. (3) High computational complexity. Some deep learning-based methods, such as neural networks and ResUNet, have good performance in completion accuracy, but their computational overhead is large. Especially when processing large-scale data sets, they may face significant computational bottlenecks, which limits their application in real-time or large-scale point cloud completion tasks. (4) Difficulty in model training and optimization. Many existing methods rely on complex network architectures and multi-stage training. The training process is complex and prone to overfitting. Especially when data is sparse or missing, the model may not be able to converge stably, resulting in unsatisfactory completion results. Summary of the Invention

[0004] The present disclosure provides a point cloud data completion method, apparatus, device, and storage medium based on CP decomposition and neural network.

[0005] According to a first aspect of the present disclosure, a point cloud data completion method based on CP decomposition and neural network is provided. The method comprises:

[0006] Map the points in the point cloud data to be completed into a voxel grid and obtain the tensor data of the voxel grid;

[0007] Preprocessing the tensor data, inputting the preprocessed tensor data into a trained neural network model, so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix, and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs reconstructed tensor data;

[0008] Map the reconstructed tensor data to three-dimensional space to obtain the completed point cloud data.

[0009] In some implementations of the first aspect, mapping points in the to-be-completed point cloud data into a voxel grid includes:

[0010] According to the following mapping formula, the mapping position of the points in the point cloud data to be completed in the voxel grid is calculated:

[0011]

[0012] Among them, x i 、y i 、z i Respectively represent the position coordinates of the i-th point in the point cloud data to be completed in the x-axis, y-axis, and z-axis directions, m, j, and k represent the position coordinates of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, Δx, Δy, and Δz represent the sizes of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, respectively, and [·] represents a rounding down operation.

[0013] In some implementations of the first aspect, cp decomposition is performed using the following formula:

[0014]

[0015] Among them, T represents the tensor data to be decomposed, r represents the rank of decomposition, and Q represents the maximum rank of decomposition. represents the Nth factor vector of rank 1, represents the outer product operation, A (N) represents the Nth factor matrix.

[0016] In some implementations of the first aspect, the neural network model is trained by the following method:

[0017] Map the points in the existing point cloud data to the voxel grid, obtain the corresponding tensor data, perform masking and preprocessing on the obtained tensor data, and use the preprocessed tensor data as training samples;

[0018] Input the training samples into the pre-trained neural network model to train the neural network model so that the neural network model performs CP decomposition on the training samples, generates a factor matrix, and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs the reconstructed tensor data;

[0019] Calculating a difference between the output reconstructed tensor data and the tensor data in the training sample, adjusting the model parameters according to the difference, and re-inputting the adjusted model parameters into the neural network model;

[0020] Repeat model training, calculating difference values, adjusting model parameters, and re-inputting the model until the comprehensive loss function converges.

[0021] In some implementations of the first aspect, the neural network model includes an input layer, a hidden layer, and an output layer; wherein,

[0022] The input layer is a multi-channel input, each channel receives tensor data coordinate information on a coordinate dimension;

[0023] The hidden layer includes a structure composed of multiple neurons. Each neuron performs a linear transformation on the input and then performs a nonlinear transformation through an activation function. The data after the nonlinear transformation undergoes another linear transformation to extract features at different levels from the input and understand the CP decomposition task.

[0024] The output layer outputs a factor matrix.

[0025] In some implementations of the first aspect, the comprehensive loss function includes a reconstruction loss function, a smoothing loss function, and a regularization loss function; wherein;

[0026] The reconstruction loss function L1 is expressed as L1=||TT c |‖ F , T c represents the reconstructed tensor data, T represents the tensor data to be decomposed (i.e., the observed tensor data), ||·|| F represents the F norm;

[0027] The smooth loss function L2 is expressed as L2=∑ u，v ||D(u)-D(v)||2, D(u) and D(v) represent the tensor of the u-th point and the v-th point in the reconstructed tensor data, respectively, and the difference between u and v is 1;

[0028] The regularized loss function L3 is expressed as A (n) represents the nth factor matrix;

[0029] The comprehensive loss function L is expressed as L=L1+λ1L2+λ2L3, where λ1 and λ2 represent weight hyperparameters corresponding to the smoothing loss function and the regularization loss function, respectively.

[0030] According to a second aspect of the present disclosure, a point cloud data completion device based on CP decomposition and a neural network is provided. The device comprises:

[0031] A tensor acquisition module is used to map points in the point cloud data to be completed into a voxel grid and obtain tensor data of the voxel grid;

[0032] A tensor reconstruction module is used to preprocess the tensor data, input the preprocessed tensor data into a trained neural network model, so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix, and performs CP decomposition on the missing tensor data according to the generated factor matrix, and outputs reconstructed tensor data;

[0033] The point cloud completion module is used to map the reconstructed tensor data into three-dimensional space and obtain the completed point cloud data.

[0034] According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0035] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described above.

[0036] The present disclosure provides a method and apparatus for completing point cloud data based on CP decomposition and a neural network. The method comprises mapping points in the point cloud data to be completed into a voxel grid to obtain tensor data of the voxel grid; preprocessing the tensor data, inputting the preprocessed tensor data into a trained neural network model, so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix, and performs CP decomposition on the missing tensor data based on the generated factor matrix, outputting reconstructed tensor data; and mapping the reconstructed tensor data into three-dimensional space to obtain completed point cloud data. In this way, data integrity and completion accuracy are improved, computing resources are reduced, and the limitations of existing point cloud completion methods are effectively addressed. For sparse and high-dimensional point cloud data, the method efficiently completes the point cloud data while preserving the structural characteristics of the point cloud data. It can recover more local details in complex scenes and efficiently recover missing point cloud data based on single-view point cloud data, thereby enhancing the robustness of the completion and improving completion accuracy while reducing computing resources.

[0037] It should be understood that the contents described in the Summary of the Invention section are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are provided for a better understanding of the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0039] Figure 1 A flowchart of a point cloud data completion method based on CP decomposition and neural network provided by an embodiment of the present disclosure is shown;

[0040] Figure 2 A network architecture diagram showing CP decomposition of a neural network model provided by an embodiment of the present disclosure is shown;

[0041] Figure 3 The figure shows a structure diagram of a point cloud data completion device based on CP decomposition and neural network provided by an embodiment of the present disclosure;

[0042] Figure 4 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0043] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0044] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0045] To address the problems encountered in the background technology, the present disclosure provides a method and apparatus for completing point cloud data based on CP decomposition and a neural network. The method includes mapping points in the point cloud data to be completed into a voxel grid to obtain tensor data of the voxel grid; preprocessing the tensor data and inputting the preprocessed tensor data into a trained neural network model so that the neural network model performs CP decomposition on the preprocessed tensor data to generate a factor matrix and performs CP decomposition on the missing tensor data based on the generated factor matrix to output reconstructed tensor data; and mapping the reconstructed tensor data into three-dimensional space to obtain completed point cloud data. In this way, data integrity and completion accuracy are improved, computing resources are reduced, and the limitations of existing point cloud completion methods are effectively addressed. For sparse and high-dimensional point cloud data, the method efficiently completes the point cloud data while preserving the structural characteristics of the point cloud data. It can recover more local details in complex scenes and efficiently recover missing point cloud data based on single-view point cloud data, thereby enhancing the robustness of the completion and improving completion accuracy while reducing computing resources.

[0046] The point cloud data completion method and device based on CP decomposition and neural network provided by the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings through specific embodiments.

[0047] Figure 1 A flowchart of a point cloud data completion method based on CP decomposition and neural network provided by an embodiment of the present disclosure is shown. The method 100 includes the following steps:

[0048] S110 , mapping points in the point cloud data to be completed to a voxel grid, and obtaining tensor data of the voxel grid.

[0049] In some embodiments, each point in the point cloud data to be completed needs to be spatially discretized in order to map the points in the point cloud data to be completed into a voxel grid.

[0050] In some embodiments, mapping points in the point cloud data to be completed into a voxel grid includes:

[0051] According to the following mapping formula, the mapping position of the points in the point cloud data to be completed in the voxel grid is calculated:

[0052]

[0053] Among them, x i 、y i 、z iRespectively represent the position coordinates of the i-th point in the point cloud data to be completed in the x-axis, y-axis, and z-axis directions, m, j, and k represent the position coordinates of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, Δx, Δy, and Δz represent the sizes of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, and [·] represents the rounding down operation;

[0054] Through the above mapping formula, all points in the point cloud data to be completed can be mapped to the corresponding positions of the voxel grid.

[0055] In some embodiments, the point cloud data to be completed can be converted into corresponding tensor data T, T∈R by mapping the points in the point cloud data to be completed to the voxel grid and obtaining the tensor data of the voxel grid. W×H×G , R is the real number field, W, H, and G are the resolutions of the voxel grid in the x-axis, y-axis, and z-axis directions, respectively. In the tensor data T, the voxel value is 1 or 0. The voxel value of 1 indicates that the voxel contains at least one point in the point cloud data, that is, the voxel is occupied by a point in the point cloud data.

[0056] In some embodiments, a dense voxel grid can be constructed by minimizing the following function:

[0057]

[0058] Where Ω represents the set of three-dimensional position coordinates corresponding to non-empty voxels in the voxel grid X, X mjk 、 They represent the voxel value corresponding to the voxel with coordinates (m, j, k) and the voxel reference value corresponding to the voxel with coordinates (m, j, k), λ is the regularization parameter, ||X|| * is the tensor nuclear norm.

[0059] S120, preprocessing the tensor data, inputting the preprocessed tensor data into a trained neural network model, so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs reconstructed tensor data.

[0060] In some embodiments, preprocessing includes standardizing and normalizing the tensor data corresponding to the point cloud data to be completed to ensure the stability and effectiveness of the data when it is input into the neural network model. Standardization and normalization can eliminate dimensional differences and avoid excessive influence of certain features.

[0061] In some embodiments, cp decomposition is performed using the following formula:

[0062]

[0063] Among them, T represents the tensor data to be decomposed, r represents the rank of decomposition, and Q represents the maximum rank of decomposition. represents the Nth factor vector of rank 1, represents the outer product operation, A (N) represents the Nth factor matrix.

[0064] In some embodiments, CP decomposition is performed on the three-dimensional tensor data to be decomposed, and three factor matrices can be obtained: A (1) , A (2) , A (3) , the dimensions of these three factor matrices are I1×Q, I2×Q, and I3×Q respectively.

[0065] In some embodiments, performing CP decomposition on the missing tensor data according to the generated factor matrix and outputting reconstructed tensor data includes:

[0066] According to the formula for CP decomposition, the missing tensor data is CP decomposed by the generated factor matrix to obtain reconstructed tensor data.

[0067] In some embodiments, the neural network model is trained by:

[0068] Map the points in the existing point cloud data to the voxel grid, obtain the corresponding tensor data, perform masking and preprocessing on the obtained tensor data, and use the preprocessed tensor data as training samples;

[0069] Input the training samples into the pre-trained neural network model to train the neural network model so that the neural network model performs CP decomposition on the training samples, generates a factor matrix, and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs the reconstructed tensor data;

[0070] Calculating a difference between the output reconstructed tensor data and the tensor data in the training sample, adjusting the model parameters according to the difference, and re-inputting the adjusted model parameters into the neural network model;

[0071] Repeat model training, calculating difference values, adjusting model parameters, and re-inputting the model until the comprehensive loss function converges.

[0072] In some embodiments, the acquired tensor data is masked and some values ​​of the tensor data are set to 0 to simulate missing conditions.

[0073] In some embodiments, the input layer, hidden layer, and output layer constitute a network architecture of a neural network model for CP decomposition, such as Figure 2 As shown, the neural network model includes an input layer, a hidden layer and an output layer; wherein,

[0074] The input layer is a multi-channel input, each channel receives the coordinate information of the tensor data on a coordinate dimension; for example, the first input channel receives the coordinate information M of the tensor data on the x-axis 1×E , the second input channel receives the coordinate information J of the tensor data on the y-axis 1×E , the third input channel receives the coordinate information k of the tensor data on the z axis 1×E , E represents the number of tensor data;

[0075] The hidden layer includes a structure composed of multiple neurons. Each neuron performs a linear transformation on the input through a linear function, and then performs a nonlinear transformation through an activation function (ReLU). The data after the nonlinear transformation is then linearly transformed through a linear function to extract features at different levels from the input and understand the CP decomposition task.

[0076] The output layer outputs a factor matrix through multiple channels. For example, the output dimension of the first output channel is I 1×Q The factor matrix The output dimension of the second output channel is I 2×Q The factor matrix The output dimension of the third output channel is I 3×Q The factor matrix

[0077] In some embodiments, the design of the network architecture of the neural network model for CP decomposition can adapt to the characteristics of high-dimensional tensor data and extract deep features in the data representation;

[0078] For example, the input layer provides the neural network model with the position information of the tensor data in each dimension, so that the neural network model can learn the corresponding factor matrix based on each position information in the high-dimensional space;

[0079] The hidden layer introduces nonlinear processing by using an activation function, which enhances the model's fitting ability and enables it to handle complex patterns and high-dimensional relationships in tensor data. The activation function formula is ReLU(q)=max(0,q), where q represents the sum of the inputs to a neuron. After processing by the hidden layer, the model can effectively learn the relationship between the coordinate information received by the input layer and the column vectors of the factor matrix, where the column vectors of the factor matrix represent a low-rank representation of the tensor.

[0080] The output layer maps the high-dimensional features learned by the hidden layer to the column vectors of each factor matrix to achieve CP decomposition.

[0081] In some embodiments, when the neural network model performs CP decomposition for the first time, a random initialization method is used to initialize the factor matrix obtained by the decomposition.

[0082] In some embodiments, the Adam optimizer is used to execute a gradient descent strategy to optimize the comprehensive loss function and converge the comprehensive loss function. In the process of optimizing the comprehensive loss function, the model automatically adjusts the model parameters, the weights of the factor matrix, etc. through the back propagation algorithm until the optimal cp decomposition parameters are obtained.

[0083] In some embodiments, the comprehensive loss function includes a reconstruction loss function, a smoothing loss function, and a regularization loss function; wherein;

[0084] The reconstruction loss function L1 is expressed as L1=||TT c || F , T c represents the reconstructed tensor data, T represents the tensor data to be decomposed (i.e., the observed tensor data), ||·|| F represents the F norm;

[0085] The smooth loss function L2 is expressed as L2=∑ u，v ||D(u)-D(v)||2, D(u) and D(v) represent the tensor of the u-th point and the v-th point in the reconstructed tensor data, respectively, and the difference between u and v is 1 (u and v are adjacent points);

[0086] The regularized loss function L3 is expressed as A (n) represents the nth factor matrix;

[0087] The comprehensive loss function L is expressed as L=L1+λ1L2+λ2L3, where λ1 and λ2 represent weight hyperparameters corresponding to the smoothing loss function and the regularization loss function, respectively.

[0088] In some embodiments, a smoothing loss function is used to ensure the continuity and smoothness of the reconstructed tensor data. In particular, for dense or irregular data, adding a smoothing loss function can avoid generating jumpy point clouds or discrete structures, thereby making the completed point cloud data more continuous.

[0089] The regularization loss function is used to prevent the model from overfitting. By constraining the model to learn the CP decomposition process through the regularization loss function, it helps to control the size of the factor matrix, prevent the model from overfitting the training samples, and also control the complexity of the model to ensure the generalization ability of the model.

[0090] S130, mapping the reconstructed tensor data to three-dimensional space to obtain completed point cloud data.

[0091] In some embodiments, the reconstructed tensor data includes all missing values ​​based on low-rank structure completion. The reconstruction process can maintain the overall pattern and structure of the original tensor (tensor data corresponding to the point cloud data to be completed), ensure that the completed tensor data conforms to the distribution of the original tensor data, thereby effectively recovering the missing parts in the sparse tensor, mapping the reconstructed tensor data to three-dimensional space, and obtaining the completed point cloud data. The completed point cloud data contains the complete spatial information of the original point cloud data to be completed.

[0092] According to an embodiment of the present disclosure, a point cloud data completion method and device based on CP decomposition and neural network are provided. The method includes mapping points in the point cloud data to be completed into a voxel grid to obtain tensor data of the voxel grid; preprocessing the tensor data, inputting the preprocessed tensor data into a trained neural network model, so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs reconstructed tensor data; mapping the reconstructed tensor data into three-dimensional space to obtain the completed point cloud data. This method transforms the point cloud completion problem into a tensor completion problem. It uses CP decomposition to perform low-rank decomposition on the tensor, representing the high-dimensional point cloud data in the form of the outer product of a factor matrix, thereby extracting the potential low-rank structural features of the high-dimensional point cloud data (i.e., the factor matrix). A neural network model is then used to optimize the factor matrix to improve the stability of the tensor decomposition and enhance its adaptability to different data distributions. Based on the optimized factor matrix, the CP inverse decomposition method is used to reconstruct the tensor data, achieving high-precision completion of missing point clouds. This ensures that the completion results are consistent in global structure and accurate in recovering local geometric details. In this way, this method not only improves data integrity and completion accuracy, reduces computational resources, but also effectively addresses the limitations of existing point cloud completion methods. For sparse and high-dimensional point cloud data, it efficiently completes point cloud data while preserving its structural characteristics. It can recover more local details in complex scenes and efficiently recover missing point cloud data based on single-view point cloud data, enhancing the robustness of the completion, improving completion accuracy, and reducing computational resources.

[0093] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0094] The above is an introduction to the method embodiment. The following is a further explanation of the solution disclosed in the present disclosure through an apparatus embodiment.

[0095] Figure 3 The structure diagram of a point cloud data completion device based on CP decomposition and neural network provided by an embodiment of the present disclosure is shown. The device 300 includes:

[0096] The tensor acquisition module 310 is used to map points in the point cloud data to be completed into a voxel grid and obtain tensor data of the voxel grid.

[0097] In some embodiments, the tensor acquisition module 310 is specifically configured to:

[0098] Mapping the points in the point cloud data to be completed into a voxel grid includes:

[0099] According to the following mapping formula, the mapping position of the points in the point cloud data to be completed in the voxel grid is calculated:

[0100]

[0101] Among them, x i 、y i 、z i Respectively represent the position coordinates of the i-th point in the point cloud data to be completed in the x-axis, y-axis, and z-axis directions, m, j, and k represent the position coordinates of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, Δx, Δy, and Δz represent the sizes of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, respectively, and [·] represents a rounding down operation.

[0102] The tensor reconstruction module 320 is used to preprocess the tensor data and input the preprocessed tensor data into the trained neural network model so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs the reconstructed tensor data.

[0103] In some embodiments, the tensor reconstruction module 320 is specifically configured to:

[0104] The CP decomposition is performed using the following formula:

[0105]

[0106] Among them, T represents the tensor data to be decomposed, r represents the rank of decomposition, and Q represents the maximum rank of decomposition. represents the Nth factor vector of rank 1, represents the outer product operation, A (N)represents the Nth factor matrix.

[0107] In some embodiments, the tensor reconstruction module 320 is further configured to:

[0108] The neural network model is trained by the following method:

[0109] Map the points in the existing point cloud data to the voxel grid, obtain the corresponding tensor data, perform masking and preprocessing on the obtained tensor data, and use the preprocessed tensor data as training samples;

[0110] Input the training samples into the pre-trained neural network model to train the neural network model so that the neural network model performs CP decomposition on the training samples, generates a factor matrix, and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs the reconstructed tensor data;

[0111] Calculating a difference between the output reconstructed tensor data and the tensor data in the training sample, adjusting the model parameters according to the difference, and re-inputting the adjusted model parameters into the neural network model;

[0112] Repeat model training, calculating difference values, adjusting model parameters, and re-inputting the model until the comprehensive loss function converges.

[0113] In some embodiments, the tensor reconstruction module 320 is further configured to:

[0114] The neural network model includes an input layer, a hidden layer and an output layer; wherein,

[0115] The input layer is a multi-channel input, each channel receives tensor data coordinate information on a coordinate dimension;

[0116] The hidden layer includes a structure composed of multiple neurons. Each neuron performs a linear transformation on the input and then performs a nonlinear transformation through an activation function. The data after the nonlinear transformation undergoes another linear transformation to extract features at different levels from the input and understand the CP decomposition task.

[0117] The output layer outputs a factor matrix.

[0118] In some embodiments, the tensor reconstruction module 320 is further configured to:

[0119] The comprehensive loss function includes a reconstruction loss function, a smoothing loss function and a regularization loss function; wherein;

[0120] The reconstruction loss function L1 is expressed as L1=||TT c || F , T crepresents the reconstructed tensor data, T represents the tensor data to be decomposed (i.e., the observed tensor data), ||·|| F represents the F norm;

[0121] The smooth loss function L2 is expressed as L2=∑ u，v ||D(u)-D(v)||2, D(u) and D(v) represent the tensor of the u-th point and the v-th point in the reconstructed tensor data, respectively, and the difference between u and v is 1;

[0122] The regularized loss function L3 is expressed as A (n) represents the nth factor matrix;

[0123] The comprehensive loss function L is expressed as L=L1+λ1L2+λ2L3, where λ1 and λ2 represent weight hyperparameters corresponding to the smoothing loss function and the regularization loss function, respectively.

[0124] The point cloud completion module 330 is used to map the reconstructed tensor data into three-dimensional space to obtain completed point cloud data.

[0125] It is understandable that Figure 3 Each module / unit in the illustrated device 300 has the function of implementing each step in the method 100 provided in the embodiment of the present disclosure and can achieve its corresponding technical effect. For the sake of brevity, they will not be described in detail here.

[0126] Figure 4 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Electronic device 400 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 400 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0127] like Figure 4 As shown, electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. Various programs and data required for the operation of electronic device 400 can also be stored in RAM 403. Computing unit 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An I / O interface 405 is also connected to bus 404.

[0128] Multiple components in the electronic device 400 are connected to the I / O interface 405, including an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0129] The computing unit 401 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as method 100. For example, in some embodiments, the method 100 can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method 100 described above can be performed. Alternatively, in other embodiments, the computing unit 401 can be configured to perform the method 100 in any other appropriate manner (e.g., by means of firmware).

[0130] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0131] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0132] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0133] It should be noted that the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute method 100 and achieve the corresponding technical effect achieved by executing the method in the embodiment of the present disclosure. For the sake of concise description, they will not be repeated here.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0137] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0138] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A point cloud data completion method based on CP decomposition and neural network, characterized in that: include: Mapping points in the point cloud data to be completed to the voxel grid includes: calculating the mapping position of the points in the point cloud data to be completed in the voxel grid according to the following mapping formula: Among them, x i 、y i 、z i Respectively represent the position coordinates of the i-th point in the point cloud data to be completed in the x-axis, y-axis, and z-axis directions, m, j, and k represent the position coordinates of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, Δx, Δy, and Δz represent the sizes of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, and [·] represents the rounding down operation; obtain the tensor data of the voxel grid; The tensor data is preprocessed, and the preprocessed tensor data is input into a trained neural network model so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix, and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs reconstructed tensor data; wherein the CP decomposition is performed by the following formula: Among them, T represents the tensor data to be decomposed, r represents the rank of decomposition, and Q represents the maximum rank of decomposition. represents the Nth factor vector of rank 1, represents the outer product operation, A (N) represents the Nth factor matrix; The reconstructed tensor data is mapped to three-dimensional space to obtain the completed point cloud data; wherein, the neural network model includes an input layer, a hidden layer and an output layer; wherein, the input layer provides the neural network model with position information of the tensor data in each dimension, so that the neural network model can learn the corresponding factor matrix according to each position information in the high-dimensional space; the hidden layer introduces nonlinear processing by using an activation function, so that the neural network model can learn the relationship between the coordinate information received by the input layer and the column vector of the factor matrix; the output layer maps the high-dimensional features learned by the hidden layer to the column vector of each factor matrix to realize CP decomposition.

2. The method according to claim 1, characterized in that The neural network model is trained by the following method: Map the points in the existing point cloud data to the voxel grid, obtain the corresponding tensor data, perform masking and preprocessing on the obtained tensor data, and use the preprocessed tensor data as training samples; Input the training samples into the pre-trained neural network model to train the neural network model so that the neural network model performs CP decomposition on the training samples, generates a factor matrix, and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs the reconstructed tensor data; Calculating a difference between the output reconstructed tensor data and the tensor data in the training sample, adjusting the model parameters according to the difference, and re-inputting the adjusted model parameters into the neural network model; Repeat model training, calculating difference values, adjusting model parameters, and re-inputting the model until the comprehensive loss function converges.

3. The method according to claim 2, characterized in that The input layer is a multi-channel input, each channel receives tensor data coordinate information on a coordinate dimension; The hidden layer includes a structure composed of multiple neurons. Each neuron performs a linear transformation on the input and then performs a nonlinear transformation through an activation function. The data after the nonlinear transformation undergoes another linear transformation to extract features at different levels from the input and understand the CP decomposition task. The output layer outputs a factor matrix.

4. The method according to claim 2, characterized in that The comprehensive loss function includes a reconstruction loss function, a smoothing loss function and a regularization loss function; wherein; The reconstruction loss function L1 is expressed as L1=||TT c || F , T c Represents the reconstruction of tensor data, ||·|| F represents the F norm; The smooth loss function L2 is expressed as L2=∑ u，v ||D(u)-D(v)||2, D(u) and D(v) represent the tensor of the u-th point and the v-th point in the reconstructed tensor data, respectively, and the difference between u and v is 1; The regularized loss function L3 is expressed as A (n) represents the nth factor matrix; The comprehensive loss function L is expressed as L=L1+λ1L2+λ2L3, where λ1 and λ2 represent weight hyperparameters corresponding to the smoothing loss function and the regularization loss function, respectively.

5. A point cloud data completion device based on CP decomposition and neural network, characterized in that: include: The tensor acquisition module is used to map the points in the point cloud data to be completed to the voxel grid, including: calculating the mapping position of the points in the point cloud data to be completed in the voxel grid according to the following mapping formula: Among them, x i 、y i 、z i Respectively represent the position coordinates of the i-th point in the point cloud data to be completed in the x-axis, y-axis, and z-axis directions, m, j, and k represent the position coordinates of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, Δx, Δy, and Δz represent the sizes of the voxel corresponding to the i-th point in the point cloud data to be completed in the voxel grid in the x-axis, y-axis, and z-axis directions, and [·] represents the rounding down operation; obtain the tensor data of the voxel grid; A tensor reconstruction module is used to preprocess the tensor data and input the preprocessed tensor data into a trained neural network model so that the neural network model performs CP decomposition on the preprocessed tensor data, generates a factor matrix, and performs CP inverse decomposition on the missing tensor data according to the generated factor matrix, and outputs the reconstructed tensor data; wherein, the neural network model includes an input layer, a hidden layer and an output layer; wherein, the input layer provides the neural network model with position information of the tensor data in each dimension, so that the neural network model can learn the corresponding factor matrix according to each position information in the high-dimensional space; the hidden layer introduces nonlinear processing by using an activation function, so that the neural network model can learn the relationship between the coordinate information received by the input layer and the column vector of the factor matrix; the output layer maps the high-dimensional features learned by the hidden layer to the column vector of each factor matrix to realize CP decomposition; wherein, CP decomposition is performed by the following formula: Among them, T represents the tensor data to be decomposed, r represents the rank of decomposition, and Q represents the maximum rank of decomposition. represents the Nth factor vector of rank 1, represents the outer product operation, A (N) represents the Nth factor matrix; The point cloud completion module is used to map the reconstructed tensor data into three-dimensional space and obtain the completed point cloud data.

6. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

Images