Point cloud data completion model training method and point cloud data completion method

Abstract

The embodiment of the invention provides a point cloud data completion model training method and a point cloud data completion method, and is applied to the technical field of smart power grids. The training method comprises the following steps: acquiring training sample data of an initial point cloud data complementing model; the training sample data comprises incomplete point cloud data of the target object and corresponding complete point cloud data; a feature extraction module of the initial point cloud data completion model is utilized to extract global data features and local data features of each point cloud data in the incomplete point cloud data; according to the global data features and the local data features, a data completion module of an initial point cloud data completion model is utilized to obtain full-amount point cloud data; and training an initial point cloud data completion model according to a loss function value between the full point cloud data and the complete point cloud data. The method achieves the technical effect of improving the point cloud data processing efficiency.

Description

Technical Field

[0001] The present application relates to the field of smart grid technology, and in particular to a point cloud data completion model training method and a point cloud data completion method. Background Art

[0002] With the development of power grids, the inspection of power towers has become a major means of maintaining the normal operation of power grids. During the operation, maintenance and inspection of power towers, the main method is to obtain the global point cloud data of the power towers to perform 3D modeling and analysis of the towers.

[0003] In the existing technology, the method of reconstructing the tower point cloud data is to use lidar to obtain high-precision point cloud data of the tower, which can accurately reflect the spatial structure and shape of the tower. Based on this point cloud data, the power tower can be three-dimensionally modeled, providing basic support for subsequent maintenance, monitoring and planning.

[0004] Due to the limitations of hardware costs and the viewing angle of the lidar when acquiring point cloud data in existing technologies, it is impossible to obtain high-precision comprehensive point cloud data; thus, there is a technical problem of low efficiency in point cloud data processing in existing technologies. Summary of the Invention

[0005] The embodiments of the present application provide a point cloud data completion model training method and a point cloud data completion method to achieve the technical effect of improving the efficiency of point cloud data processing.

[0006] In a first aspect, an embodiment of the present application provides a method for training a point cloud data completion model, the method comprising:

[0007] Obtaining training sample data for the initial point cloud data completion model; the training sample data includes incomplete point cloud data of the target object and corresponding complete point cloud data;

[0008] The feature extraction module of the initial point cloud data completion model is used to extract the global data features and local data features of each point cloud data in the incomplete point cloud data;

[0009] According to the global data features and local data features, the data completion module of the initial point cloud data completion model is used to obtain the full point cloud data;

[0010] The initial point cloud data completion model is trained based on the loss function value between the full point cloud data and the complete point cloud data.

[0011] In one possible implementation, obtaining training sample data for the initial point cloud data completion model includes:

[0012] Acquire a first depth image and a second depth image of the target object respectively by using multiple sets of virtual binocular cameras; wherein the multiple sets of virtual binocular cameras are set at different sampling perspectives;

[0013] Performing perspective occlusion simulation on the target object according to the sampling perspective of the virtual binocular camera, and the first depth image and the second depth image, and obtaining incomplete point cloud data corresponding to the first depth image and the incomplete point cloud data corresponding to the second depth image respectively;

[0014] Training sample data is obtained based on multiple groups of incomplete point cloud data corresponding to the first depth image, incomplete point cloud data corresponding to the second depth image, and corresponding complete point cloud data.

[0015] In one possible implementation, the feature extraction module of the initial point cloud data completion model is used to extract the global data features corresponding to each point cloud data in the incomplete point cloud data, including:

[0016] Sampling incomplete point cloud data to obtain sampled point cloud data;

[0017] Performing neighborhood division on each sampling point cloud data to obtain a plurality of first neighborhood point cloud data corresponding to each sampling point cloud data;

[0018] Performing multi-layer convolution feature extraction on each grouped data to obtain data features corresponding to each sampled point cloud data in the grouped data, as well as data features corresponding to each first neighborhood point cloud data; wherein the grouped data includes the sampled point cloud data and the corresponding plurality of first neighborhood point cloud data;

[0019] Determine the group data features corresponding to each group data based on the maximum pooling algorithm;

[0020] Based on the grouped data features corresponding to the multiple grouped data, a global data feature corresponding to each point cloud data in the incomplete point cloud data is obtained.

[0021] In one possible implementation, the feature extraction module of the initial point cloud data completion model is used to extract local data features corresponding to each point cloud data in the incomplete point cloud data, including:

[0022] Using the feature extraction module of the initial point cloud data completion model, the attribute information of each point cloud data in the incomplete point cloud data is reshaped to obtain the high-dimensional attribute features of the point cloud data, and the coordinate information of the point cloud data is convolved to obtain the high-dimensional coordinates of the point cloud data;

[0023] Determine a local area corresponding to each point cloud data, where the local area includes the point cloud data and a plurality of second neighboring point cloud data corresponding to the point cloud data;

[0024] For each local area, based on the high-dimensional coordinates corresponding to the point cloud data and the high-dimensional coordinates corresponding to each second-neighborhood point cloud data, the spatial relative relationship between the point cloud data and the second-neighborhood point cloud data, as well as the attention score of each second-neighborhood point cloud data relative to the point cloud data, is calculated;

[0025] Based on the normalization of the attention score of each second neighborhood point cloud data relative to the point cloud data, the weight of each second neighborhood point cloud data relative to the point cloud data is obtained respectively;

[0026] updating the high-dimensional features of the point cloud data according to the weight of each second-neighborhood point cloud data relative to the point cloud data and the high-dimensional features of each second-neighborhood point cloud data; wherein the high-dimensional features of the second-neighborhood point cloud data are obtained by fusing the high-dimensional attribute features and high-dimensional coordinates of the second-neighborhood point cloud data; and the high-dimensional features of the point cloud data are obtained by fusing the high-dimensional attribute features and high-dimensional coordinates of the point cloud data;

[0027] Based on the updated high-dimensional features of the point cloud data, the local data features of the point cloud data are obtained.

[0028] In one possible implementation, the full point cloud data is obtained by using the data completion module of the initial point cloud data completion model based on the global data features and the local data features, including:

[0029] The data completion module of the initial point cloud data completion model obtains rough point cloud data corresponding to the point cloud data based on the global data features of the point cloud data;

[0030] The global data features and local data features of the point cloud data are fused to obtain fused data features;

[0031] Based on the fusion data features and rough point cloud data, the full point cloud data is obtained.

[0032] In one possible implementation, training an initial point cloud data completion model based on a loss function value between the full point cloud data and the complete point cloud data includes:

[0033] When the loss function value between the full point cloud data and the complete point cloud data is higher than a preset threshold, the parameters of the initial point cloud data completion model are adjusted according to the loss function value;

[0034] When the loss function value between the full point cloud data and the complete point cloud data is lower than or equal to a preset threshold, the initial point cloud data completion model is determined as the target point cloud data completion model.

[0035] In a second aspect, an embodiment of the present application provides a point cloud data completion method, comprising:

[0036] Obtain the point cloud data to be completed of the target object;

[0037] The point cloud data to be completed is input into the point cloud data completion model to obtain the full point cloud data of the target object; wherein, the point cloud data completion model is obtained by training the initial point cloud data completion model based on the point cloud data completion model training method of any one of the first aspects.

[0038] In a third aspect, an embodiment of the present application provides a point cloud data completion model training device, comprising:

[0039] An acquisition module is used to obtain training sample data for the initial point cloud data completion model; the training sample data includes incomplete point cloud data of the target object and the corresponding complete point cloud data;

[0040] The first processing module is used to use the initial point cloud data to complete the feature extraction module of the model, and extract the global data features and local data features of each point cloud data in the incomplete point cloud data;

[0041] The second processing module is used to obtain full point cloud data by using the data completion module of the initial point cloud data completion model based on the global data features and the local data features;

[0042] The third processing module is used to train the initial point cloud data completion model based on the loss function value between the full point cloud data and the complete point cloud data.

[0043] In a possible implementation, the acquisition module is further configured to:

[0044] Acquire a first depth image and a second depth image of the target object respectively by using multiple sets of virtual binocular cameras; wherein the multiple sets of virtual binocular cameras are set at different sampling perspectives;

[0045] Performing perspective occlusion simulation on the target object according to the sampling perspective of the virtual binocular camera, and the first depth image and the second depth image, and obtaining incomplete point cloud data corresponding to the first depth image and the incomplete point cloud data corresponding to the second depth image respectively;

[0046] Training sample data is obtained based on multiple groups of incomplete point cloud data corresponding to the first depth image, incomplete point cloud data corresponding to the second depth image, and corresponding complete point cloud data.

[0047] In a possible implementation, the first processing module is further configured to:

[0048] Sampling incomplete point cloud data to obtain sampled point cloud data;

[0049] Performing neighborhood division on each sampling point cloud data to obtain a plurality of first neighborhood point cloud data corresponding to each sampling point cloud data;

[0050] Performing multi-layer convolution feature extraction on each grouped data to obtain data features corresponding to each sampled point cloud data in the grouped data, as well as data features corresponding to each first neighborhood point cloud data; wherein the grouped data includes the sampled point cloud data and the corresponding plurality of first neighborhood point cloud data;

[0051] Determine the group data features corresponding to each group data based on the maximum pooling algorithm;

[0052] Based on the grouped data features corresponding to the multiple grouped data, a global data feature corresponding to each point cloud data in the incomplete point cloud data is obtained.

[0053] In a possible implementation, the first processing module is further configured to:

[0054] Using the feature extraction module of the initial point cloud data completion model, the attribute information of each point cloud data in the incomplete point cloud data is reshaped to obtain the high-dimensional attribute features of the point cloud data, and the coordinate information of the point cloud data is convolved to obtain the high-dimensional coordinates of the point cloud data;

[0055] Determine a local area corresponding to each point cloud data, where the local area includes the point cloud data and a plurality of second neighboring point cloud data corresponding to the point cloud data;

[0056] For each local area, based on the high-dimensional coordinates corresponding to the point cloud data and the high-dimensional coordinates corresponding to each second-neighborhood point cloud data, the spatial relative relationship between the point cloud data and the second-neighborhood point cloud data, as well as the attention score of each second-neighborhood point cloud data relative to the point cloud data, is calculated;

[0057] Based on the normalization of the attention score of each second neighborhood point cloud data relative to the point cloud data, the weight of each second neighborhood point cloud data relative to the point cloud data is obtained respectively;

[0058] updating the high-dimensional features of the point cloud data according to the weight of each second-neighborhood point cloud data relative to the point cloud data and the high-dimensional features of each second-neighborhood point cloud data; wherein the high-dimensional features of the second-neighborhood point cloud data are obtained by fusing the high-dimensional attribute features and high-dimensional coordinates of the second-neighborhood point cloud data; and the high-dimensional features of the point cloud data are obtained by fusing the high-dimensional attribute features and high-dimensional coordinates of the point cloud data;

[0059] Based on the updated high-dimensional features of the point cloud data, the local data features of the point cloud data are obtained.

[0060] In a possible implementation, the second processing module is further configured to:

[0061] The data completion module of the initial point cloud data completion model obtains rough point cloud data corresponding to the point cloud data based on the global data features of the point cloud data;

[0062] The global data features and local data features of the point cloud data are fused to obtain fused data features;

[0063] Based on the fusion data features and rough point cloud data, the full point cloud data is obtained.

[0064] In a possible implementation, the third processing module is further configured to:

[0065] When the loss function value between the full point cloud data and the complete point cloud data is higher than a preset threshold, the parameters of the initial point cloud data completion model are adjusted according to the loss function value;

[0066] When the loss function value between the full point cloud data and the complete point cloud data is lower than or equal to a preset threshold, the initial point cloud data completion model is determined as the target point cloud data completion model.

[0067] In a fourth aspect, an embodiment of the present application provides a point cloud data completion device, comprising:

[0068] The acquisition module is used to obtain the point cloud data to be completed of the target object;

[0069] A processing module is used to input the point cloud data to be completed into a point cloud data completion model to obtain the full point cloud data of the target object; wherein the point cloud data completion model is obtained by training the initial point cloud data completion model based on the point cloud data completion model training method of any one of the first aspects.

[0070] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0071] Memory stores computer-executable instructions;

[0072] The processor executes the computer-executable instructions stored in the memory, so that the processor executes possible implementations of the first aspect or the second aspect described above.

[0073] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement possible implementation methods as described in the first or second aspect above.

[0074] In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements possible implementation methods as described in the first or second aspect above.

[0075] The point cloud data completion model training method and the point cloud data completion method provided in the embodiments of the present application. The point cloud data completion model training method obtains the training sample data of the initial point cloud data completion model, and extracts the global data features and local data features of each incomplete point cloud data in the training sample data based on the feature extraction module, and uses the data completion module of the initial point cloud data completion model to process the global data features and local data features to obtain the full point cloud data corresponding to the incomplete point cloud data, and uses the loss function value between the full point cloud data and the complete point cloud data to train the initial point cloud data completion model, thereby obtaining a model that can generate the corresponding full point cloud data based on the incomplete point cloud data. Compared with the prior art, the present application uses the model training method combined with the training sample data to realize the point cloud completion of the incomplete point cloud data. In the point cloud data acquisition, the model can be used to realize fast point cloud completion, which reduces the time cost consumed by the point cloud completion and repair in the prior art, thereby achieving the technical effect of improving the efficiency of point cloud data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0077] Figure 1 Schematic diagram of the process of training the point cloud data completion model provided in this application Figure 1 ;

[0078] Figure 2 A schematic diagram of camera viewing angle distribution for capturing tower images using a multi-view binocular camera provided in an embodiment of the present application;

[0079] Figure 3 A schematic diagram of a flow chart of a method for generating a tower data set for a point cloud completion model provided in an embodiment of the present application;

[0080] Figure 4 Schematic diagram of the process of training the point cloud data completion model provided in this application Figure 2 ;

[0081] Figure 5 Schematic diagram of the process of training the point cloud data completion model provided in this application Figure 3 ;

[0082] Figure 6 A flowchart of the point cloud data completion method provided in this application;

[0083] Figure 7 Schematic diagram of the process of training the point cloud data completion model provided in this application Figure 4 ;

[0084] Figure 8 Schematic diagram of the data generation process for generating tower point cloud data provided in the embodiment of the present application Figure 1 ;

[0085] Figure 9 Schematic diagram of the data generation process for generating tower point cloud data provided in the embodiment of the present application Figure 2 ;

[0086] Figure 10 Schematic diagram of the data generation process for generating tower point cloud data provided in the embodiment of the present application Figure 3 ;

[0087] Figure 11 A schematic diagram of the structure of the point cloud data completion model training device provided in this application;

[0088] Figure 12 This is a schematic diagram of the structure of the point cloud data completion device provided in this application;

[0089] Figure 13 This is a schematic diagram of the structure of the electronic device provided in this application.

[0090] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0091] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0092] The existing technology uses the point cloud data of the tower to realize the reconstruction of the tower as follows: using laser radar to obtain high-precision point cloud data of the tower, which can accurately reflect the spatial structure and shape of the tower, and based on this point cloud data, the power tower is three-dimensionally modeled; when it is detected that the point cloud data is missing, the point cloud data is re-collected to obtain new point cloud data, and the point cloud data collected multiple times are fused to obtain complete point cloud data.

[0093] However, when acquiring tower point cloud data in the existing technology, the acquired point cloud data is often sparse or partially missing due to hardware cost limitations and laser radar viewing angle limitations. If complete point cloud data is required, multiple acquisitions are required or the number of laser radar installations is increased, thereby increasing hardware and time costs. This leads to the technical problem of low point cloud data processing efficiency in the existing technology.

[0094] In response to the above technical problems, the present application proposes the following technical concept: In response to the technical problem of low efficiency in point cloud data processing in the prior art, the present application proposes: using a model training method to complete incomplete point cloud data to obtain full point cloud data. Specifically, the following are: obtaining training sample data for model training, and based on the feature extraction module in the initial data completion model, extracting features from the incomplete point cloud data in the training sample data to obtain global data features and local data features corresponding to each incomplete point cloud data; using the data completion module in the initial point cloud completion model to process the global data features and local data features to obtain full point cloud data corresponding to the incomplete point cloud data; using the complete point cloud data in the training sample data and the calculated full point cloud data to calculate the loss function value, and using the loss function value to train the initial point cloud completion model, thereby obtaining a point cloud completion model that can achieve full point cloud generation. Compared with the prior art method of obtaining complete point cloud data through multiple data collections, the present application uses a model training method to complete incomplete point cloud data, reducing the cost required for point cloud data completion, thereby achieving the technical effect of improving the efficiency of point cloud data processing.

[0095] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0096] Figure 1 Schematic diagram of the process of training the point cloud data completion model provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0097] S101. Obtain training sample data for the initial point cloud data completion model.

[0098] In this step, the training sample data includes incomplete point cloud data of the target object and the corresponding complete point cloud data.

[0099] Optionally, a possible implementation method for obtaining training sample data is:

[0100] S1011 , respectively acquiring a first depth image and a second depth image of a target object using multiple sets of virtual binocular cameras.

[0101] In this step, multiple sets of virtual binocular cameras are set at different sampling angles. The sampling angles are determined by the positions and orientations of the virtual binocular cameras. Each set of virtual binocular cameras observes the target object from a different perspective.

[0102] For example, Figure 2 A schematic diagram of camera viewing angle distribution for collecting tower images using a multi-view binocular camera provided in an embodiment of the present application is shown in FIG. Figure 2 As shown in FIG, the target object is a three-dimensional model of a pole tower. Each sampling view corresponds to a set of virtual binocular cameras, and a set of virtual binocular cameras can obtain two depth images.

[0103] S1012. Perform perspective occlusion simulation on the target object according to the sampling perspective of the virtual binocular camera, and the first depth image and the second depth image, to obtain incomplete point cloud data corresponding to the first depth image and incomplete point cloud data corresponding to the second depth image, respectively.

[0104] In this step, each depth image records the distance information from the virtual binocular camera to each point in the target object; by analyzing the two depth images, the points that are occluded under the current sampling perspective can be determined. Due to occlusion, some points are invisible under the current sampling perspective, so the generated point cloud is incomplete point cloud data.

[0105] exist Figure 2 In the camera view distribution diagram shown, there are four virtual binocular cameras, each of which corresponds to two virtual cameras for collecting two depth images.

[0106] S1013 : Obtain training sample data based on multiple groups of incomplete point cloud data corresponding to the first depth image, incomplete point cloud data corresponding to the second depth image, and corresponding complete point cloud data.

[0107] In this step, the complete point cloud data refers to the complete point cloud data corresponding to the target object. Each training sample data contains: incomplete point cloud data corresponding to a sampling perspective, and complete point cloud data corresponding to the target. The complete point cloud data can be used as the label of each training data.

[0108] For example, Figure 3 A flow chart of a method for generating a tower data set for a point cloud completion model provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the method includes:

[0109] a1. Obtain the 3D model of the tower.

[0110] a2. Input the 3D model of the tower into the Blender environment.

[0111] Blender is an open source computer graphics software used for 3D modeling.

[0112] a3. Generate multiple perspective observation points based on the blender environment.

[0113] a4. Collect depth images based on observation points from multiple perspectives and a virtual binocular camera.

[0114] The virtual binocular camera in the Blender environment refers to a two-camera setup that models human eyes for stereo vision and depth perception.

[0115] a5. Determine the corresponding completed point cloud data based on the 3D model of the tower.

[0116] a6. Determine the corresponding incomplete point cloud data based on the depth image collected by the binocular camera.

[0117] a7. Generate a point cloud completion training dataset and a point cloud completion test dataset based on the complete point cloud data and the incomplete point cloud data corresponding to each test point.

[0118] It should be noted that, when acquiring training sample data, the embodiment of the present application may collect data corresponding to multiple targets to obtain training sample data.

[0119] For example, there are 10 3D models of transmission towers, and each model corresponds to 500 sampling perspectives. The size of the obtained training sample data is 5000. Each training sample data contains incomplete point cloud data corresponding to a sampling perspective, and the complete point cloud data of the transmission tower to which the sampling perspective belongs. The complete point cloud data can be used as the label of each training sample data, or the identification of the transmission tower can be used as the label of each training sample data.

[0120] S102: Utilize the feature extraction module of the initial point cloud data completion model to extract global data features and local data features of each point cloud data in the incomplete point cloud data.

[0121] In this step, global data features refer to the representative features of incomplete point cloud data, which include the geometric information of the incomplete point cloud data; local data features refer to the local detailed features of the incomplete point cloud data, which include the local spatial information of the incomplete point cloud data.

[0122] It should be noted that the extraction of global data features and local data features in this step will be further explained in the following embodiments and will not be elaborated here.

[0123] S103. According to the global data features and the local data features, the data completion module of the initial point cloud data completion model is used to obtain the full point cloud data.

[0124] Optionally, a possible implementation method for obtaining the full point cloud data is:

[0125] S1031. The data completion module of the initial point cloud data completion model obtains rough point cloud data corresponding to the point cloud data based on the global data features of the point cloud data.

[0126] In this step, coarse point cloud data is generated by feeding global input features into a multi-layer perceptron for feature extraction, resulting in coarse point cloud data. This coarse point cloud data is used to determine the accuracy of the target object's macrostructure. Comparing the coarse point cloud data with the target object's structure determines whether the coarse point cloud data accurately corresponds to the target object's macrostructure. Generating coarse point cloud data allows for a rapid assessment of the initial point cloud data completion model's ability to understand and reconstruct macrostructures without requiring the generation of a complete, high-precision point cloud.

[0127] S1032: Fusing the global data features and local data features of the point cloud data to obtain fused data features.

[0128] In this step, the global data features and local data features can be fused by feature splicing, weighted fusion, or dynamic fusion using an attention mechanism.

[0129] For example, feature concatenation can be achieved by directly concatenating global and local features to form a high-dimensional feature vector, thereby preserving the complete information of both features. Weighted fusion can be achieved by assigning different weights to global and local features and performing a weighted sum to generate a fused data feature. Dynamic fusion using an attention mechanism can be achieved by dynamically verifying the contribution scores of global and local features using the attention mechanism, performing feature selection based on the contribution scores, and obtaining a fused data feature.

[0130] S1033. Obtain full point cloud data based on the fused data features and the rough point cloud data.

[0131] In this step, the fused data features contain global and local information extracted from the incomplete point cloud data, providing overall structural information and detailed information about the incomplete point cloud data; the coarse point cloud data is obtained by high-level mapping of the global data features, providing the basic shape and general layout of the target object.

[0132] Alternatively, a possible way to obtain the full point cloud data is:

[0133] b1. Splice the coordinates of each point cloud in the rough point cloud data with the fusion data features to obtain the coarse point cloud data after feature splicing.

[0134] In this step, the purpose of splicing the coarse point cloud data and the fused data features is to assign the fused data features to each point of the coarse point cloud.

[0135] b2. Upsample each point in the coarse point cloud data to obtain dense point cloud data.

[0136] b3. Refine the dense point cloud data to obtain the full point cloud data.

[0137] S104: Training an initial point cloud data completion model based on the loss function value between the full point cloud data and the complete point cloud data.

[0138] Optionally, one possible implementation of training the initial point cloud completion model is:

[0139] S1041. When the loss function value between the full point cloud data and the complete point cloud data is higher than a preset threshold, adjust the parameters of the initial point cloud data completion model according to the loss function value.

[0140] In this step, the loss function value is the loss value calculated after inputting the full point cloud data and the complete point cloud data into the loss function. This loss value is used to evaluate the point cloud completion performance of the current initial point cloud data completion model. The preset threshold is the evaluation benchmark for model performance. If the loss function value is higher than the preset threshold, it indicates that the model completion performance is unsatisfactory and requires optimization.

[0141] Among them, the method of achieving model optimization by adjusting the parameters of the initial point cloud completion model can be: back propagation algorithm, using the back propagation algorithm combined with the loss function value to update the parameters of the model, minimizing the loss function value as the optimization goal, and achieving model optimization through multiple iterative training.

[0142] S1042. When the loss function value between the full point cloud data and the complete point cloud data is lower than or equal to a preset threshold, the initial point cloud data completion model is determined as the target point cloud data completion model.

[0143] In this step, when the loss function value is lower than or equal to the preset threshold, it indicates that the point cloud completion effect of the current initial point cloud data completion model has reached the requirements corresponding to the preset threshold, then the initial point cloud data completion model of the current training stage can be determined as the target point cloud data completion model.

[0144] For example, there is a point cloud completion task for transmission towers in a power grid, which requires that the completed point cloud meet the following preset conditions: the chamfer distance loss is less than or equal to 0.25, the tower straightness deviation is less than or equal to 0.5 degrees, and the cross-arm symmetry error is less than or equal to 0.1 meter.

[0145] c1. Perform model training on the initial point cloud completion model based on the training sample data to obtain the point cloud completion model to be determined.

[0146] c2. Obtain the full point cloud data corresponding to the incomplete point cloud data based on the point cloud completion model to be determined.

[0147] c3. Based on the full point cloud data and the complete point cloud data corresponding to the incomplete point cloud data, calculate the chamfer distance and verify the tower straightness deviation and crossarm symmetry error between the two data.

[0148] In this step, the chamfer distance measures the overall shape similarity between the completed full point cloud data and the true complete point cloud data. This is calculated by averaging the distances between the closest points in the two point clouds. Tower straightness deviation is used to assess the verticality or linearity of the tower's main structure. It is typically calculated through principal component analysis by calculating the angle between the principal direction of the point cloud and the theoretical axis. Crossarm symmetry error quantifies the geometric symmetry of the tower's left and right crossarms.

[0149] c4. Determine whether the chamfer distance, tower body straightness deviation, and crossarm symmetry error meet the preset conditions. If so, the point cloud completion model to be determined is determined as the target point cloud completion model. If not, optimize the parameters of the point cloud completion model to be determined and re-iterate training.

[0150] The point cloud data completion model training method provided in the embodiment of the present application obtains training sample data of the initial point cloud data completion model, and extracts global data features and local data features of each incomplete point cloud data in the training sample data based on a feature extraction module, and uses the data completion module of the initial point cloud data completion model to process the global data features and local data features to obtain the full point cloud data corresponding to the incomplete point cloud data, and uses the loss function value between the full point cloud data and the complete point cloud data to train the initial point cloud data completion model, thereby obtaining a model that can generate corresponding full point cloud data based on the incomplete point cloud data. Compared with the prior art, the present application utilizes a model training method combined with training sample data to achieve point cloud completion of incomplete point cloud data. In point cloud data acquisition, the model can be used to achieve rapid point cloud completion, reducing the time cost consumed by point cloud completion and repair in the prior art, thereby achieving the technical effect of improving the efficiency of point cloud data processing.

[0151] Figure 4Schematic diagram of the process of training the point cloud data completion model provided in this application Figure 2 Based on the above embodiment, the extraction of global data features in step S102 of this embodiment is further explained as follows: Figure 4 As shown, the method includes:

[0152] S401: Sampling incomplete point cloud data to obtain sampled point cloud data.

[0153] In this step, the sampling methods for the incomplete point cloud data can be: uniform sampling, voxel sampling, curvature sensitive sampling, and farthest point sampling.

[0154] Exemplarily, the incomplete point cloud data includes N three-dimensional points.

[0155] S4011. Uniform sampling is implemented by traversing the point cloud list corresponding to the incomplete point cloud data in a preset order, selecting a point every K points as a sampling point, and obtaining N / K sampling points, where N and K are both positive integers greater than 0, and N is greater than K.

[0156] S4012. The voxel sampling method is as follows: based on the number N of incomplete point clouds and the three-dimensional coordinates of each point cloud, the incomplete point cloud data is treated as an N×3 matrix, the voxel size is set to V, and the point cloud matrix is divided into a three-dimensional grid with a side length of V according to the voxel size. The average value of all points within each voxel is taken as a representative value to obtain multiple sampling points. The number of sampling points depends on the voxel size and whether the point cloud is evenly distributed.

[0157] S4013. The curvature-sensitive sampling method is: setting a neighborhood radius, segmenting the incomplete point cloud data according to the neighborhood radius to obtain multiple neighborhoods, and calculating the curvature of each neighborhood. Sampling points are selected based on the criterion of retaining high curvature points to obtain multiple sampling points.

[0158] S4014. The farthest point sampling method is: set the target number of points, randomly select a point from the incomplete point cloud, iteratively select the next point farthest from the selected point set until the target number of points is reached, and select the obtained point as the sampling point.

[0159] S402 : performing neighborhood division for each sampling point cloud data to obtain a plurality of first neighborhood point cloud data corresponding to each sampling point cloud data.

[0160] In this step, the acquisition of the first neighborhood point cloud data can be achieved by setting the neighborhood radius or limiting the number of neighborhood points. Each sampling point and the multiple first neighborhood points corresponding to each sampling point form a group, and the point cloud data of each point in a group constitutes the group data of the group.

[0161] For example, if the number of neighborhood points corresponding to each sampling point is set to 30, the 30 closest first neighborhood points to each sampling point can be obtained through spatial distance calculation. If the neighborhood radius is set to 0.5 meters, the points with a spatial distance less than or greater than 0.5 meters from each sampling point are used as first neighborhood points, thus obtaining multiple first neighborhood point cloud data corresponding to each sampling point.

[0162] S403 , performing multi-layer convolution feature extraction on each grouped data to obtain data features corresponding to each sampling point cloud data in the grouped data, and data features corresponding to each first neighborhood point cloud data.

[0163] In this step, the grouped data includes the sampling point cloud data and the corresponding plurality of first neighborhood point cloud data. The grouped data includes: the three-dimensional coordinates and attribute information of the sampling point and the first neighborhood point.

[0164] Optionally, the data feature extraction method may be:

[0165] S4031. For each group of data, organize the three-dimensional coordinates and attribute information of the sampling points in each group and the corresponding multiple neighborhood points to obtain a structured data set.

[0166] S4032. Input the structured data set into the input layer of the multi-layer convolution to obtain a data set in tensor format.

[0167] S4033. Input the tensor format data set corresponding to each group data into the multi-layer convolutional network to obtain the data features corresponding to each point cloud data in each group.

[0168] S404: Determine the group data features corresponding to each group data based on the maximum pooling algorithm.

[0169] In this step, each group data contains multiple data features. The multiple data features of each group are used as a feature matrix, and the maximum feature is selected for each feature dimension to obtain the group data features corresponding to each group.

[0170] For example, consider a grouped dataset consisting of 10 point cloud data points. The data feature corresponding to each point cloud data point is a vector of length 20, where each parameter in the vector represents the parameter value of a feature dimension. The 10 point cloud data points are aligned according to their feature dimensions to form a 10×20 matrix. For each of the 20 feature dimensions, the maximum value of the vector values of the 10 point cloud data points corresponding to each feature dimension is selected to obtain the maximum value corresponding to each of the 20 feature dimensions. The maximum values corresponding to the 20 feature dimensions are then arranged in the order of the 20 feature dimensions to obtain the grouped data feature corresponding to the grouped data point.

[0171] S405 : Obtain a global data feature corresponding to each point cloud data in the incomplete point cloud data based on the group data features corresponding to the plurality of group data.

[0172] In this step, the group data features corresponding to each group data are aggregated to obtain the global data features, and the global data features are assigned to each point in the incomplete point cloud data to obtain the global data features corresponding to each point cloud data.

[0173] Among them, the feature aggregation method can be: maximum pooling algorithm, average pooling algorithm and weighted average algorithm.

[0174] For example, there are currently 20 grouped data features, each data feature corresponds to 20 feature dimensions, and the mean of the 20 grouped data features is calculated for these 20 feature dimensions to obtain the mean corresponding to each feature dimension. The means are combined to obtain the global data feature, and the global data feature is assigned to each point cloud data to obtain incomplete point cloud data after global feature extraction.

[0175] In this embodiment, by selecting and grouping sampling points of incomplete point cloud data, representative local features are selected, and global data features of the incomplete point cloud data are obtained by feature aggregation.

[0176] Figure 5 Schematic diagram of the process of training the point cloud data completion model provided in this application Figure 3 Based on the above embodiment, the extraction of local data features in step S102 of this embodiment is further explained, as shown in FIG. Figure 4 As shown, the method includes:

[0177] S501. Using the feature extraction module of the initial point cloud data completion model, perform feature reshaping processing on the attribute information of each point cloud data in the incomplete point cloud data to obtain high-dimensional attribute features of the point cloud data, and perform convolution processing on the coordinate information of the point cloud data to obtain high-dimensional coordinates of the point cloud data.

[0178] In this step, the attribute information of each point cloud data includes: geometric attributes, physical attributes and semantic attributes; the geometric attributes include: point normal vector and curvature; the physical attributes include: reflection intensity and color; the semantic attributes include: component label, defect mark and data acquisition timestamp; it can also include advanced attributes such as local density.

[0179] For example, the high-dimensional attribute features may be extracted in the following manner:

[0180] The attribute information of each point cloud data is input into a multi-layer perceptron, and the attribute information is mapped from low-dimensional space to high-dimensional space to obtain the high-dimensional attribute features corresponding to each point cloud data.

[0181] The high-dimensional coordinates can be extracted in the following ways:

[0182] The three-dimensional coordinates of each point cloud data are input into the convolutional neural network, and the three-dimensional coordinates of the low-dimensional space are mapped to the high-dimensional space to obtain the high-dimensional coordinates.

[0183] The high-dimensional coordinate is a high-dimensional coordinate encoding that matches the high-dimensional attribute features, and is used to achieve alignment between the high-dimensional coordinate and the high-dimensional attribute features.

[0184] S502: Determine the local area corresponding to each point cloud data.

[0185] In this step, the local area includes point cloud data and a plurality of second neighborhood point cloud data corresponding to the point cloud data.

[0186] Optionally, one possible implementation method for determining the local region corresponding to each point cloud data point is to select the K nearest second-neighbor point cloud data points for each point cloud data point based on the K-Nearest Neighbors Algorithm (KNN) algorithm to form the local region corresponding to each point cloud data point. The KNN algorithm is a supervised learning classification and regression algorithm that calculates the distance between the sample to be classified and the known category sample to find its K nearest neighbors. The category or value of the target sample is predicted based on the category votes or mean of these neighbors.

[0187] For example, the number of second neighborhood points corresponding to each point cloud data is 30. Then, based on the three-dimensional coordinates of the current point cloud data, the spatial distance between each point cloud data in the incomplete point cloud data and the current point cloud data is calculated, and the spatial distance is used to select the nearest 30 second neighborhood point cloud data for each point cloud data.

[0188] S503. For each local area, based on the high-dimensional coordinates corresponding to the point cloud data and the high-dimensional coordinates corresponding to each second-neighborhood point cloud data, calculate the spatial relative relationship between the point cloud data and the second-neighborhood point cloud data, as well as the attention score of each second-neighborhood point cloud data relative to the point cloud data.

[0189] In this step, the calculation of spatial relative relationship and attention score is as follows:

[0190] S5031. Based on the high-dimensional coordinates of the point cloud data and the high-dimensional coordinates of the second neighborhood point cloud data, calculate the relative coordinates between the two point cloud data.

[0191] S5032. Based on the relative coordinates between each second neighborhood point cloud data and the point cloud data, calculate a distance metric between each second neighborhood point cloud data and the point cloud data.

[0192] S5033. Calculate the attention score corresponding to each second neighborhood point cloud data based on the distance metric, or calculate the attention score corresponding to each second neighborhood point cloud data based on the relative coordinates.

[0193] It should be noted that the calculation of the attention score in this step can use high-dimensional coordinates or original three-dimensional coordinates.

[0194] S504. Based on the normalization processing of the attention score of each second neighborhood point cloud data relative to the point cloud data, the weight of each second neighborhood point cloud data relative to the point cloud data is obtained.

[0195] In this step, normalization processing refers to processing the attention score of each second neighborhood point according to the normalization processing function, obtaining the weight corresponding to each attention score, and ensuring that the sum of the weights is equal to 1.

[0196] For example, the current local area contains a point cloud data as the center point and 5 second-neighborhood point cloud data. The attention scores corresponding to the 5 second-neighborhood point cloud data obtained by calculating the attention score are [1.0, 2.0, 3.0, 4.0, 5.0]. The normalized weights obtained by normalizing the multiple attention scores through the Softmax function are: [0.01165623, 0.03168492, 0.08612814, 0.23412118, 0.63640953].

[0197] S505 : Update the high-dimensional features of the point cloud data according to the weight of each second neighborhood point cloud data relative to the point cloud data and the high-dimensional features of each second neighborhood point cloud data.

[0198] In this step, the high-dimensional features of the second neighborhood point cloud data are obtained by fusing the high-dimensional attribute features and high-dimensional coordinates of the second neighborhood point cloud data; the high-dimensional features of the point cloud data are obtained by fusing the high-dimensional attribute features and high-dimensional coordinates of the point cloud data. The fusion of high-dimensional coordinates and high-dimensional attribute features can be achieved through feature concatenation, weighted summation, and deep learning fusion.

[0199] Exemplarily, feature extraction is performed on the attribute information of the second domain point cloud data to obtain an 8-dimensional high-dimensional attribute feature vector, and high-bit mapping is performed on the three-dimensional coordinates to obtain an 8-dimensional high-dimensional coordinate; weights are assigned to the coordinates of each dimension and the attribute features of each dimension using feature weighted fusion, and the high-dimensional coordinates and high-dimensional attribute features are weightedly fused using the assigned weights to obtain high-dimensional features corresponding to the second neighborhood point cloud data.

[0200] In this step, the high-dimensional features of the point cloud data are updated by performing weighted calculation based on the weight of each second-neighboring point cloud data relative to the point cloud data and the high-dimensional features of each second-neighboring point cloud data to obtain the high-dimensional features of the updated point cloud data.

[0201] For example, there are 100 point cloud data in the current incomplete point cloud data, and the size of the local area of each point cloud data is 51, that is, each point cloud data corresponds to 50 second-neighborhood point cloud data. Through feature extraction and high-bit mapping, the high-dimensional coordinates and high-dimensional attribute features of each point cloud data are obtained, and the high-dimensional features corresponding to each point cloud data are obtained by feature fusion. For each point cloud data and the second-neighborhood point cloud data corresponding to the point cloud data, the high-dimensional features of each point cloud data after update are calculated, that is, the high-dimensional features of each point cloud data in the incomplete point cloud data are updated, so that the high-dimensional features of each point cloud data are associated with the spatial relationship between the point cloud data. What is finally obtained are 100 point cloud data and their updated high-dimensional features.

[0202] S506 : Obtain local data features of the point cloud data based on the updated high-dimensional features of the point cloud data.

[0203] In this step, local data features refer to data features obtained by mapping high-dimensional features to a preset low-dimensional space. The mapping method can be: using a convolutional layer to map the high-dimensional features to the target dimension to obtain local data features.

[0204] In this embodiment, the attention mechanism is used to extract local data features for each point cloud data in the incomplete point cloud data, and the spatial relationship between the point cloud data is embedded into the expression of local data features in combination with the three-dimensional coordinates of the point cloud data, thereby realizing the extraction of local data features while deepening the expression of the spatial relationship between the point cloud data.

[0205] Based on the above embodiment, the present application embodiment further provides a point cloud data completion method. Figure 6 The flowchart of the point cloud data completion method provided in this application is as follows: Figure 6 As shown, the method includes:

[0206] S601: Acquire the point cloud data to be completed of the target object.

[0207] In this step, the method for obtaining the to-be-completed point cloud data of the target object can be: using a lidar method to scan the target object to obtain the to-be-completed point cloud data; or using a drone combined with a camera to obtain images of the target object from multiple angles, and analyzing the images to obtain the to-be-completed point cloud data.

[0208] For example, a transmission tower is located. Initial point cloud data of the tower is collected using a drone in conjunction with a lidar system. This initial point cloud data is then denoised to generate point cloud data to be completed. The point cloud data to be completed includes the 3D coordinates of each point cloud data point and its attribute information. The point cloud data to be completed can be the general structure of the transmission tower or the point cloud data of a specific portion of the tower.

[0209] S602: Input the point cloud data to be completed into the point cloud data completion model to obtain the full point cloud data of the target object.

[0210] In this step, the point cloud data completion model is based on Figure 1 The point cloud data completion model training method of any one of the items is obtained by training the initial point cloud data completion model.

[0211] The point cloud data to be completed is input into the point cloud data completion model. The feature extraction module of the point cloud data completion model extracts the global data features and local data features of the point cloud data to be completed. The data completion module of the point cloud data completion model extracts the coarse point cloud data corresponding to the global data features, and completes the coarse point cloud data according to the fusion data features corresponding to the global data features and the local data features, obtains the full point cloud data corresponding to the coarse point cloud data, and outputs the full point cloud data.

[0212] The point cloud data completion method provided in the embodiment of the present application obtains the point cloud data to be completed, and completes the point cloud data to be completed based on the point cloud data completion model to obtain the full amount of point cloud data after completion. The point cloud data completion model is used to complete the incomplete point cloud, thereby improving the efficiency of point cloud completion and achieving the technical effect of improving the efficiency of point cloud data processing.

[0213] Figure 7 Schematic diagram of the process of training the point cloud data completion model provided in this application Figure 4 ,like Figure 7 As shown, the method includes:

[0214] A1. Obtain point cloud dataset.

[0215] The point cloud dataset includes: incomplete point cloud data and the complete point cloud data corresponding to each incomplete point cloud data.

[0216] A2. Learning global data features of incomplete point cloud data based on the maximum pooling algorithm.

[0217] Feature compression pooling and multi-layer perceptron learning are performed on each incomplete point cloud data to obtain the global data features corresponding to the incomplete point cloud data.

[0218] A3. Learning local data features of incomplete point cloud data based on KNN algorithm and self-attention mechanism.

[0219] The high-dimensional attribute features and high-dimensional coordinates of each point cloud data are extracted, and the multi-head self-attention mechanism is used to learn the attention scores of multiple neighboring point cloud data corresponding to each point cloud data to obtain the local data features of each point cloud data.

[0220] A4. Reshape the point cloud of the target object based on global data features and local data features, and generate complete point cloud data of the target object from coarse to fine.

[0221] For example, Figures 8 to 10 As shown, the embodiments of the present application provide three schematic diagrams of a data generation process for generating complete point cloud data of a tower from coarse to fine based on incomplete point cloud data of the tower.

[0222] Figure 8 Schematic diagram of the data generation process for generating tower point cloud data provided in the embodiment of the present application Figure 1 ,like Figure 8 As shown in the figure, the incomplete point cloud data obtained are concentrated in the lower half of the tower structure. The coarse point cloud data corresponding to the incomplete point cloud is obtained through the point cloud data completion model. The coarse point cloud data is upsampled to obtain dense point cloud data. The dense point cloud data is refined to obtain the full point cloud data.

[0223] Figure 9 Schematic diagram of the data generation process for generating tower point cloud data provided in the embodiment of the present application Figure 2 The incomplete point cloud data obtained are concentrated at the top of the tower structure, and the point cloud data completion model is used to obtain the corresponding coarse point cloud data, dense point cloud data and the final full point cloud data from coarse to fine.

[0224] Figure 10 Schematic diagram of the data generation process for generating tower point cloud data provided in the embodiment of the present application Figure 3 The incomplete point cloud data obtained are concentrated in the middle part of the tower structure, and the point cloud data is used to complete the model from coarse to fine to obtain the corresponding coarse point cloud data, dense point cloud data and the final full point cloud data.

[0225] Figure 11 This is a schematic diagram of the structure of the point cloud data completion model training device provided in this application, such as Figure 11 As shown, the point cloud data completion model training device provided in this embodiment includes:

[0226] The acquisition module 1101 is used to acquire training sample data of the initial point cloud data completion model; the training sample data includes incomplete point cloud data of the target object and the corresponding complete point cloud data.

[0227] The first processing module 1102 is configured to utilize the initial point cloud data to complete the feature extraction module of the model, thereby extracting global data features and local data features of each point cloud data in the incomplete point cloud data.

[0228] The second processing module 1103 is used to obtain full point cloud data based on the global data features and the local data features using the data completion module of the initial point cloud data completion model.

[0229] The third processing module 1104 is used to train the initial point cloud data completion model according to the loss function value between the full point cloud data and the complete point cloud data.

[0230] In a possible implementation, the acquisition module 1101 is further configured to:

[0231] A first depth image and a second depth image of the target object are respectively acquired through multiple sets of virtual binocular cameras; wherein the multiple sets of virtual binocular cameras are set at different sampling perspectives.

[0232] According to the sampling perspective of the virtual binocular camera, and the first depth image and the second depth image, the perspective occlusion simulation is performed on the target object to obtain incomplete point cloud data corresponding to the first depth image and incomplete point cloud data corresponding to the second depth image, respectively.

[0233] Training sample data is obtained based on multiple groups of incomplete point cloud data corresponding to the first depth image, incomplete point cloud data corresponding to the second depth image, and corresponding complete point cloud data.

[0234] In a possible implementation, the first processing module 1102 is further configured to:

[0235] The incomplete point cloud data is sampled to obtain sampled point cloud data.

[0236] Neighborhood division is performed for each sampling point cloud data to obtain a plurality of first neighborhood point cloud data corresponding to each sampling point cloud data.

[0237] Multi-layer convolution feature extraction is performed on each grouped data to obtain data features corresponding to each sampling point cloud data in the grouped data, as well as data features corresponding to each first neighborhood point cloud data; wherein the grouped data includes the sampling point cloud data and the corresponding multiple first neighborhood point cloud data.

[0238] The group data features corresponding to each group data are determined based on the maximum pooling algorithm.

[0239] Based on the grouped data features corresponding to the multiple grouped data, a global data feature corresponding to each point cloud data in the incomplete point cloud data is obtained.

[0240] In a possible implementation, the first processing module 1102 is further configured to:

[0241] The feature extraction module of the initial point cloud data completion model is used to perform feature reshaping on the attribute information of each point cloud data in the incomplete point cloud data to obtain the high-dimensional attribute features of the point cloud data, and convolution processing is performed on the coordinate information of the point cloud data to obtain the high-dimensional coordinates of the point cloud data.

[0242] A local area corresponding to each point cloud data is determined, where the local area includes the point cloud data and a plurality of second neighboring point cloud data corresponding to the point cloud data.

[0243] For each local area, based on the high-dimensional coordinates corresponding to the point cloud data and the high-dimensional coordinates corresponding to each second-neighborhood point cloud data, the spatial relative relationship between the point cloud data and the second-neighborhood point cloud data, as well as the attention score of each second-neighborhood point cloud data relative to the point cloud data are calculated.

[0244] Based on the normalization of the attention score of each second neighborhood point cloud data relative to the point cloud data, the weight of each second neighborhood point cloud data relative to the point cloud data is obtained respectively.

[0245] The high-dimensional features of the point cloud data are updated according to the weight of each second-neighborhood point cloud data relative to the point cloud data and the high-dimensional features of each second-neighborhood point cloud data; wherein the high-dimensional features of the second-neighborhood point cloud data are obtained by fusing the high-dimensional attribute features and high-dimensional coordinates of the second-neighborhood point cloud data; the high-dimensional features of the point cloud data are obtained by fusing the high-dimensional attribute features and high-dimensional coordinates of the point cloud data.

[0246] Based on the updated high-dimensional features of the point cloud data, the local data features of the point cloud data are obtained.

[0247] In a possible implementation, the second processing module 1103 is further configured to:

[0248] The data completion module of the initial point cloud data completion model obtains coarse point cloud data corresponding to the point cloud data based on the global data features of the point cloud data.

[0249] The global data features and local data features of the point cloud data are fused to obtain fused data features.

[0250] Based on the fusion data features and rough point cloud data, the full point cloud data is obtained.

[0251] In a possible implementation, the third processing module 1104 is further configured to:

[0252] When the loss function value between the full point cloud data and the complete point cloud data is higher than a preset threshold, the parameters of the initial point cloud data completion model are adjusted according to the loss function value.

[0253] When the loss function value between the full point cloud data and the complete point cloud data is lower than or equal to a preset threshold, the initial point cloud data completion model is determined as the target point cloud data completion model.

[0254] Figure 12 This is a schematic diagram of the structure of the point cloud data completion device provided in this application, such as Figure 12 As shown, the point cloud data completion device provided in this embodiment includes:

[0255] The acquisition module 1201 is used to acquire the to-be-completed point cloud data of the target object.

[0256] The processing module 1202 is used to input the point cloud data to be completed into the point cloud data completion model to obtain the full point cloud data of the target object; wherein the point cloud data completion model is based on Figure 1 The point cloud data completion model training method of any one of the illustrated embodiments is obtained by training the initial point cloud data completion model.

[0257] The point cloud data completion model training device and the point cloud data completion device provided in this embodiment can execute the method provided in the above method embodiment. Their implementation principles and technical effects are similar, and are not described in detail in this embodiment.

[0258] Figure 13 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 13 As shown, the electronic device provided by this embodiment includes: at least one processor 1301 and a memory 1302. Optionally, the device also includes a communication component 1303. The processor 1301, the memory 1302 and the communication component 1303 are connected via a bus 1304.

[0259] During the specific implementation process, at least one processor 1301 executes the computer execution instructions stored in the memory 1302, so that at least one processor 1301 executes the above-mentioned point cloud data completion model training method or point cloud data completion method.

[0260] The specific implementation process of the processor 1301 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0261] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0262] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0263] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0264] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned point cloud data completion model training method or point cloud data completion method.

[0265] The present application also provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, the above-mentioned point cloud data completion model training method or point cloud data completion method is implemented.

[0266] The readable storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0267] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may reside in an application-specific integrated circuit (ASIC). The processor and the readable storage medium may also reside in a device as discrete components.

[0268] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, whether electrical, mechanical, or otherwise, through some interface.

[0269] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0270] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0271] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0272] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0273] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. A point cloud data completion model training method, characterized in that: include: Obtain training sample data for the initial point cloud data completion model; The training sample data includes incomplete point cloud data of the target object and corresponding complete point cloud data; Extracting global data features and local data features of each point cloud data in the incomplete point cloud data using a feature extraction module of the initial point cloud data completion model; According to the global data features and the local data features, using the data completion module of the initial point cloud data completion model to obtain full point cloud data; The initial point cloud data completion model is trained according to a loss function value between the full point cloud data and the complete point cloud data.

2. The method according to claim 1, characterized in that The step of obtaining training sample data for the initial point cloud data completion model includes: Acquire a first depth image and a second depth image of the target object respectively by using multiple sets of virtual binocular cameras; wherein the multiple sets of virtual binocular cameras are set at different sampling perspectives; Performing a perspective occlusion simulation on the target object according to the sampling perspective of the virtual binocular camera, and the first depth image and the second depth image, to obtain incomplete point cloud data corresponding to the first depth image and incomplete point cloud data corresponding to the second depth image, respectively; The training sample data is obtained based on multiple groups of incomplete point cloud data corresponding to the first depth image, incomplete point cloud data corresponding to the second depth image, and the corresponding complete point cloud data.

3. The method according to claim 2, characterized in that The feature extraction module of the model complementing the initial point cloud data is used to extract the global data features corresponding to each point cloud data in the incomplete point cloud data, including: Sampling the incomplete point cloud data to obtain sampled point cloud data; Performing neighborhood division on each of the sampled point cloud data to obtain a plurality of first neighborhood point cloud data corresponding to each of the sampled point cloud data; Performing multi-layer convolution feature extraction on each grouped data to obtain data features corresponding to each of the sampled point cloud data in the grouped data, and data features corresponding to each of the first neighborhood point cloud data; wherein the grouped data includes the sampled point cloud data and the corresponding plurality of first neighborhood point cloud data; Determine the group data features corresponding to each of the group data based on a maximum pooling algorithm; Based on the group data features corresponding to the plurality of group data, the global data feature corresponding to each point cloud data in the incomplete point cloud data is obtained.

4. The method according to claim 2, characterized in that The feature extraction module of the model complementing the initial point cloud data is used to extract local data features corresponding to each point cloud data in the incomplete point cloud data, including: Using the feature extraction module of the initial point cloud data completion model, perform feature reconstruction processing on the attribute information of each point cloud data in the incomplete point cloud data to obtain high-dimensional attribute features of the point cloud data, and perform convolution processing on the coordinate information of the point cloud data to obtain high-dimensional coordinates of the point cloud data; Determine a local area corresponding to each point cloud data, where the local area includes the point cloud data and a plurality of second neighboring point cloud data corresponding to the point cloud data; For each of the local areas, calculating, based on the high-dimensional coordinates corresponding to the point cloud data and the high-dimensional coordinates corresponding to each of the second-neighborhood point cloud data, a spatial relative relationship between the point cloud data and the second-neighborhood point cloud data, and an attention score of each of the second-neighborhood point cloud data relative to the point cloud data; Based on a normalized processing of the attention score of each piece of the second neighborhood point cloud data relative to the point cloud data, respectively obtaining a weight of each piece of the second neighborhood point cloud data relative to the point cloud data; Updating the high-dimensional features of the point cloud data according to the weight of each piece of second neighborhood point cloud data relative to the point cloud data and the high-dimensional features of each piece of second neighborhood point cloud data; wherein the high-dimensional features of the second neighborhood point cloud data are obtained by fusing the high-dimensional attribute features and the high-dimensional coordinates of the second neighborhood point cloud data; and the high-dimensional features of the point cloud data are obtained by fusing the high-dimensional attribute features and the high-dimensional coordinates of the point cloud data; Based on the updated high-dimensional features of the point cloud data, local data features of the point cloud data are obtained.

5. The method according to claim 1, wherein The method of obtaining full point cloud data by using a data completion module of the initial point cloud data completion model according to the global data features and the local data features includes: The data completion module of the initial point cloud data completion model obtains rough point cloud data corresponding to the point cloud data based on the global data features of the point cloud data; fusing the global data features and the local data features of the point cloud data to obtain fused data features; The full point cloud data is obtained according to the fused data features and the rough point cloud data.

6. The method according to claim 1, characterized in that The training of the initial point cloud data completion model according to the loss function value between the full point cloud data and the complete point cloud data includes: When a loss function value between the full point cloud data and the complete point cloud data is higher than a preset threshold, adjusting parameters of the initial point cloud data completion model according to the loss function value; When the loss function value between the full point cloud data and the complete point cloud data is lower than or equal to the preset threshold, the initial point cloud data completion model is determined as the target point cloud data completion model.

7. A point cloud data completion method, characterized in that: The method comprises: Obtain the point cloud data to be completed of the target object; The point cloud data to be completed is input into a point cloud data completion model to obtain the full point cloud data of the target object; wherein, the point cloud data completion model is obtained by training an initial point cloud data completion model based on the point cloud data completion model training method described in any one of claims 1-6.

8. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 6 or 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 or 7 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 or 7 when executed by a processor.

Images