Point cloud data completion and point cloud data processing

Abstract

The disclosed embodiments provide a method, device and system for completing and processing point cloud data, which can acquire first point cloud data; use a point cloud completion network to complete the first point cloud data to obtain second point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates third point cloud data according to the target latent space vector, and the difference between fourth point cloud data obtained after the third point cloud data is subjected to preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

Description

Cross-references to related publications

[0001] This application is a national phase application of the PCT application with application number PCT / IB2021 / 054749 filed on May 31, 2021, and the PCT application claims priority to the Singapore patent application with application number 10202103258S filed on March 30, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of computer vision technology, and in particular to a method, device and system for completing point cloud data, and a method, device and system for processing point cloud data. Background Art

[0003] Point cloud completion is used to repair missing point cloud data (i.e., incomplete point cloud data) and estimate complete point cloud data from the incomplete point cloud data. Point cloud completion has many applications in many fields such as autonomous driving and robot navigation. Most traditional point cloud completion methods use paired incomplete point cloud data and complete point cloud data as training data, perform full supervision training on the point cloud completion network, and then use the trained point cloud completion network to complete the incomplete point cloud data. However, the form of incomplete point cloud data is often diverse due to sensor types or occlusions. The point cloud completion network obtained based on the above method is difficult to generalize to other forms of incomplete point cloud data, and the generalization performance is poor. Summary of the invention

[0004] According to a first aspect of an embodiment of the present disclosure, a method for completing point cloud data is provided, the method comprising: acquiring first point cloud data; completing the first point cloud data using a point cloud completion network to obtain second point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates third point cloud data according to a target latent space vector, and the difference between fourth point cloud data obtained after a preset degradation process on the third point cloud data and real point cloud data collected from the physical space is within a preset difference range.

[0005] In some embodiments, the method further includes: acquiring original point cloud data collected by a point cloud collection device from a physical space; and performing point cloud segmentation on the original point cloud data to obtain the first point cloud data.

[0006] In some embodiments, the method further includes: performing association processing on multiple frames of the second point cloud data.

[0007] In some embodiments, the method also includes: obtaining the point cloud completion network based on the following manner: training the initial point cloud completion network based on the sample complete point cloud data; obtaining the third point cloud data generated by the trained point cloud completion network based on the target latent space vector; optimizing the trained point cloud completion network based on the fourth point cloud data obtained after the preset degradation processing of the third point cloud data and the real point cloud data to obtain the point cloud completion network.

[0008] In some embodiments, the method also includes: obtaining multiple point cloud blocks in the third point cloud data; respectively determining the distribution characteristics of the points in each point cloud block; establishing a loss function based on the distribution characteristics of the points in each point cloud block; and optimizing the trained point cloud completion network based on the loss function.

[0009] In some embodiments, the method further includes: obtaining multiple initial latent space vectors sampled from the latent space; for each initial latent space vector, determining the objective function of the initial latent space vector based on the point cloud data corresponding to the initial latent space vector and the real point cloud data; and determining the target latent space vector from the initial latent space vectors based on the objective functions of the initial latent space vectors.

[0010] In some embodiments, the preset degradation processing includes: for any target point in the real point cloud data, determining at least one point closest to the target point from the third point cloud data as a neighboring point of the target point; and determining the union of the neighboring points of each target point in the real point cloud data as the fourth point cloud data obtained by the preset degradation processing of the third point cloud data.

[0011] According to a second aspect of an embodiment of the present disclosure, a training method for a point cloud completion network is provided, the method comprising: training an initial point cloud completion network based on sample complete point cloud data; obtaining completed point cloud data generated by the trained point cloud completion network based on a target latent space vector; optimizing the trained point cloud completion network based on degraded point cloud data obtained after preset degradation processing of the completed point cloud data and real point cloud data collected from a physical space, so that the difference between the degraded point cloud data and the real point cloud data is within a preset difference range.

[0012] In some embodiments, the method further includes: obtaining multiple initial latent space vectors sampled from the latent space; respectively obtaining degraded point cloud data obtained after preset degradation processing of the completed point cloud data generated by the trained point cloud completion network for each initial latent space vector; determining the objective function corresponding to the initial latent space vector based on the degraded point cloud data corresponding to the initial latent space vector and the real point cloud data; and determining the target latent space vector from each initial latent space vector based on the objective function corresponding to each initial latent space vector.

[0013] In some embodiments, the preset degradation processing includes: for any target point in the real point cloud data, determining at least one point closest to the target point from the sample complete point cloud data as a neighboring point of the target point; and determining the union of the neighboring points of each target point in the real point cloud data as the degraded point cloud data obtained by the preset degradation processing of the sample complete point cloud data.

[0014] According to a third aspect of an embodiment of the present disclosure, a method for processing point cloud data is provided, the method comprising: obtaining first point cloud data to be processed corresponding to game participants in a game area and second point cloud data to be processed corresponding to game objects; obtaining first processed point cloud data obtained after a point cloud completion network completes the first point cloud data to be processed, and second processed point cloud data obtained after the point cloud completion network completes the second point cloud data to be processed; and performing association processing on the first processed point cloud data and the second processed point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to a target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0015] In some embodiments, the method further includes: determining an action performed by the game participant on the game object based on an association result of the first processed point cloud data and the second processed point cloud data.

[0016] In some embodiments, the obtaining of first point cloud data to be processed of game participants in the game area and second point cloud data to be processed corresponding to game objects includes: obtaining initial point cloud data collected by a point cloud collection device arranged around the game area; performing point cloud segmentation on the initial point cloud data to obtain first point cloud data to be processed of the game participants and second point cloud data to be processed corresponding to game objects.

[0017] In some embodiments, the point cloud completion network is used to complete first point cloud data to be processed corresponding to multiple categories of game participants and / or second point cloud data to be processed corresponding to multiple categories of game objects; or the point cloud completion network includes a first point cloud completion network and a second point cloud completion network, the first point cloud completion network is used to complete the first point cloud data to be processed corresponding to the first category of game participants, and the second point cloud completion network is used to complete the second point cloud data to be processed corresponding to the second category of game objects.

[0018] According to a fourth aspect of an embodiment of the present disclosure, a method for processing point cloud data is provided, the method comprising: obtaining unprocessed point cloud data of a target object in a game area; the target object comprises at least one of a game participant and a game object; obtaining processed point cloud data obtained after a point cloud completion network completes the unprocessed point cloud data; identifying the target object based on the processed point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to a target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0019] In some embodiments, the step of obtaining the unprocessed point cloud data of the target objects within the game area includes: obtaining the initial point cloud data collected by a point cloud collection device arranged around the game area; performing point cloud segmentation on the initial point cloud data to obtain the unprocessed point cloud data of the target objects of each category within the game area.

[0020] In some embodiments, the point cloud completion network is used to complete the to-be-processed point cloud data of target objects of multiple categories; or the point cloud completion network is used to complete the to-be-processed point cloud data of target objects of a specific category.

[0021] According to a fifth aspect of an embodiment of the present disclosure, a point cloud data completion device is provided, the device comprising: a first acquisition module, for acquiring first point cloud data; a point cloud completion module, for completing the first point cloud data using a point cloud completion network to obtain second point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates third point cloud data according to a target latent space vector, and the difference between fourth point cloud data obtained after a preset degradation process on the third point cloud data and real point cloud data collected from the physical space is within a preset difference range.

[0022] In some embodiments, the device further includes: an original point cloud acquisition module, used to acquire original point cloud data collected by a point cloud acquisition device from a physical space; and a point cloud segmentation module, used to perform point cloud segmentation on the original point cloud data to obtain the first point cloud data.

[0023] In some embodiments, the device further includes: an association processing module, configured to perform association processing on multiple frames of the second point cloud data.

[0024] In some embodiments, the device also includes: a training module, used to train the initial point cloud completion network based on the sample complete point cloud data; a third point cloud acquisition module, used to obtain the third point cloud data generated by the trained point cloud completion network based on the target latent space vector; a first optimization processing module, used to optimize the trained point cloud completion network based on the fourth point cloud data obtained after the preset degradation processing of the third point cloud data and the real point cloud data to obtain the point cloud completion network.

[0025] In some embodiments, the device also includes: a point cloud block acquisition module, used to acquire multiple point cloud blocks in the third point cloud data; a feature determination module, used to respectively determine the distribution characteristics of the points in each point cloud block; a loss function establishment module, used to establish a loss function based on the distribution characteristics of the points in each point cloud block; a second optimization processing module, used to optimize the trained point cloud completion network based on the loss function.

[0026] In some embodiments, the device also includes: a vector acquisition module, used to acquire multiple initial latent space vectors sampled from the latent space; an objective function determination module, used to determine the objective function of the initial latent space vector for each initial latent space vector based on the point cloud data corresponding to the initial latent space vector and the real point cloud data; a vector determination module, used to determine the target latent space vector from the initial latent space vectors based on the objective functions of the initial latent space vectors.

[0027] In some embodiments, the preset degradation processing is implemented based on the following modules: a neighboring point determination module, which is used to determine, for any target point in the real point cloud data, at least one point closest to the target point from the third point cloud data as a neighboring point of the target point; a point cloud determination module, which is used to determine the union of the neighboring points of each target point in the real point cloud data as the fourth point cloud data obtained by the preset degradation processing of the third point cloud data.

[0028] According to a sixth aspect of an embodiment of the present disclosure, a training device for a point cloud completion network is provided, the device comprising: a training module for training an initial point cloud completion network based on sample complete point cloud data; a second acquisition module for acquiring completed point cloud data generated by the trained point cloud completion network based on a target latent space vector; an optimization processing module for optimizing the trained point cloud completion network based on degraded point cloud data obtained after preset degradation processing of the completed point cloud data and real point cloud data collected from a physical space, so that the difference between the degraded point cloud data and the real point cloud data is within a preset difference range.

[0029] In some embodiments, the device also includes: a vector acquisition module, used to acquire multiple initial latent space vectors sampled from the latent space; a degenerate point cloud data acquisition module, used to respectively acquire the degenerate point cloud data obtained after preset degradation processing of the completed point cloud data generated by the trained point cloud completion network for each initial latent space vector; an objective function determination module, used to determine the objective function corresponding to the initial latent space vector based on the degenerate point cloud data corresponding to the initial latent space vector and the real point cloud data; a target latent space vector determination module, used to determine the target latent space vector from the initial latent space vectors based on the objective functions corresponding to the initial latent space vectors.

[0030] In some embodiments, the preset degradation processing includes: for any target point in the real point cloud data, determining at least one point closest to the target point from the sample complete point cloud data as a neighboring point of the target point; and determining the union of the neighboring points of each target point in the real point cloud data as the degraded point cloud data obtained by the preset degradation processing of the sample complete point cloud data.

[0031] According to a seventh aspect of an embodiment of the present disclosure, a point cloud data processing device is provided, the device comprising: a third acquisition module, used to acquire first point cloud data to be processed of game participants in a game area and second point cloud data to be processed corresponding to game objects; an input module, used to acquire first processed point cloud data obtained after a point cloud completion network completes the first point cloud data to be processed, and second processed point cloud data obtained after the point cloud completion network completes the second point cloud data to be processed; an association processing module, used to perform association processing on the first processed point cloud data and the second processed point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to a target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0032] In some embodiments, the device further includes: an action determination module for determining an action performed by the game participant on the game object based on an association result of the first processed point cloud data and the second processed point cloud data.

[0033] In some embodiments, the third acquisition module includes: an initial point cloud data acquisition unit, used to acquire the initial point cloud data collected by a point cloud acquisition device set around the game area; a point cloud segmentation unit, used to perform point cloud segmentation on the initial point cloud data to obtain the first point cloud data to be processed of the game participants and the second point cloud data to be processed corresponding to the game objects.

[0034] In some embodiments, the point cloud completion network is used to complete first point cloud data to be processed corresponding to multiple categories of game participants and / or second point cloud data to be processed corresponding to multiple categories of game objects; or the point cloud completion network includes a first point cloud completion network and a second point cloud completion network, the first point cloud completion network is used to complete the first point cloud data to be processed corresponding to the first category of game participants, and the second point cloud completion network is used to complete the second point cloud data to be processed corresponding to the second category of game objects.

[0035] According to an eighth aspect of an embodiment of the present disclosure, a point cloud data processing device is provided, the device comprising: a fourth acquisition module, for acquiring point cloud data to be processed of a target object in a game area; the target object comprises at least one of a game participant and a game object; a fifth acquisition module, for acquiring processed point cloud data obtained after a point cloud completion network completes the point cloud data to be processed; an identification module, for identifying the target object based on the processed point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to a target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0036] According to a ninth aspect of an embodiment of the present disclosure, a point cloud data processing system is provided, the system comprising: a point cloud acquisition device, arranged around a game area, for acquiring first point cloud data to be processed of game participants in the game area and second point cloud data to be processed corresponding to game objects; and a processing unit, communicatively connected to the point cloud acquisition device, for acquiring first processed point cloud data obtained after a point cloud completion network completes the first point cloud data to be processed, and second processed point cloud data obtained after the point cloud completion network completes the second point cloud data to be processed, and performing association processing on the first processed point cloud data and the second processed point cloud data; wherein, after being trained based on complete point cloud data, the point cloud completion network generates completed point cloud data according to a target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0037] According to a tenth aspect of an embodiment of the present disclosure, a point cloud data processing system is provided, the system comprising: a point cloud acquisition device, arranged around a game area, for acquiring point cloud data to be processed of a target object within the game area; the target object comprises at least one of a game participant and a game object; and a processing unit, communicatively connected to the point cloud acquisition device, for acquiring processed point cloud data obtained after a point cloud completion network completes the point cloud data to be processed, and identifying the target object based on the processed point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to a target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0038] According to an eleventh aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in any one of the embodiments is implemented.

[0039] According to the twelfth aspect of the embodiments of the present disclosure, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method described in any one of the embodiments is implemented when the processor executes the program.

[0040] During the training process, the point cloud completion network in the embodiment of the present disclosure only needs to use complete point cloud data as training data, without using point cloud pairs consisting of complete point cloud data and incomplete point cloud data. Furthermore, the difference between the fourth point cloud data obtained after the preset degradation processing of the third point cloud data generated by the trained point cloud completion network and the real point cloud data collected from the physical space is within a preset difference range, thereby ensuring that the output result of the point cloud completion network is close to the real point cloud data. Therefore, the solution of the embodiment of the present disclosure can complete any form of incomplete point cloud data, and the generalization performance is high.

[0041] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0043] Figure 1 is a schematic diagram of residual cloud data of some embodiments.

[0044] Figure 2 4 is a flowchart of a method for completing point cloud data according to an embodiment of the present disclosure.

[0045] Figure 3 Schematic diagram of the training and optimization process of the point cloud completion network according to an embodiment of the present disclosure.

[0046] Figure 4 It is a schematic diagram of the distribution characteristics of points in the point cloud data of an embodiment of the present disclosure.

[0047] Figure 5 Schematic diagram of the degradation process of the embodiment of the present disclosure.

[0048] Figure 6 It is a schematic diagram of various candidate complete point cloud data output by the point cloud completion network.

[0049] Figure 7 It is a flowchart of a training method of a point cloud completion network according to an embodiment of the present disclosure.

[0050] Figure 8 It is a flowchart of a method for processing point cloud data according to an embodiment of the present disclosure.

[0051] Fig. 9 The present invention is a flowchart of a method for processing point cloud data according to another embodiment of the present invention.

[0052] Fig.10 4 is a block diagram of a device for completing point cloud data according to an embodiment of the present disclosure.

[0053] Fig.11 is a block diagram of a training device for a point cloud completion network according to an embodiment of the present disclosure.

[0054] Fig.12 It is a block diagram of a point cloud data processing device according to an embodiment of the present disclosure.

[0055] Fig.13 It is a block diagram of a point cloud data processing device according to another embodiment of the present disclosure.

[0056] Fig.14 Schematic diagram of a point cloud data processing system according to an embodiment of the present disclosure.

[0057] Fig.15 It is a schematic diagram of the structure of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0059] The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of.

[0060] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings.

[0061] In practical applications, it is often necessary to collect point cloud data and perform some processing on the collected point cloud data. For example, in the field of autonomous driving, a laser radar will be installed on an automatically driven vehicle, and the laser radar will be used to collect point cloud data around the vehicle, and the point cloud data will be analyzed to determine the moving speed of obstacles around the vehicle, so as to effectively plan the path of the vehicle. For another example, in the field of robot navigation, point cloud data of the robot's surrounding environment can be collected, and the robot can be positioned based on various objects identified from the point cloud data. For another example, in some game scenarios, point cloud data in the game area can be collected, and various targets identified from the point cloud data (for example, game participants and game objects) can be associated.

[0062] However, in real scenes, due to occlusion and other reasons, the collected 3D point cloud is often not complete point cloud data, but incomplete point cloud data. For example, for a 3D object, its surface facing away from the point cloud acquisition device will be occluded by the surface facing the point cloud acquisition device, resulting in the point cloud on the surface facing away from the point cloud acquisition device cannot be collected. Even for planar objects, since there are often multiple overlapping objects in the scene, the surface of one object may be occluded by the surface of another object, resulting in incomplete collected point cloud data. In addition, there are many other reasons for the generation of incomplete point clouds, and the collected incomplete point clouds are also in various forms. For example, Figure 1 Shown is a schematic diagram of a residual point cloud collected in a physical space and its corresponding complete point cloud in some embodiments.

[0063] It should be noted that the incomplete point cloud data in the present disclosure refers to point cloud data that cannot represent the complete shape of an object. For example, when an object includes one or more surfaces, part of the surface or part of a surface area may be obscured, and the collected point cloud data does not include the points of the obscured surface or area, so that the collected point cloud data cannot represent the shape corresponding to the obscured surface or area. The multiple surfaces face different directions, or there are sudden changes in orientation between each other. Accordingly, complete point cloud data refers to point cloud data that can represent the complete shape of an object. For example, when an object includes one or more surfaces, the point cloud data includes points on each surface, so that the point cloud data can fully represent the shape of each surface.

[0064] Various operations based on the residual point cloud often fail to achieve the expected results. Therefore, it is necessary to perform point cloud completion on the residual point cloud data to obtain the complete point cloud data corresponding to the residual point cloud data. In the related art, generally, pairs of residual point cloud data and complete point cloud data are used as training data to perform full-supervision training on the point cloud completion network, and then the residual point cloud data is completed by the trained point cloud completion network. However, the form of the residual point cloud data is often diverse due to sensor types or occlusions, and the residual point cloud data in the training set is generally artificially generated based on a certain strategy, and its form is far less rich than the real residual point cloud collected from the physical space. Therefore, the point cloud completion network obtained based on the above method is difficult to generalize to other forms of residual point cloud data, and the generalization performance is poor.

[0065] Based on this, the present disclosure provides a method for completing point cloud data, such as Figure 2 , the method comprising:

[0066] Step 201: Acquire first point cloud data;

[0067] Step 202: using a point cloud completion network to complete the first point cloud data to obtain second point cloud data;

[0068] Among them, after being trained based on the complete point cloud data, the point cloud completion network generates third point cloud data according to the target latent space vector (latent code), and the difference between the fourth point cloud data obtained after the third point cloud data is subjected to a preset degradation process and the real point cloud data collected from the physical space is within a preset difference range.

[0069] In step 201 , the first point cloud data is partial point cloud data, and the first point cloud data may be acquired from a point cloud acquisition device after being acquired by the point cloud acquisition device.

[0070] In step 202, the first point cloud data may be input into a point cloud completion network, so as to complete the first point cloud data through the point cloud completion network to obtain second point cloud data, wherein the second point cloud data is complete point cloud data. The first point cloud data collected by the point cloud acquisition device may be directly input into the point cloud completion network for completion processing, or may be pre-processed first, and then the pre-processed first point cloud data may be input into the point cloud completion network for completion processing, wherein the pre-processing may include, for example, filtering processing for filtering out obviously erroneous points from the first point cloud data. The pre-processing may also include performing point cloud segmentation on the original point cloud data collected by the point cloud acquisition device from the physical space to obtain the first point cloud data.

[0071] The point cloud completion network in this step can be obtained based on any type of generative adversarial networks (GAN) including but not limited to tree-GAN or r-GAN. That is, the point cloud completion network can be obtained as a generator through adversarial training with a preset discriminator. The point cloud completion network can be trained using only complete point cloud data as training data, without collecting point cloud pairs consisting of complete point cloud data and incomplete point cloud data as training data for training. Since it is often difficult to collect complete point cloud data in real scenes, the complete point cloud data used in the present disclosure can be artificially generated, for example, complete point cloud data on the ShapeNet dataset. Through training, the point cloud completion network can learn better prior information of spatial geometry based on the complete point cloud data. After training, the network parameters of the point cloud completion network can be further optimized so that the fourth point cloud data obtained by the optimized point cloud completion network according to the target latent space vector after the preset degradation processing and the real point cloud data collected from the physical space are within the preset difference range, thereby achieving the point cloud completion network in an unsupervised manner. The optimized point cloud completion network is used to complete the first point cloud data.

[0072] In practice, the above-mentioned "the difference between the fourth point cloud data obtained after the preset degradation processing of the third point cloud data generated by the point cloud completion network according to the target latent space vector and the real point cloud data collected from the physical space is within a preset difference range" can be used as the optimization goal, and the parameters of the point cloud completion network can be adjusted by setting the corresponding optimization function to achieve the optimization of the point cloud completion network.

[0073] The method of the disclosed embodiment does not need to use point cloud pairs consisting of complete point cloud data and incomplete point cloud data. Since the entire training process does not involve incomplete point clouds, it can be applied to complete various forms of incomplete point clouds, has high generalization performance, and has good robustness for point clouds with different degrees of incompleteness. Moreover, since the point cloud completion network is optimized, the fourth point cloud data obtained by the third point cloud data generated by it after preset degradation processing has a small difference from the real point cloud data collected from the physical space. Therefore, the point cloud completion result is more accurate.

[0074] In some embodiments, the point cloud completion network can be obtained based on the following methods: training the initial point cloud completion network based on the sample complete point cloud data; obtaining the third point cloud data generated by the trained point cloud completion network based on the target latent space vector; optimizing the trained point cloud completion network based on the fourth point cloud data obtained after the third point cloud data is subjected to a preset degradation process and the real point cloud data, to obtain the point cloud completion network. The point cloud completion network obtained by the above method does not need to use residual point cloud data as training data during the training process, and can be applied to the completion of various forms of residual point cloud, with good generalization performance.

[0075] In the above embodiment, the best one can be selected from multiple initial latent space vectors as the target latent space vector. The multiple initial latent space vectors can be sampled from the latent space, and the sampling method can be random sampling. In some embodiments, the latent space can be a 96-dimensional space, and each sampling can randomly generate one or more 96-dimensional vectors, i.e., the initial latent space vector. For each initial latent space vector, the point cloud data generated by the initial point cloud completion network based on the initial latent space vector can be obtained, and the objective function of the initial latent space vector can be determined based on the point cloud data corresponding to the initial latent space vector and the real point cloud data. Then, based on the objective function of each initial latent space vector, the target latent space vector is determined from the initial latent space vectors.

[0076] The distance between the point cloud data corresponding to the initial latent space vector and the real point cloud data can be used as the objective function. In some embodiments, the chamfer distance (chamfer distance) and the feature distance (feature distance) between the fourth point cloud data and the real point cloud data can be determined, and the sum of the chamfer distance and the feature distance is determined as the objective function. The chamfer distance and the feature distance are as follows:

[0077] ;

[0078] ;

[0079] In the formula, and Represent chamfer distance and feature distance respectively, represents the fourth point cloud data obtained after the third point cloud data is subjected to a preset degradation process, Represents real point cloud data, and Respectively represent the points in the fourth point cloud data and the points in the real point cloud data, and denote the 1-norm and 2-norm respectively, and Respectively and The above is only an example of the objective function. In addition to the above objective function, other types of objective functions can also be used according to actual needs, which will not be repeated here.

[0080] After obtaining the objective functions corresponding to the initial latent space vectors, the initial latent space vector with the minimum objective function can be determined as the target latent space vector. Then, the objective function corresponding to the target latent space vector is obtained, and the network parameters of the point cloud completion network are optimized based on the objective function corresponding to the target latent space vector. The optimization method includes but is not limited to using the gradient descent method. During the optimization process, the target latent space vector and the network parameters of the point cloud completion network can be optimized at the same time to minimize the objective function corresponding to the target latent space vector.

[0081] The training and optimization process of the above point cloud completion network is as follows Figure 3 As shown in FIG. 1 , the generator in the generative adversarial network is used as the point cloud completion network. The generative adversarial network includes a generator G and a discriminator D. The two Ds shown in the figure can be the same discriminator. The above feature distance can be calculated based on the features output by the intermediate layer of the discriminator D in the trained generative adversarial network. In the training stage, multiple initial latent space vectors are randomly sampled from the latent space Rd and input into the generator G to obtain the complete point cloud data corresponding to each initial latent space vector, that is, the third point cloud data x c , x c After degradation, the residual cloud data x is obtained p , x p Compared with the real point cloud data x sampled in the physical space in The distance calculation is performed to obtain the objective function, thereby obtaining the optimal target latent space vector z. The gradient descent method is used to adjust the target latent space vector z and the parameters of the generator. Optimize to minimize the objective function corresponding to the target latent space vector, so as to obtain the optimized point cloud completion network.

[0082] Through the above method, the embodiment of the present disclosure can select the optimal target latent space vector from multiple initial latent space vectors for the optimization process of the point cloud completion network, which can accelerate the optimization speed of the point cloud completion network and improve the optimization efficiency.

[0083] In some embodiments, the third point cloud data may be degraded in the following manner: for any target point in the real point cloud data, at least one point adjacent to the target point is determined from the third point cloud data as a neighboring point of the target point; and the union of the neighboring points of each target point in the real point cloud data is determined as fourth point cloud data obtained by subjecting the third point cloud data to a preset degradation process.

[0084] like Figure 5 As shown, for the real point cloud data x in Point P1 in the third point cloud data x can be obtained c The neighboring points in the c The k points closest to P1 in the image are the points shown in area S1. Similarly, the real point cloud data x can be obtained. in Point P2 in the third point cloud data x c The neighboring points in the region S2 are similarly obtained by obtaining the real point cloud data x in Other target points in the third point cloud data x c The target point may include the real point cloud data x in Some points in the point cloud can be obtained from the real point cloud data x in The points uniformly sampled at the set sampling rate in . Optionally, the set sampling rate is less than 1 / k, so that the number of points in the point cloud data will be reduced after the degradation processing. The neighboring points of each target point may partially overlap, so the point cloud composed of the union of the neighboring points of each target point can be determined as the fourth point cloud data obtained after the third point cloud data is degraded.

[0085] In some embodiments, the distribution of points in the point cloud data is uneven, that is, the points in the complete point cloud data are densely distributed in some areas, and are dispersed in other areas. Figure 4 As shown in the figure, it is a comparison diagram of evenly distributed point cloud data a and unevenly distributed point cloud data b. It can be seen that in point cloud data b, most of the collected points are distributed in the positions shown in the dotted box, while the distribution of points in other areas is relatively scattered. Since the number of points that the point cloud completion network can process is relatively fixed, the unevenness of the point cloud data means that the number of points in some areas may not be sufficient for the point cloud completion network to obtain enough information for point cloud completion, resulting in inaccurate point cloud completion results. Therefore, in order to solve the above problems, during the training and optimization process of the point cloud completion network, the network parameters of the point cloud completion network can also be optimized to make the distribution of points in the point cloud data obtained by the point cloud completion network for point cloud completion more uniform.

[0086] During the training and optimization stages of the point cloud completion network, the point cloud data C output by the point cloud completion network based on the latent vector can be obtained, the distribution characteristics of the points in the point cloud data C can be obtained, a loss function can be established based on the distribution characteristics of the points in the point cloud data C, and the point cloud completion network can be optimized based on the loss function.

[0087] N seed positions can be randomly sampled from the point cloud data C. For example, the sampling method can adopt the farthest point sampling (FPS), so that the distance between each seed position is the farthest. The distribution characteristics of points in a point cloud block can be determined based on the average distance between each point in the point cloud block and a certain position in the point cloud block (for example, it can be a seed position). The loss function can be expressed as:

[0088] ;

[0089] In the formula, is the loss function, represents the variance, is the average distance between the midpoints of the jth point cloud block, n is the total number of point cloud blocks, k is the total number of points in the point cloud blocks, is the distance between the ith point in the jth point cloud block and the seed position. The network parameters of the point cloud completion network can be adjusted to minimize the variance of the average distances corresponding to each point cloud block in the point cloud data C output by the point cloud completion network. In this way, the distribution of points in each point cloud block can be made closer, thereby improving the uniformity of the distribution of points in the point cloud data output by the point cloud completion network.

[0090] The network optimization process based on the above-mentioned loss function can be performed simultaneously with the network optimization process based on the fourth point cloud data obtained after the degradation of the third point cloud data and the objective function obtained by the real point cloud data, or in any order, and the present disclosure does not impose any restrictions on this.

[0091] For each first point cloud data input, the point cloud completion network can output one or more candidate complete point cloud data. Figure 6 , which is a schematic diagram of the first point cloud data of a lamp and its corresponding candidate complete point cloud data provided by some embodiments. Based on the first point cloud data of the lamp, the point cloud completion network outputs a total of 4 candidate complete point cloud data for selection. Further, a selection instruction for each candidate complete point cloud data can be obtained, and in response to the selection instruction, one of the candidate complete point cloud data is selected as the second point cloud data.

[0092] The present disclosure can be used for any scene equipped with a 3D sensor (such as a depth camera or a laser radar), and the residual point cloud data of the entire scene can be obtained by scanning the 3D sensor. The residual point cloud data of each object in the scene is used to generate complete point cloud data through a point cloud completion network, and then the entire scene can be reconstructed in 3D. The reconstructed scene can provide accurate spatial information, such as detecting the distance between the human body and other objects in the scene, the distance between people, etc. The above spatial information can be used to associate people with objects and people with people, thereby improving the accuracy of the association.

[0093] In some embodiments, multiple frames of second point cloud data can be acquired, and the multiple frames of the second point cloud data can be associated. The multiple frames of second point cloud data can be the second point cloud data of objects of the same category. For example, in a game scene, each frame of second point cloud data can be the point cloud data corresponding to a game participant. By associating the point cloud data corresponding to multiple game participants, multiple game participants participating in the same game in the same game area can be determined. The multiple frames of second point cloud data can also be the second point cloud data of objects of different categories. Still taking the game scene as an example, the multiple frames of second point cloud data can include the point cloud data corresponding to the game participants and the point cloud data corresponding to the game objects. By associating the point cloud data corresponding to the game participants with the point cloud data corresponding to the game objects, the relationship between the game participants and the game objects can be determined, for example, the game chess and cards belonging to the game participants, the game area where the game participants are located, the seats where the game participants sit, etc.

[0094] The position and status of game participants and game objects in the game scene may change in real time, and the relationship between game participants and the relationship between game participants and game objects may also change in real time. These real-time changing information is of great significance for the analysis of game status and monitoring of game progress. Completing the incomplete point cloud data of game participants and / or game objects collected by the point cloud acquisition device is conducive to improving the accuracy of the association results between point cloud data, and further improving the reliability of game status analysis, game progress monitoring and other results based on the association results.

[0095] In some embodiments, after acquiring the second point cloud data, the object included in the second point cloud data may be identified to determine the category of the object. Multiple frames of the second point cloud data may also be associated based on the identification result. Further, in order to improve the accuracy of the association process and / or object identification, the second point cloud data may be homogenized before the association process and / or object identification.

[0096] In some embodiments, Figure 7As shown, the embodiment of the present disclosure also provides a training method for a point cloud completion network, the method comprising:

[0097] Step 701: training an initial point cloud completion network based on sample complete point cloud data;

[0098] Step 702: Acquire the completed point cloud data generated by the trained point cloud completion network based on the target latent space vector;

[0099] Step 703: Optimizing the trained point cloud completion network based on the degraded point cloud data obtained after the preset degradation processing of the completed point cloud data and the real point cloud data collected from the physical space, so that the difference between the degraded point cloud data and the real point cloud data is within a preset difference range.

[0100] In some embodiments, the method further includes: obtaining multiple initial latent space vectors sampled from the latent space; respectively obtaining degraded point cloud data obtained after preset degradation processing of the completed point cloud data generated by the trained point cloud completion network for each initial latent space vector; determining the objective function corresponding to the initial latent space vector based on the degraded point cloud data corresponding to the initial latent space vector and the real point cloud data; and determining the target latent space vector from each initial latent space vector based on the objective function corresponding to each initial latent space vector.

[0101] In some embodiments, the preset degradation processing includes: for any target point in the real point cloud data, determining at least one point closest to the target point from the sample complete point cloud data as a neighboring point of the target point; and determining the union of the neighboring points of each target point in the real point cloud data as the degraded point cloud data obtained by the preset degradation processing of the sample complete point cloud data.

[0102] The point cloud completion network trained based on the method of this embodiment can be used in the embodiment of the point cloud data completion method disclosed in the present invention to complete the first point cloud data. The details involved in the training process are detailed in the embodiment of the point cloud data completion method, which will not be repeated here.

[0103] In some embodiments, Figure 8 As shown, the embodiment of the present disclosure also provides a method for processing point cloud data, the method comprising:

[0104] Step 801: Acquire first to-be-processed point cloud data corresponding to the game participants in the game area and second to-be-processed point cloud data corresponding to the game objects;

[0105] Step 802: obtaining first processed point cloud data obtained by completing the first to-be-processed point cloud data by a point cloud completion network, and second processed point cloud data obtained by completing the second to-be-processed point cloud data by the point cloud completion network;

[0106] Step 803: performing association processing on the first processed point cloud data and the second processed point cloud data;

[0107] Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0108] The game participants may include but are not limited to at least one of a game referee, a game player, a game spectator, and the like.

[0109] In some embodiments, the method further comprises: determining an action performed by the game participant on the game object based on the association result of the first processed point cloud data and the second processed point cloud data. The action may include taking a seat, dealing cards, etc.

[0110] In some embodiments, the obtaining of first point cloud data to be processed of game participants in the game area and second point cloud data to be processed corresponding to game objects includes: obtaining initial point cloud data collected by a point cloud collection device arranged around the game area; performing point cloud segmentation on the initial point cloud data to obtain first point cloud data to be processed of the game participants and second point cloud data to be processed corresponding to game objects.

[0111] In some embodiments, the point cloud completion network is used to complete the first point cloud data to be processed corresponding to multiple categories of game participants and / or the second point cloud data to be processed corresponding to multiple categories of game objects. In this case, the point cloud completion network can be trained using multiple categories of complete point cloud data, and the network can be optimized using multiple categories of real point cloud data in the network optimization stage.

[0112] Alternatively, the point cloud completion network includes a first point cloud completion network and a second point cloud completion network, wherein the first point cloud completion network is used to complete the first point cloud data to be processed corresponding to the first category of game participants, and the second point cloud completion network is used to complete the second point cloud data to be processed corresponding to the second category of game objects. In this case, different point cloud completion networks can be trained with complete point cloud data of different categories, and each trained point cloud completion network is then optimized based on the real point cloud data of the corresponding category.

[0113] The point cloud completion network used in the embodiment of the present disclosure can be trained based on the training method of the aforementioned point cloud completion network. The point cloud completion network completes the first point cloud data to be processed and the second point cloud data to be processed in the same manner as the first point cloud data in the embodiment of the aforementioned point cloud data completion method. For details, please refer to the embodiment of the aforementioned point cloud data completion method, which will not be repeated here.

[0114] In some embodiments, Fig. 9 As shown, the embodiment of the present disclosure also provides a method for processing point cloud data, the method comprising:

[0115] Step 901: obtaining to-be-processed point cloud data of a target object in a game area; the target object includes at least one of a game participant and a game object;

[0116] Step 902: Obtain processed point cloud data obtained after the point cloud completion network completes the point cloud data to be processed;

[0117] Step 903: identifying the target object based on the processed point cloud data;

[0118] Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0119] In some embodiments, the step of obtaining the unprocessed point cloud data of the target objects within the game area includes: obtaining the initial point cloud data collected by a point cloud collection device arranged around the game area; performing point cloud segmentation on the initial point cloud data to obtain the unprocessed point cloud data of the target objects of each category within the game area.

[0120] In some embodiments, the point cloud completion network is used to complete the to-be-processed point cloud data of target objects of multiple categories; or the point cloud completion network is used to complete the to-be-processed point cloud data of target objects of a specific category.

[0121] The point cloud completion network used in the disclosed embodiment can be trained based on the training method of the aforementioned point cloud completion network. The point cloud completion network completes the point cloud data in the same way as the first point cloud data in the embodiment of the aforementioned point cloud data completion method. For details, please refer to the embodiment of the aforementioned point cloud data completion method, which will not be repeated here. Those skilled in the art will understand that in the above method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0122] like Fig.10 As shown, the present disclosure also provides a point cloud data completion device, the device comprising: a first acquisition module 1001, used to acquire first point cloud data; a point cloud completion module 1002, used to complete the first point cloud data using a point cloud completion network to obtain second point cloud data; wherein, after the point cloud completion network is trained based on the complete point cloud data, it generates third point cloud data according to the target latent space vector, and the difference between the fourth point cloud data obtained after the third point cloud data is subjected to a preset degradation process and the real point cloud data collected from the physical space is within a preset difference range.

[0123] In some embodiments, the device further includes: an original point cloud acquisition module, used to acquire original point cloud data collected by a point cloud acquisition device from a physical space; and a point cloud segmentation module, used to perform point cloud segmentation on the original point cloud data to obtain the first point cloud data.

[0124] In some embodiments, the device further includes: an association processing module, configured to perform association processing on multiple frames of the second point cloud data.

[0125] In some embodiments, the device also includes: a training module, used to train the initial point cloud completion network based on the sample complete point cloud data; a third point cloud acquisition module, used to obtain the third point cloud data generated by the trained point cloud completion network based on the target latent space vector; a first optimization processing module, used to optimize the trained point cloud completion network based on the fourth point cloud data obtained after the preset degradation processing of the third point cloud data and the real point cloud data to obtain the point cloud completion network.

[0126] In some embodiments, the device also includes: a point cloud block acquisition module, used to acquire multiple point cloud blocks in the third point cloud data; a feature determination module, used to respectively determine the distribution characteristics of the points in each point cloud block; a loss function establishment module, used to establish a loss function based on the distribution characteristics of the points in each point cloud block; a second optimization processing module, used to optimize the trained point cloud completion network based on the loss function.

[0127] In some embodiments, the device also includes: a vector acquisition module, used to acquire multiple initial latent space vectors sampled from the latent space; an objective function determination module, used to determine the objective function of the initial latent space vector for each initial latent space vector based on the point cloud data corresponding to the initial latent space vector and the real point cloud data; a vector determination module, used to determine the target latent space vector from the initial latent space vectors based on the objective functions of the initial latent space vectors.

[0128] In some embodiments, the preset degradation processing is implemented based on the following modules: a neighboring point determination module, which is used to determine, for any target point in the real point cloud data, at least one point closest to the target point from the third point cloud data as a neighboring point of the target point; a point cloud determination module, which is used to determine the union of the neighboring points of each target point in the real point cloud data as the point cloud data obtained by the preset degradation processing of the third point cloud data.

[0129] like Fig.11 As shown, the present disclosure also provides a training device for a point cloud completion network, the device comprising: a training module 1101, used to train an initial point cloud completion network based on sample complete point cloud data; a second acquisition module 1102, used to obtain the completed point cloud data generated by the trained point cloud completion network based on the target latent space vector; an optimization processing module 1103, used to optimize the trained point cloud completion network based on the degraded point cloud data obtained after the preset degradation processing of the completed point cloud data and the real point cloud data collected from the physical space, so that the difference between the degraded point cloud data and the real point cloud data is within a preset difference range.

[0130] In some embodiments, the device also includes: a vector acquisition module, used to acquire multiple initial latent space vectors sampled from the latent space; a degenerate point cloud data acquisition module, used to respectively acquire the degenerate point cloud data obtained after preset degradation processing of the completed point cloud data generated by the trained point cloud completion network for each initial latent space vector; an objective function determination module, used to determine the objective function corresponding to the initial latent space vector based on the degenerate point cloud data corresponding to the initial latent space vector and the real point cloud data; a target latent space vector determination module, used to determine the target latent space vector from the initial latent space vectors based on the objective functions corresponding to the initial latent space vectors.

[0131] In some embodiments, the preset degradation processing includes: for any target point in the real point cloud data, determining at least one point closest to the target point from the sample complete point cloud data as a neighboring point of the target point; and determining the union of the neighboring points of each target point in the real point cloud data as the degraded point cloud data obtained by the preset degradation processing of the sample complete point cloud data.

[0132] like Fig.12As shown, the present disclosure also provides a point cloud data processing device, the device comprising: a third acquisition module 1201, used to acquire first unprocessed point cloud data of game participants in a game area and second unprocessed point cloud data corresponding to game objects; an input module 1202, used to acquire first processed point cloud data obtained after a point cloud completion network completes the first unprocessed point cloud data, and second processed point cloud data obtained after the point cloud completion network completes the second unprocessed point cloud data; an association processing module 1203, used to perform association processing on the first processed point cloud data and the second processed point cloud data; wherein, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0133] In some embodiments, the device further includes: an action determination module for determining an action performed by the game participant on the game object based on an association result of the first processed point cloud data and the second processed point cloud data.

[0134] In some embodiments, the third acquisition module includes: an initial point cloud data acquisition unit, used to acquire the initial point cloud data collected by a point cloud acquisition device set around the game area; a point cloud segmentation unit, used to perform point cloud segmentation on the initial point cloud data to obtain the first point cloud data to be processed of the game participants and the second point cloud data to be processed corresponding to the game objects.

[0135] In some embodiments, the point cloud completion network is used to complete first point cloud data to be processed corresponding to multiple categories of game participants and / or second point cloud data to be processed corresponding to multiple categories of game objects; or the point cloud completion network includes a first point cloud completion network and a second point cloud completion network, the first point cloud completion network is used to complete the first point cloud data to be processed corresponding to the first category of game participants, and the second point cloud completion network is used to complete the second point cloud data to be processed corresponding to the second category of game objects.

[0136] like Fig.13As shown, the present disclosure also provides a point cloud data processing device, the device comprising: a fourth acquisition module 1301, used to obtain the unprocessed point cloud data of the target object in the game area; the target object includes at least one of the game participants and the game objects; a fifth acquisition module 1302, used to obtain the processed point cloud data obtained after the point cloud completion network completes the unprocessed point cloud data; an identification module 1303, used to identify the target object based on the processed point cloud data; wherein, after the point cloud completion network is trained based on the complete point cloud data, it generates the completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the preset degradation processing of the completed point cloud data and the real point cloud data collected from the physical space is within a preset difference range.

[0137] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0138] like Fig.14 As shown, the embodiment of the present disclosure further provides a point cloud data processing system, the system comprising a point cloud acquisition device 1401 arranged around a game area 1403 ; and a processing unit 1402 communicatively connected to the point cloud acquisition device 1401 .

[0139] The point cloud acquisition device 1401 can collect the first unprocessed point cloud data of the game participants 1404 in the game area 1403 and the second unprocessed point cloud data corresponding to the game object 1405. The processing unit 1402 can obtain the first processed point cloud data obtained after the point cloud completion network completes the first unprocessed point cloud data, and the second processed point cloud data obtained after the point cloud completion network completes the second unprocessed point cloud data, and perform association processing on the first processed point cloud data and the second processed point cloud data.

[0140] The point cloud acquisition device 1401 can collect the to-be-processed point cloud data of the target object in the game area; the target object includes at least one of a game participant and a game object. The processing unit 1402 can obtain the processed point cloud data obtained after the point cloud completion network completes the to-be-processed point cloud data, and identify the target object based on the processed point cloud data.

[0141] Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

[0142] In some embodiments, the point cloud acquisition device 1401 may be a laser radar or a depth camera. One or more point cloud acquisition devices 1401 may be arranged around the game area, and different point cloud acquisition devices 1401 may acquire point cloud data of different sub-areas in the game area, and the sub-areas acquired by different point cloud acquisition devices 1401 may overlap.

[0143] The number of game participants in the game area may be one or more, and each game participant may correspond to one or more game objects, including but not limited to seats, chess and cards, and sign-type props. Based on the processed point cloud data, the target object is identified, and the categories of objects included in different point cloud data can be determined, and the spatial information of objects of each category can also be determined. By associating the first processed point cloud data with the second processed point cloud data, the relationship between various game objects and game participants can be obtained, and the actions performed by the game participants can also be determined, and then it can be judged whether the actions performed by the game participants comply with the pre-set game rules.

[0144] An embodiment of the present specification also provides a computer device, which at least includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any of the above embodiments is implemented.

[0145] Fig.15 A more specific hardware structure diagram of a computing device provided in an embodiment of this specification is shown, and the device may include: a processor 1501, a memory 1502, an input / output interface 1503, a communication interface 1504, and a bus 1505. The processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504 are connected to each other in communication within the device through the bus 1505.

[0146] The processor 1501 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided in the embodiments of this specification. The processor 1501 can also include a graphics card, which can be a Nvidia titan X graphics card or a 1080Ti graphics card, etc.

[0147] The memory 1502 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1502 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1502 and are called and executed by the processor 1501.

[0148] The input / output interface 1503 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0149] The communication interface 1504 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0150] The bus 1505 includes a path that transmits information between various components of the device (eg, the processor 1501 , the memory 1502 , the input / output interface 1503 , and the communication interface 1504 ).

[0151] It should be noted that, although the above device only shows the processor 1501, the memory 1502, the input / output interface 1503, the communication interface 1504 and the bus 1505, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0152] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0153] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0154] It can be known from the above description of the implementation mode that the technicians in this field can clearly understand that the embodiments of this specification can be implemented by means of software plus the necessary general hardware platform. Based on such an understanding, the technical solution of the embodiments of this specification can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments of this specification.

[0155] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0156] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment scheme of this specification. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment scheme. A person of ordinary skill in the art can understand and implement it without paying creative labor.

Claims

1. A method for completing point cloud data, comprising: Acquire the first point cloud data; Using a point cloud completion network to complete the first point cloud data to obtain second point cloud data; Among them, after being trained based on the complete point cloud data, the point cloud completion network generates third point cloud data according to the target latent space vector, and the difference between the fourth point cloud data obtained after the third point cloud data is subjected to a preset degradation process and the real point cloud data collected from the physical space is within a preset difference range.

2. The method according to claim 1, further comprising: Acquire original point cloud data collected by a point cloud collection device from a physical space; Performing point cloud segmentation on the original point cloud data to obtain the first point cloud data; and / or Perform association processing on multiple frames of the second point cloud data.

3. The method according to claim 1 or 2, further comprising obtaining the point cloud completion network based on the following manner: Train the initial point cloud completion network based on the sample complete point cloud data; Acquire third point cloud data generated by the trained point cloud completion network based on the target latent space vector; Based on the fourth point cloud data and the real point cloud data, the trained point cloud completion network is optimized to obtain the point cloud completion network.

4. The method according to claim 1, wherein: Acquire a plurality of point cloud blocks in the third point cloud data; Determine the distribution characteristics of the points in each point cloud block respectively; Based on the distribution characteristics of the points in each point cloud block, a loss function is established; The trained point cloud completion network is optimized based on the loss function.

5. The method according to claim 1, wherein: Obtain multiple initial latent space vectors sampled from the latent space; For each initial latent space vector, determining an objective function of the initial latent space vector based on point cloud data corresponding to the initial latent space vector and the real point cloud data; Based on the objective function of each initial latent space vector, the target latent space vector is determined from the initial latent space vectors.

6. The method according to claim 1, wherein: The preset degradation process includes: For any target point in the real point cloud data, determining at least one point adjacent to the target point from the third point cloud data as a neighboring point of the target point; The union of neighboring points of each target point in the real point cloud data is determined as the fourth point cloud data obtained by subjecting the third point cloud data to a preset degradation process.

7. A method for training a point cloud completion network, comprising: Train the initial point cloud completion network based on the sample complete point cloud data; Acquire the completed point cloud data generated by the trained point cloud completion network based on the target latent space vector; The trained point cloud completion network is optimized based on the degraded point cloud data obtained after the preset degradation processing of the completed point cloud data and the real point cloud data collected from the physical space, so that the difference between the degraded point cloud data and the real point cloud data is within a preset difference range.

8. A method for processing point cloud data, comprising: Acquire first to-be-processed point cloud data corresponding to the game participants in the game area and second to-be-processed point cloud data corresponding to the game objects; Acquire first processed point cloud data obtained after the point cloud completion network completes the first to-be-processed point cloud data, and obtain second processed point cloud data obtained after the point cloud completion network completes the second to-be-processed point cloud data; performing association processing on the first processed point cloud data and the second processed point cloud data; Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

9. A method for processing point cloud data, the method comprising: Get the point cloud data to be processed of the target object in the game area; The target object includes at least one of a game participant and a game object; Acquire processed point cloud data obtained after the point cloud completion network completes the point cloud data to be processed; identifying the target object based on the processed point cloud data; Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

10. A device for completing point cloud data, the device comprising: A first acquisition module, used to acquire first point cloud data; A point cloud completion module, used to complete the first point cloud data using a point cloud completion network to obtain second point cloud data; Among them, after being trained based on the complete point cloud data, the point cloud completion network generates third point cloud data according to the target latent space vector, and the difference between the fourth point cloud data obtained after the third point cloud data is subjected to a preset degradation process and the real point cloud data collected from the physical space is within a preset difference range.

11. A training device for a point cloud completion network, the device comprising: The training module is used to train the initial point cloud completion network based on the sample complete point cloud data; A second acquisition module is used to acquire the completed point cloud data generated by the trained point cloud completion network based on the target latent space vector; The optimization processing module is used to optimize the trained point cloud completion network based on the degraded point cloud data obtained after the preset degradation processing of the completed point cloud data and the real point cloud data collected from the physical space, so that the difference between the degraded point cloud data and the real point cloud data is within a preset difference range.

12. A point cloud data processing device, the device comprising: A third acquisition module is used to acquire first to-be-processed point cloud data of game participants in the game area and second to-be-processed point cloud data corresponding to the game objects; An input module, used to obtain first processed point cloud data obtained after the point cloud completion network completes the first to-be-processed point cloud data, and second processed point cloud data obtained after the point cloud completion network completes the second to-be-processed point cloud data; An association processing module, used for performing association processing on the first processed point cloud data and the second processed point cloud data; Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

13. A point cloud data processing device, the device comprising: A fourth acquisition module is used to acquire the to-be-processed point cloud data of the target object in the game area; The target object includes at least one of a game participant and a game object; A fifth acquisition module, used to acquire processed point cloud data obtained after the point cloud completion network completes the point cloud data to be processed; an identification module, configured to identify the target object based on the processed point cloud data; Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

14. A point cloud data processing system, the system comprising: A point cloud acquisition device is arranged around the game area and is used to acquire first to-be-processed point cloud data corresponding to the game participants in the game area and second to-be-processed point cloud data corresponding to the game objects; as well as a processing unit, communicatively connected to the point cloud acquisition device, configured to obtain first processed point cloud data obtained after the point cloud completion network completes the first to-be-processed point cloud data, and second processed point cloud data obtained after the point cloud completion network completes the second to-be-processed point cloud data, and to perform association processing on the first processed point cloud data and the second processed point cloud data; Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

15. A point cloud data processing system, the system comprising: A point cloud acquisition device is arranged around the game area and is used to acquire the to-be-processed point cloud data of the target object in the game area; The target object includes at least one of a game participant and a game object; as well as a processing unit, which is in communication with the point cloud acquisition device, and is used to obtain processed point cloud data obtained after the point cloud completion network completes the point cloud data to be processed, and identify the target object based on the processed point cloud data; Among them, after being trained based on the complete point cloud data, the point cloud completion network generates completed point cloud data according to the target latent space vector, and the difference between the degraded point cloud data obtained after the completed point cloud data is subjected to a preset degradation processing and the real point cloud data collected from the physical space is within a preset difference range.

16. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

17. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 9 when executing the program.

Images