Dynamic point cloud up-sampling method based on generative adversarial network inverse mapping technology

Abstract

The invention belongs to the technical field of computer vision, and provides a dynamic point cloud up-sampling method based on a generative adversarial network inverse mapping technology, which comprises the following steps: inputting a to-be-processed point cloud into a dynamic point cloud up-sampling model to obtain an up-sampling point cloud; the point cloud dynamic up-sampling model comprises an L-level generator and a multi-level refining mechanism module; (l-1) -level up-sampling processing in the multi-level refining mechanism module: when l = 1, mapping the to-be-processed point cloud into a potential spatial feature; when lgt; the method comprises the following steps: 1, mapping an (l-2) th level generation point cloud into potential spatial features; performing potential space dynamic optimization on the potential space features to obtain optimized potential vectors; decoding the optimized potential vector to obtain an (l-1) th-level generation point cloud; and if l is equal to L, taking the (l-1) th level generation point cloud as an upper sampling point cloud. The invention also discloses two point cloud dynamic upsampling model training methods and an electronic device. According to the method, the distribution uniformity and geometric detail expressive force of the upper sampling point cloud can be optimized.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a point cloud dynamic upsampling method based on a generative adversarial network inverse mapping technology. Background Art

[0002] With the rapid development of computer vision technology, point cloud data, as an important form of expression of three-dimensional spatial information, has been widely used in fields such as autonomous driving, 3D modeling, robot navigation, virtual reality and augmented reality. However, the sparsity and irregularity of point cloud data are still the main problems that hinder its further promotion and application in high-precision perception tasks. Sparse point clouds are usually characterized by low sampling density, uneven spatial distribution and missing details, which puts higher requirements on the integrity and geometric characteristics of point clouds.

[0003] The existence of sparse point clouds has a significant impact on the effectiveness of data processing and analysis. For example, in autonomous driving scenarios, sparse point clouds are difficult to fully express the complex geometric structure of the object surface, thus affecting the accuracy of environmental perception and object detection; in 3D reconstruction tasks, the uneven distribution of sparse point clouds will lead to missing details in the reconstruction results, increasing the difficulty of restoring complex geometric shapes. Therefore, how to efficiently improve the resolution, distribution uniformity, and geometric consistency of sparse point clouds has become a key issue that needs to be solved urgently.

[0004] At present, point cloud upsampling technology is regarded as an important means to solve the problem of sparse point clouds. Traditional geometric interpolation methods, such as spline interpolation and shortest path interpolation, generate new points by fitting curves or surfaces to point cloud data. Such methods have low computational cost and are suitable for regular scenes, but it is difficult to maintain details and global consistency in complex geometric structures. Deep learning methods, such as PointNet++, generate higher density data by learning point cloud features, but their generated results often have problems with local inhomogeneity and lack of geometric details. In recent years, generative adversarial networks (GANs) have been introduced into the field of point cloud upsampling due to their outstanding performance in data generation tasks. Some studies have shown that GANs can be used to achieve densification and reconstruction of point clouds, but the generated point clouds may still have problems such as insufficient geometric consistency or uneven distribution. Although these methods have made some progress in the enhancement task of sparse point clouds, in the upsampling process, local areas may be overly dense, while other areas are still sparse, with poor overall distribution uniformity and insufficient details. Summary of the invention

[0005] The present invention aims to at least solve the technical problems of poor overall distribution uniformity and insufficient details of point clouds existing in existing point cloud upsampling methods, and provide a point cloud dynamic upsampling method, training method and electronic device based on generative adversarial network inverse mapping technology.

[0006] To achieve the above object of the present invention, according to the first aspect of the present invention, there is provided a point cloud dynamic upsampling method based on the inverse mapping technology of the generative adversarial network, including:

[0007] Obtain the point cloud to be processed, and input the point cloud to be processed into a pre-trained point cloud dynamic upsampling model to obtain an upsampled point cloud;

[0008] Wherein, the pre-trained point cloud dynamic upsampling model includes an L-level generator and a multi-level refinement mechanism module; the multi-level refinement mechanism module uses the L-level generator to perform L-level upsampling processing on the point cloud to be processed to obtain an upsampled point cloud, L is a positive integer, and the (l - 1)-th level upsampling processing process includes:

[0009] When l = 1, use the encoding network of the (l - 1)-th level generator to map the point cloud to be processed into latent space features; when l > 1, use the encoding network of the (l - 1)-th level generator to map the (l - 2)-th level generated point cloud into latent space features;

[0010] Perform latent space dynamic optimization processing on the latent space features to obtain an optimized latent vector;

[0011] Use the decoding network of the (l - 1)-th level generator to decode the optimized latent vector to obtain the (l - 1)-th level generated point cloud; 1 ≤ l ≤ L;

[0012] If l = L, then use the (l - 1)-th level generated point cloud as the upsampled point cloud, if l < L, then enter the next level of upsampling processing.

[0013] The beneficial technical effects of the above point cloud dynamic upsampling method based on the inverse mapping technology of the generative adversarial network are as follows: a multi-level refinement mechanism module is adopted to map the low-resolution point cloud to be processed into a high-dimensional latent space, so as to obtain a compact global feature representation, that is, latent space features. The latent space features can capture the global structure information of the point cloud and provide guidance for generating high-resolution point clouds level by level. It can generate high-density point clouds level by level, that is, upsampled point clouds, effectively solving the problem of poor global consistency in traditional methods, optimizing the uniformity and geometric characteristics of the generated point clouds, improving the resolution of sparse point clouds, and at the same time introducing latent space dynamic optimization processing (online data augmentation mechanism) in each level of upsampling processing. Combining with the adversarial network framework, the generator is made to focus on the generation of point clouds in sparse regions first. Combining with the step-by-step refinement strategy, it can optimize the distribution uniformity and geometric detail expressiveness of the upsampled point clouds.

[0014] To achieve the above object of the present invention, according to the second aspect of the present invention, there is provided a method for training a point cloud dynamic upsampling model, including:

[0015] Step A1, construct a training sample point cloud set, and each training sample point cloud corresponds to a real point cloud;

[0016] Step A2, construct a generative adversarial network and initialize the network parameters of the adversarial network. The generative adversarial network includes a generator network and a discriminator network. Among them, the generator network includes an L-level generator and a multi-level refinement mechanism module. The multi-level refinement mechanism module performs L-level upsampling on the training sample point cloud using the L-level generator to obtain a sample generated point cloud. L is a positive integer. The (l - 1)-th level upsampling process includes:

[0017] When l = 1, use the encoding network of the (l - 1)-th level generator to map the training sample point cloud to latent space features. When l > 1, use the encoding network of the (l - 1)-th level generator to map the (l - 2)-th level generated point cloud to latent space features.

[0018] Perform latent space dynamic optimization processing on the latent space features to obtain an optimized latent vector.

[0019] Use the decoding network of the (l - 1)-th level generator to decode the optimized latent vector to obtain the (l - 1)-th level generated point cloud. 1 ≤ l ≤ L.

[0020] If l = L, then use the (l - 1)-th level generated point cloud as the sample generated point cloud. If l < L, then proceed to the next level of upsampling processing.

[0021] Step A3, alternately train the discriminator network and the generator network using the training sample point cloud set until the training stop condition is reached. During the alternate training, optimize the network parameters of the discriminator network according to the discriminator loss through the backpropagation algorithm, and optimize the network parameters of the generator network according to the generator loss through the backpropagation algorithm.

[0022] Step A4, use the generator network obtained after training as the point cloud dynamic upsampling model.

[0023] The beneficial technical effects of the above point cloud dynamic upsampling model training method: In addition to the generator network having the beneficial technical effects of the first aspect of the present invention, it also has the technical effect that in view of the sparsity and irregularity of point cloud data, a generative adversarial network structure is designed, and an alternate training method is adopted, so that the generated point cloud is closer to the real point cloud.

[0024] To achieve the above object of the present invention, according to the third aspect of the present invention, the present invention provides a point cloud dynamic upsampling model training method, including:

[0025] Step B1, construct a teacher generator network, a student generator network, and a discriminator network.

[0026] Step B2, alternately train the teacher generator network and the discriminator network according to the point cloud dynamic upsampling model training method described in the second aspect of the present invention until the training stop condition is reached, and use the teacher generator network obtained after training as the teacher upsampling model.

[0027] Step B3, using the training sample point cloud set to train the student generated network until the student training stop condition is reached, the student generated network is used as the point cloud dynamic upsampling model, in each training, the training sample point cloud is input into the student generated network and the teacher upsampling model respectively, the student generated point cloud output by the student generated network and the teacher generated point cloud output by the teacher upsampling model are calculated distillation loss, and the network parameters of the student generated network are updated by the back propagation method according to the distillation loss;

[0028] Step B4, using the student generated network obtained after training as the point cloud dynamic upsampling model.

[0029] The beneficial technical effects of the above-mentioned point cloud dynamic upsampling model training method: In addition to the beneficial technical effects of the first and second aspects of the present invention possessed by the generated network, the above-mentioned method introduces a model distillation method as an innovative solution for model lightweighting and compression to address the problems of complex structure and slow inference speed of the generative adversarial network (GAN). By transferring the knowledge of the high-performance teacher upsampling model to the lightweight student generation network, the parameters are greatly reduced while retaining the generation performance, thereby improving the usability of the point cloud dynamic upsampling model.

[0030] In order to achieve the above-mentioned object of the present invention, according to a fourth aspect of the present invention, the present invention provides an electronic device, the electronic device comprising:

[0031] at least one processor; and,

[0032] A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method described in the first aspect, the second aspect, or the third aspect above.

[0033] The above-mentioned electronic device has the beneficial technical effects of the first aspect, the second aspect or the third aspect, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of a point cloud dynamic upsampling method based on a generative adversarial network inverse mapping technology in a preferred embodiment of the present invention;

[0035] Figure 2 It is a structural schematic diagram of a point cloud dynamic upsampling model in a preferred embodiment of the present invention;

[0036] Figure 3 It is a schematic diagram of a generative adversarial network in a point cloud dynamic upsampling model training method in a preferred embodiment of the present invention;

[0037] Figure 4 It is a schematic diagram of the overall process in an application scenario of the present invention;

[0038] Figure 5 It is a schematic diagram of the structure of an electronic device in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0039] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0040] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0041] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0042] The execution subject of the point cloud dynamic upsampling method or point cloud dynamic upsampling model training method based on the generative adversarial network inverse mapping technology provided by the present invention includes but is not limited to at least one of the electronic devices such as the server and the terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the point cloud dynamic upsampling method or point cloud dynamic upsampling model training method based on the generative adversarial network inverse mapping technology can be executed by software or hardware installed on the terminal device or the server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0043] The present invention discloses a point cloud dynamic upsampling method based on a generative adversarial network inverse mapping technology. In a preferred embodiment, Figure 1 As shown, the upsampling method includes:

[0044] Step S1, obtaining the point cloud to be processed. The point cloud to be processed is preferably, but not limited to, point cloud data of a vehicle or robot road or the surface of an object to be three-dimensionally reconstructed, collected by at least one point cloud acquisition device such as a laser radar (LiDAR), a structured light scanner, and a depth camera. Each point is a three-dimensional coordinate of a position point. These point cloud data are sparse and irregular, and have problems such as low sampling density, uneven spatial distribution, and missing details. In order to improve processing efficiency, after obtaining the original point cloud output by the point cloud acquisition device, it also includes preprocessing the original point cloud:

[0045] Normalization process, adjust the original point cloud range to the unit cube [0,1] 3 In order to ensure the consistency of distribution;

[0046] De-noising filtering, preferably but not limited to, removing abnormal points in the point normalized point cloud by an outlier filtering algorithm, thereby obtaining a point cloud to be processed.

[0047] Step S2: input the point cloud to be processed into a pre-trained point cloud dynamic upsampling model to obtain an upsampled point cloud.

[0048] The pre-trained point cloud dynamic upsampling model includes an L-level generator and a multi-level refinement mechanism module. Figure 2 It can be seen that the L-level generator (generator G0, generator G1, ..., generator G L-1 ) cascade, each level of generator refines the point cloud based on the generated point cloud output by the previous level generator and the optimized potential vector calculated at this level to obtain a new generated point cloud. The multi-level refinement mechanism module uses L-level generators to perform L-level upsampling processing on the processed point cloud to obtain an upsampled point cloud, where L is a positive integer. The l-1th level upsampling processing process includes:

[0049] Step S21, when l=1, the encoding network of the l-1th level generator (generator G0) is used to map the point cloud to be processed into latent space features; when l>1, the encoding network of the l-1th level generator is used to map the l-2th level generated point cloud into latent space features. The l-2th level generated point cloud represents the generated point cloud output by the l-2th level generator.

[0050] Latent space features Among them, Encoder(·) represents the encoding network of the generator, Represents the input point cloud of the generator’s encoder network. k represents the dimension of the latent space. The latent space feature z includes multiple latent space sampling vectors, and the number of latent space sampling vectors may not be consistent with the number of points in the point cloud input to the encoding network. The number of latent space sampling vectors is empirically set or adjusted according to the complexity of the point cloud, the resolution requirement, computing resources, etc.

[0051] Step S22: Perform latent space dynamic optimization processing on the latent space feature to obtain the optimized latent vector z optimized～l-1 ;

[0052] Step S23: Use the decoding network of the (l - 1)-th level generator to decode the optimized latent vector to obtain the (l - 1)-th level generated point cloud; 1 ≤ l ≤ L;

[0053] Step S24: If l = L, use the (l - 1)-th level generated point cloud as the upsampled point cloud; if l < L, proceed to the next level of upsampling processing.

[0054] In this embodiment, the encoding network of the generator is preferably but not limited to an existing multi-scale feature extraction network model, such as the existing PointNet++ network or multi-scale convolutional network, which can efficiently extract local and global geometric features of the point cloud. The decoding network of the generator can adopt an existing decoding network, such as the decoding network of the point cloud generation model PointFlow, which is designed under the variational autoencoder framework VAE and can generate point clouds through the latent representation of data.

[0055] In this embodiment, referring to Figure 2 , the process of the L-level generator and the multi-level refinement mechanism module of the point cloud dynamic upsampling model gradually refining the point cloud resolution is to gradually generate a higher-resolution point cloud by using the optimized latent vector in combination with the low-resolution point cloud. Each level of the generator combines local geometric characteristics and global latent features and is responsible for refining the point cloud from one resolution level to a higher level:

[0056] Initial layer (generator G0), starting from the point cloud to be processed and simultaneously receiving the optimized latent vector z optimized～0 of the 0-th level as input for initial refinement:

[0057]

[0058] Middle layer (generator G l ), each middle generator G l receives the generated point cloud of the previous level and z optimized～l as input and outputs the generated point cloud

[0059]

[0060] The final layer (Generator G L-1 ), the last generator G L-1 Output the final high-resolution point cloud

[0061]

[0062] Among them, z optimized The features provide guidance for global consistency. The refinement operation of each layer includes: the generator of this layer uses the network parameters obtained through training to add new sampling points at appropriate locations according to local geometric features and global latent features to fill the sparse areas of the original point cloud; and optimizes the positions of the newly generated points by combining local geometric features and global latent features. Starting from the low-resolution point cloud, the level l generator gradually generates a high-resolution point cloud through the guidance of the latent space features, and finally generates a high-resolution point cloud with a distribution close to the real point cloud.

[0063] In a preferred embodiment, during the upsampling process at level l-1, the latent space feature is subjected to latent space dynamic optimization processing to obtain an optimized latent vector in step S22, including:

[0064] Step S221, determining the sparse area of ​​the input point cloud for the l-1th level upsampling processing and the potential vector target in the sparse area.

[0065] Specifically, the local density in the neighborhood of each point in the input point cloud is obtained, and points with local density less than a preset local density threshold are classified as sparse areas, so that more than one sparse area is identified. The local density calculation method of the point is preferably but not limited to calculating the number of points in the neighborhood of the point.

[0066] Specifically, for each sparse region, one or more latent space sampling vectors corresponding to each sparse region are extracted from the input point cloud of the l-1th level generator, and clustering methods such as K-means and DBSACN are used to determine the cluster center points of the one or more latent space sampling vectors corresponding to each sparse region, and the latent space sampling vectors of the cluster center points are used as the latent vector targets of each sparse region. When there are more than two sparse regions, the latent vector targets of all sparse regions are grouped together, and this group is called a latent vector target group.

[0067] Step S222, obtaining the probability density of each latent space sampling vector according to the difference between each latent space sampling vector and the latent target vector or the latent vector target group, wherein the latent space feature includes a plurality of latent space sampling vectors.

[0068] In this embodiment, the probability density P(z i' )for:

[0069]

[0070] Among them, z i' represents the i'th latent space sampling vector, z target represents the potential target vector, and exp(·) represents the exponential function. When there is only one sparse region in the input point cloud of the l-1th level generator, ||z i' -z target || 2 Indicates the calculation of z i' The square of the two-norm difference between the potential target vector of the sparse region and the vector of the potential target vector of the sparse region. When there are more than two sparse regions in the input point cloud of the l-1th level generator, ||z i' -z target || 2 Indicates the calculation of z i' The square of the two-norm difference between the potential target vector group of two or more sparse regions, the difference can be z i' The sum of the differences with all potential target vectors in the potential target vector set, or z i' The maximum or average of the differences from all potential target vectors in the set of potential target vectors.

[0071] Step S223, taking the probability density of each latent space sampling vector as the weight, weighted sum is performed on all latent space sampling vectors in the latent space feature to obtain an optimized latent vector. The optimized latent vector of the l-1th level generator is:

[0072]

[0073] Wherein, N' represents the number of latent space sampling vectors included in the latent space features obtained by the encoding network of the l-1th level generator.

[0074] In this implementation, an online data enhancement mechanism is introduced to dynamically adjust the probability density of each point in the input point cloud according to the latent vector target of the sparse area of ​​the input point cloud of the l-1 level generator, increase the latent space sampling vector weight corresponding to the sparse area, and optimize the target distribution of the l-1 level generator. The optimized probability density (sampling weight) can constrain point cloud generation on a global scale, reduce over-generation of dense areas, and enhance the details of sparse areas.

[0075] In a preferred embodiment, step S2, inputting the point cloud to be processed into a pre-trained point cloud dynamic upsampling model to obtain an upsampled point cloud, includes:

[0076] Step S21: input the point cloud to be processed into a pre-trained point cloud dynamic upsampling model to obtain a first up-sampled point cloud

[0077] Step S22: Process the point cloud to be processed using a preset upsampling algorithm to obtain a second upsampled point cloud. The preset upsampling algorithm is a traditional interpolation method, such as spline interpolation, shortest path interpolation, etc. The interpolation method generates new points by fitting curves or surfaces to the point cloud data to achieve upsampling.

[0078] Step S23: weighting the first up-sampled point cloud and the second up-sampled point cloud using the mixed sampling parameter α to obtain an up-sampled point cloud.

[0079]

[0080] The mixed sampling parameter α may be a hyperparameter, with a value range of 0.5 to 0.7, preferably 0.6. Preferably, the mixed sampling parameter α is a learnable parameter, which is learned in the point cloud dynamic upsampling model training to obtain the optimal value.

[0081] In this embodiment, a hybrid upsampling technology is proposed, which combines the first upsampling point cloud output by the point cloud dynamic upsampling model obtained based on the generative adversarial network and the second upsampling point cloud output by the traditional interpolation method. The first upsampling point cloud output by the point cloud dynamic upsampling model has advantages in overall structure and distribution consistency, while the traditional interpolation method can better optimize the detail performance of the local edge area. By weighted fusion of the two upsampling point clouds, the final fused point cloud has been significantly improved in overall resolution and local geometric details. At the same time, by designing α as a learning parameter, its optimization can be incorporated into the loss function, and the contribution of the generative adversarial network and the traditional interpolation method to the final point cloud can be automatically adjusted, thereby generating a higher quality high-resolution point cloud, avoiding the problem of local sparseness or density caused by a single technology, and optimizing the clarity of the edges of the local point cloud.

[0082] In a preferred embodiment, the encoding network of at least one level generator includes a cascaded PointNet++ network and a sparse convolutional network, which can efficiently extract local and global geometric features of the point cloud.

[0083] In this embodiment, the multi-scale sampling strategy of the PointNet++ network enhances local geometric characteristics and global structural information. Sparse convolutional networks can process non-uniformly distributed point clouds and can significantly reduce the time complexity of feature extraction. The generator introduces a multi-scale feature extraction module (i.e., encoding network) based on PointNet++ and sparse convolution technology to extract local geometric characteristics and global structural information of the point cloud through neighborhood point set grouping and feature aggregation. Through step-by-step feature extraction and aggregation, the generator can retain local details and overall geometric consistency while generating high-resolution point clouds step by step.

[0084] In the generator, the extracted features are not only used to represent the geometry of the local point cloud, but also used to improve the resolution of the point cloud in the subsequent layer-by-layer refinement process. The multi-scale feature extraction of the i-th point can be described as:

[0085]

[0086] Among them, h j represents the characteristics of the jth neighboring point in the neighborhood of the i-th point, represents the neighborhood point set of the i-th point, MLP represents the multi-layer perceptron in the PointNet++ network, and SC(·) represents the sparse convolution operator.

[0087] The present invention also discloses a point cloud dynamic upsampling model training method, which in a preferred implementation manner includes:

[0088] Step A1, construct a training sample point cloud set, each training sample point cloud corresponds to a real point cloud. First, construct a sample point cloud set, and then divide the sample point cloud set into a training set and a validation set. The sample point cloud is preferably, but not limited to, the original point cloud data of the vehicle or robot passage or the surface of the object to be three-dimensionally reconstructed collected by at least one point cloud acquisition device such as laser radar (LiDAR), structured light scanner and depth camera. The original point cloud is a sparse and irregular point cloud. After obtaining the original point cloud data, it also includes preprocessing the original point cloud:

[0089] Normalization process, adjust the original point cloud range to the unit cube [0,1] 3 In order to ensure the consistency of distribution;

[0090] De-noising filtering, preferably but not limited to, removing abnormal points in the point normalized point cloud by an outlier filtering algorithm, thereby obtaining a point cloud to be processed.

[0091] After obtaining multiple training sample point clouds, the training sample point clouds are randomly rotated, scaled, mirrored, and other operations are performed to generate diversified sample point clouds to improve the robustness of the model to diverse data.

[0092] In this embodiment, the real point cloud corresponding to the training sample point cloud can be obtained by high-precision sensor acquisition or computer-aided synthesis.

[0093] Step A2, construct a generative adversarial network and initialize the network parameters of the adversarial network, load the training sample point cloud set and the real point cloud corresponding to each training sample point cloud, the goal is to build a realistic point cloud distribution by dynamically optimizing the generative network G and the discriminative network D.

[0094] like Figure 3As shown, the generative adversarial network includes a generative network G and a discriminative network D; among them, the generative network G includes an L-level generator and a multi-level refinement mechanism module; the multi-level refinement mechanism module uses the L-level generator to perform L-level upsampling processing on the training sample point cloud to obtain a sample generated point cloud, L is a positive integer, and the (l - 1)-th level upsampling processing process includes:

[0095] Step A21, when l = 1, use the encoding network of the (l - 1)-th level generator to map the training sample point cloud into latent space features; when l > 1, use the encoding network of the (l - 1)-th level generator to map the (l - 2)-th level generated point cloud into latent space features;

[0096] Step A22, perform latent space dynamic optimization processing on the latent space features to obtain an optimized latent vector; for the specific processing process, refer to step S22 in the above point cloud dynamic upsampling method based on the generative adversarial network inverse mapping technology, which will not be elaborated here.

[0097] Step A23, use the decoding network of the (l - 1)-th level generator to decode the optimized latent vector to obtain the (l - 1)-th level generated point cloud; 1 ≤ l ≤ L; the encoding network and decoding network of the generator can also refer to the network structure in the point cloud dynamic upsampling method based on the generative adversarial network inverse mapping technology, which will not be elaborated here.

[0098] Step A24, if l = L, then use the (l - 1)-th level generated point cloud as the sample generated point cloud, if l < L, then enter the next level of upsampling processing;

[0099] Step A3, alternately train the discriminative network and the generative network using the training sample point cloud set until the training stop condition is reached. In the alternate training, optimize the network parameters of the discriminative network through the backpropagation algorithm according to the discriminative loss, and optimize the network parameters of the generative network through the backpropagation algorithm according to the generative loss.

[0100] In this embodiment, the discriminative network is used to calculate the discriminative loss and the generative loss. The training stop condition is preferably but not limited to that the number of alternate training times reaches the preset maximum number of alternate training times, or that both the discriminative loss and the generative loss tend to be stable, so that the geometric characteristics and distribution rules of the generated point cloud output by the generative network are highly consistent with the real point cloud.

[0101] In this embodiment, in step A3, alternately training the discriminative network and the generative network using the training sample point cloud set includes:

[0102] Step A31, alternate training process - discriminative network training: input the real point cloud and the generated point cloud output by the generative network to calculate the discriminative loss L dist .

[0103] Step A32, alternating training process - generative network training: freeze the discriminative network parameters, optimize the generative network to make the generated point cloud closer to the real point cloud, calculate and optimize the generated loss L through back propagation G , thereby optimizing the generation network parameters and making the generated point cloud closer to the real point cloud distribution.

[0104] Step A33, joint optimization stage: repeat the above steps A31 and A32 in a loop, and train the discriminant network and the generative network alternately for multiple times, each training is based on the latest state of the other. Comprehensively consider the adversarial loss, geometric consistency loss and distribution uniformity loss, and gradually improve the resolution and quality of the generated point cloud.

[0105] In step A4, the generated network obtained after training is used as a point cloud dynamic upsampling model.

[0106] In a preferred embodiment, the geometric characteristics difference between the sample generated point cloud output by the generation network and the real point cloud corresponding to the sample is effectively learned, such as Figure 3 As shown, the discriminant network includes:

[0107] The local feature extraction module is used to extract the local features of the sample generated point cloud output by the generation network and the local features of the real point cloud corresponding to the training sample point cloud to which the sample generated point cloud belongs. Specifically, the KD-Tree method is used to find the neighborhood points of each point on the sample generated point cloud and the real point cloud, and the sparse convolution is used to extract features from the neighborhood of each point to obtain the local features F of the sample generated point cloud. local,gen and the local feature F of the real point cloud local,real :

[0108]

[0109] The global feature extraction module is used to extract the global features of the sample generated point cloud output by the generation network and the global features of the real point cloud corresponding to the training sample point cloud to which the generated point cloud belongs. Specifically, the local features of each point in the sample generated point cloud and the real point cloud are averaged and pooled to obtain the total features F of the sample generated point cloud. global,gen And all the features of the real point cloud F global,,real :

[0110]

[0111] Where N represents the number of points in the sample generated point cloud / real point cloud, and F local,i Represents the local features of the i-th point in the sample generated point cloud / real point cloud.

[0112] In this embodiment, the discriminant network sets up a dual-channel input mechanism, accepts real point clouds and sample-generated point clouds at the same time, and judges the quality of the sample-generated point clouds by calculating the distribution differences between the real point cloud features (local features and global features) and the sample-generated point cloud features.

[0113] In a preferred embodiment, the discrimination loss L D Based on the distribution consistency loss L dist Build, L D =-L dist ;

[0114] Generate loss L G Including distribution consistency loss L dist , geometric consistency loss L geo and / or evenly distributed row loss L uniform Specifically, the generated loss L G Including distribution consistency loss L dist and geometric consistency loss L geo , or, generate loss L G Including distribution consistency loss L dist and uniformly distributed row loss L uniform , or, generate loss L G Including distribution consistency loss L dist , geometric consistency loss L geo and uniformly distributed row loss L uniform .

[0115] Preferably, L G =αL dist +βL geo +γL uniform ;

[0116] Among them, α, β, and γ are the first weight parameter, the second weight parameter, and the third weight parameter, respectively, which are used to balance the weights of different loss items.

[0117]

[0118] L dist By measuring the distribution difference with Wasserstein distance, the difference in geometric feature distribution between the generated point cloud and the real point cloud can be evaluated.

[0119] L geo Minimize the surface error between the generated point cloud and the real point cloud, and improve the geometric realism of the generated point cloud.

[0120] L uniform The spatial distribution of the point cloud is constrained by regularization terms to avoid local over-crowding or over-sparseness.

[0121] Among them, F local,real represents the local features of the real point cloud corresponding to the training sample point cloud obtained by the local feature extraction module, F global,real represents the global features of the real point cloud corresponding to the training sample point cloud obtained by the global feature extraction module, F local,gen represents the local features of the sample generated point cloud corresponding to the training sample point cloud obtained by the local feature extraction module, F global,gen It represents the global features of the sample generated point cloud corresponding to the training sample point cloud obtained by the global feature extraction module; D(,) represents the difference between the two features; represents the expectation of finding the difference; N represents the number of training sample point clouds participating in the training, and i represents the point index; Represents the curvature value of the i-th point in the real point cloud corresponding to the training sample point cloud, Represents the curvature value of the i-th point in the sample generated point cloud corresponding to the training sample point cloud; represents the neighborhood of the i-th point; p i Represents the position vector of the i-th point in the sample generated point cloud, p j represents the position vector of the jth point in the neighborhood of the i-th point in the sample generated point cloud; d represents the preset uniform distance; ∥·∥ 2 Represents the square of the two-norm of the orientation quantity: ∥·∥ represents the bi-norm of the orientation quantity:

[0122] In the above-mentioned point cloud dynamic upsampling model training method, the customized structure design of the generative adversarial network: In view of the sparsity and irregularity of point cloud data, a customized generative adversarial network structure is designed. The generator combines multi-scale feature extraction (PointNet++ and sparse convolution) with a layer-by-layer refinement mechanism to achieve gradual high-resolution generation of point clouds from local to global. The discriminator adopts a dual-channel mechanism and a distribution consistency module to evaluate the geometric distribution difference between the generated point cloud and the real point cloud through the Wasserstein distance to ensure the generation quality. At the same time, geometric consistency regularization is introduced to make the generated point cloud closer to the real point cloud in terms of local curvature.

[0123] In one embodiment, after each alternating training, a quality assessment and mixed sampling parameter α learning step is also included:

[0124] Step a, quality assessment: Evaluate the point cloud quality through the curvature error and distribution uniformity indicators to ensure that the output high-resolution point cloud meets the actual application requirements. The two indicators are specifically defined as follows:

[0125] Curvature Error Expectation

[0126] Expected uniformity of distribution

[0127] Where N is the total number of points in the sample generated point cloud and the sample real point cloud, and are the curvature values ​​of the i-th point in the sample real point cloud and the sample generated point cloud finally output by the generating network, and p i is the position vector of the i-th point in the sample generated point cloud that is finally output by the generative network, is the set of domain points of the i-th point, p j represents the position vector of the jth neighboring point of the ith point, d is the uniform spacing of the points in an ideal situation, and is a preset constant (set by experiments based on specific task requirements).

[0128] Step b, obtain the mixed sampling parameter loss L α , L α Add generation loss L G , by minimizing the loss function L α , dynamically adjust the value of the mixed sampling parameter α to optimize the effect of mixed upsampling. α =E curvature +E uniformity .

[0129] This implementation has the following innovative features:

[0130] (1) Point cloud upsampling based on inverse mapping technology and generating compact representation through latent space: It is proposed to apply the inverse mapping technology of adversarial networks to point cloud upsampling. In order to address the challenge that sparse point clouds cannot fully express the geometric characteristics of objects, a generation method based on latent space mapping is designed. By encoding low-resolution point clouds and mapping them to a high-dimensional latent space, a compact global feature representation is obtained. The latent space representation can capture the global structural information of the point cloud, provide guidance for the layer-by-layer generation of high-resolution point clouds, and effectively solve the problem of poor global consistency in traditional methods.

[0131] (2) Customized structure design of generative adversarial network: A customized generative adversarial network structure is designed to address the sparsity and irregularity of point cloud data. The generator combines multi-scale feature extraction (PointNet++ and sparse convolution) with a layer-by-layer refinement mechanism to achieve gradual high-resolution generation of point clouds from local to global. The discriminant network adopts a dual-channel mechanism and a distribution consistency module to evaluate the geometric distribution difference between the generated point cloud and the real point cloud through the Wasserstein distance to ensure the generation quality. At the same time, geometric consistency regularization is introduced to make the generated point cloud closer to the real point cloud in terms of local curvature.

[0132] (3) Introducing the online data enhancement mechanism: The online data enhancement mechanism is an important design module in this technical solution to improve the distribution quality of the generated point cloud and the details of sparse areas. Combined with the generative adversarial network (GAN) framework, the parameter z is used to dynamically adjust the latent space sampling weights and the layer-by-layer refinement generation strategy. optimized～l Optimize the target distribution of generators at all levels to solve the problems of uneven distribution and insufficient details in sparse point cloud generation.

[0133] The present invention also discloses another point cloud dynamic upsampling model training method, which in a preferred embodiment includes:

[0134] Step B1: Construct the teacher generation network G T , Student-generated Network G S And the discriminant network D.

[0135] Teacher generation network G T The network structure of the point cloud dynamic upsampling model in the above-mentioned point cloud dynamic upsampling method based on the inverse mapping technology of the generative adversarial network is consistent with that of the previous point cloud dynamic upsampling model training method. T It includes an L-level generator and a multi-level refinement mechanism module; the multi-level refinement mechanism module uses the L-level generator to perform L-level upsampling processing on the training sample point cloud to obtain the teacher generated point cloud. The discriminant network D has the same structure as the discriminant network in the previous point cloud dynamic upsampling model training method.

[0136] Student-Generated Network S The size is 50% of the teacher generation network, that is, the compressed student generation network G S The number of layers and the number of neurons in each layer are the teacher generation network G T Half of the network G is generated from the teacher T The shallow feature parameters are inherited in , while the parameters of the deep network part are determined using the Xavier initialization method.

[0137] Step B1 also includes constructing a training sample point cloud set with reference to step A1 in the previous point cloud dynamic upsampling model training method.

[0138] Step B2: According to the previous point cloud dynamic upsampling model training method, the teacher generates a network G T The teacher network G is generated by alternating training with the discriminant network D until the training stop condition is reached. T As a teacher upsampling model.

[0139] Step B3: Use the training sample point cloud set to train the student to generate the network G S Until the student training stop condition is reached, the student generates the network G SAs a point cloud dynamic upsampling model, in each training, the training sample point cloud is input into the student generation network G S And the teacher upsampling model, according to the student generation network G S The output student generated point cloud and the teacher generated point cloud output by the teacher upsampling model calculate the distillation loss, and the back propagation method is used to update the network parameters of the student generated network according to the distillation loss;

[0140] Step B4: Generate the student network G obtained after training S As a point cloud dynamic upsampling model.

[0141] In this embodiment, in each training of step B3, the low-resolution training sample point cloud Input teacher upsampling model, teacher upsampling model output teacher generated point cloud The same training sample point cloud Input student generated network G S , output student generated point cloud

[0142] Calculate the soft target loss L soft , soft target loss L soft To measure the student generated network G S Output of student generated point cloud Teacher generated point cloud outputted by the teacher upsampling model The distribution difference of

[0143]

[0144] in, Indicates to obtain The variance of Respectively express the expectations.

[0145] Calculate the hard target loss L hard =λ1L geo +λ2L uniform ;

[0146] The distillation loss function is obtained as:

[0147] L distill =λ3L soft +λ4L hard ;

[0148] In this embodiment, λ1, λ2, λ3, and λ4 represent the first hard target weight, the second hard target weight, the first distillation weight, and the second distillation weight, respectively. The initial values ​​of λ3 and λ4 can both be 0.5 and can be adjusted according to experimental results. S The parameters of the student-generated point cloud gradually approach the effect of the teacher upsampling model. After optimization through the distillation process, the final lightweight student-generated network G is obtained. S , that is, the point cloud dynamic upsampling model.

[0149] Figure 4 The overall flow chart of training, acquisition and testing of the point cloud dynamic upsampling model in this embodiment is shown.

[0150] The present invention also discloses an electronic device. In one embodiment, the electronic device includes at least one processor; and a memory connected to the at least one processor in communication; wherein:

[0151] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the point cloud dynamic upsampling method or point cloud dynamic upsampling model training method based on the generative adversarial network inverse mapping technology provided by the present invention.

[0152] like Figure 5 , which is a schematic diagram of the structure of an electronic device of a point cloud dynamic upsampling method or a point cloud dynamic upsampling model training method based on a generative adversarial network inverse mapping technology provided by an embodiment of the present invention. The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a point cloud dynamic upsampling method or a point cloud dynamic upsampling model training method program based on a generative adversarial network inverse mapping technology.

[0153] Among them, the processor 10 can be composed of an integrated circuit in some embodiments, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits packaged with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors and various control chips. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device. It runs or executes programs or modules stored in the memory 11 (for example, executing a point cloud dynamic upsampling method or a point cloud dynamic upsampling model training method based on a generative adversarial network inverse mapping technology, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.

[0154] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of the point cloud dynamic upsampling method or the point cloud dynamic upsampling model training method program based on the generative adversarial network inverse mapping technology, but also can be used to temporarily store data that has been output or is to be output.

[0155] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0156] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0157] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0158] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0159] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0160] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0161] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "an implementation", "a preferred implementation" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0162] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A point cloud dynamic upsampling method based on generative adversarial network inverse mapping technology, characterized in that: Including: Obtain the point cloud to be processed, input the point cloud to be processed into a pre-trained point cloud dynamic upsampling model, and obtain the upsampled point cloud; Among them, the pre-trained point cloud dynamic upsampling model includes an L-level generator and a multi-level refinement mechanism module; the multi-level refinement mechanism module uses the L-level generator to perform L-level upsampling processing on the point cloud to be processed to obtain the upsampled point cloud, L is a positive integer, and the (l - 1)-th level upsampling processing process includes: When l = 1, use the encoding network of the (l - 1)-th level generator to map the point cloud to be processed into latent space features; when l > 1, use the encoding network of the (l - 1)-th level generator to map the (l - 2)-th level generated point cloud into latent space features; Perform latent space dynamic optimization processing on the latent space features to obtain an optimized latent vector; Use the decoding network of the (l - 1)-th level generator to decode the optimized latent vector to obtain the (l - 1)-th level generated point cloud; 1 ≤ l ≤ L; If l = L, then use the (l - 1)-th level generated point cloud as the upsampled point cloud. If l < L, then enter the next level of upsampling processing.

2. The point cloud dynamic upsampling method based on the generative adversarial network inverse mapping technology according to claim 1, characterized in that: During the (l - 1)-th level upsampling processing, the performing latent space dynamic optimization processing on the latent space features to obtain an optimized latent vector includes: Determine the sparse region of the input point cloud for the (l - 1)-th level upsampling processing and the latent vector target of the sparse region; Obtain the probability density of each latent space sampling vector according to the difference between each latent space sampling vector and the latent target vector. The latent space features include multiple latent space sampling vectors; Perform weighted summation on all latent space sampling vectors in the latent space features with the probability density of each latent space sampling vector as the weight to obtain the optimized latent vector.

3. The point cloud dynamic upsampling method based on generative adversarial network inverse mapping technology according to claim 1 or 2, characterized in that: The inputting the point cloud to be processed into a pre-trained point cloud dynamic upsampling model to obtain the upsampled point cloud includes: Input the point cloud to be processed into a pre-trained point cloud dynamic upsampling model to obtain the first upsampled point cloud; Use a preset upsampling algorithm to process the point cloud to be processed to obtain the second upsampled point cloud; Use the hybrid sampling parameter to weight the first upsampled point cloud and the second upsampled point cloud to obtain the upsampled point cloud.

4. The point cloud dynamic upsampling method based on the generative adversarial network inverse mapping technology according to claim 3, characterized in that: The hybrid sampling parameter is learned during the training of the point cloud dynamic upsampling model.

5. The point cloud dynamic upsampling method based on generative adversarial network inverse mapping technology according to claim 1, 2 or 4, characterized in that: The encoding network of at least one level of generator includes a cascaded PointNet++ network and a sparse convolutional network.

6. A point cloud dynamic upsampling model training method, characterized in that: Including: Step A1, construct a training sample point cloud set, and each training sample point cloud corresponds to a real point cloud; Step A2, construct a generative adversarial network and initialize the network parameters of the adversarial network. The generative adversarial network includes a generative network and a discriminative network; among them, the generative network includes an L-level generator and a multi-level refinement mechanism module; the multi-level refinement mechanism module uses the L-level generator to perform L-level upsampling processing on the training sample point cloud to obtain the sample generated point cloud, L is a positive integer, and the (l - 1)-th level upsampling processing process includes: When l = 1, use the encoding network of the (l - 1)-th level generator to map the training sample point cloud into latent space features; when l > 1, use the encoding network of the (l - 1)-th level generator to map the (l - 2)-th level generated point cloud into latent space features; Perform latent space dynamic optimization processing on the latent space features to obtain an optimized latent vector; The decoding network of the (l - 1)-th level generator is used to decode the optimized latent vector to obtain the (l - 1)-th level generated point cloud; 1 ≤ l ≤ L; If l = L, the (l - 1)-th level generated point cloud is used as the sample generated point cloud. If l < L, the next level upsampling process is entered; Step A3, alternately train the discriminative network and the generative network using the training sample point cloud set until the training stop condition is reached. During the alternating training, optimize the network parameters of the discriminative network through the backpropagation algorithm according to the discriminative loss, and optimize the network parameters of the generative network through the backpropagation algorithm according to the generative loss; Step A4, use the generative network obtained after training as the point cloud dynamic upsampling model.

7. A point cloud dynamic upsampling model training method as claimed in claim 6, characterized in that: The discriminative network includes: A local feature extraction module for extracting the local features of the sample generated point cloud and the local features of the real point cloud corresponding to the training sample point cloud to which the sample generated point cloud belongs; A global feature extraction module for extracting the global features of the sample generated point cloud and the global features of the real point cloud corresponding to the training sample point cloud to which the generated point cloud belongs.

8. A point cloud dynamic upsampling model training method as claimed in claim 6 or 7, characterized in that: The discriminative loss is based on the distribution consistency loss L dist Construction, the generation loss includes the distribution consistency loss L dist , geometric consistency loss L geo and / or evenly distributed row loss L uniform ; Among them, F local,real represents the local features of the real point cloud corresponding to the training sample point cloud obtained by the local feature extraction module, F global,real represents the global features of the real point cloud corresponding to the training sample point cloud obtained by the global feature extraction module, F local,gen represents the local features of the sample generated point cloud corresponding to the training sample point cloud obtained by the local feature extraction module, F global,gen It represents the global features of the sample generated point cloud corresponding to the training sample point cloud obtained by the global feature extraction module; D(,) represents the difference between the two features; represents the expectation of finding the difference; N represents the number of training sample point clouds participating in the training, and i represents the point index; Represents the curvature value of the i-th point in the real point cloud corresponding to the training sample point cloud, Represents the curvature value of the i-th point in the sample generated point cloud corresponding to the training sample point cloud; represents the neighborhood of the i-th point; p i Represents the position vector of the i-th point in the sample generated point cloud, p j represents the position vector of the jth point in the neighborhood of the i-th point in the sample generated point cloud; d represents the preset uniform distance; ∥·∥ 2 represents the square of the binary norm of the orientation quantity; ∥·∥ represents the binary norm of the orientation quantity.

9. A point cloud dynamic upsampling model training method, characterized in that: Includes: Step B1, construct a teacher generative network, a student generative network, and a discriminative network; Step B2, alternately train the teacher generative network and the discriminative network according to the point cloud dynamic upsampling model training method described in any one of claims 6 - 7 until the training stop condition is reached, and use the teacher generative network obtained after training as the teacher upsampling model; Step B3, train the student generative network using the training sample point cloud set until the student training stop condition is reached, and then use the student generative network as the point cloud dynamic upsampling model. In each training, input the training sample point cloud into the student generative network and the teacher upsampling model respectively, calculate the distillation loss according to the student generated point cloud output by the student generative network and the teacher generated point cloud output by the teacher upsampling model, and update the network parameters of the student generative network using the backpropagation method according to the distillation loss; Step B4, use the student generative network obtained after training as the point cloud dynamic upsampling model.

10. An electronic device, characterized in that: The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 8.

Images