Method and system for upsampling three-dimensional point cloud based on dynamic scattering transform

Abstract

The application provides a three-dimensional point cloud upsampling method and system based on dynamic scattering transformation, and the method comprises the following steps: obtaining a three-dimensional point cloud to be upsampled, normalizing the spatial coordinates of each point to a unit sphere to obtain a normalized point cloud; inputting the normalized point cloud into a dense connection feature extractor based on dynamic graph scattering transformation to extract multi-scale features; performing feature splicing on the multi-scale features through multi-scale heat kernel graph convolution, performing feature upsampling, and reconstructing the normalized upsampled point cloud coordinates from the upsampled features. The application reconstructs a dense point cloud by taking into account the local and global features of the point cloud, does not require additional information other than the point cloud coordinates, ensures accurate representation of the geometric structure of the object, the spatial coordinate distribution is more uniform, the details of the point cloud are better described, certain improvement is obtained in multiple indicators of the point cloud upsampling, has greater advantages in processing of sparsely collected point clouds, and has practical application value in industrial applications such as virtual modeling and automatic driving.

Description

Technical Field

[0001] This invention relates to the field of multimedia signal processing technology, specifically to a three-dimensional point cloud upsampling method and system based on dynamic scattering transformation, and also provides a corresponding computer terminal and computer-readable storage medium. Background Technology

[0002] In recent years, advancements in sensor and depth camera technology have led to the rise of 3D data. Compared to traditional image signals, 3D signals provide a more three-dimensional depiction of objects and contain more detail and depth information. Since the signals acquired by sensors are generally discrete, point clouds are one of the most commonly used data types for depicting object contours and details among various 3D datasets. Because point cloud upsampling methods based on local geometric information optimization suffer from performance degradation at singularities, researchers have begun to focus on developing machine learning-based point cloud upsampling methods.

[0003] The search revealed:

[0004] In their paper "2018 IEEE / CVF Pu-net: Point cloud upsampling network," Yu et al. first proposed a deep learning-based point cloud upsampling model (PU-Net) and a framework comprising three steps: feature extraction, feature expansion, and coordinate reconstruction. PU-Net obtains multi-scale features of the point cloud by downsampling the input point cloud at different magnifications, employs two independent convolutional layers for feature expansion, and uses a two-layer fully connected network to reconstruct the coordinates of the upsampled point cloud. However, PU-Net's multi-scale feature extraction introduces noise in feature interpolation of low-resolution point clouds, and the convolution-based feature expansion does not consider the spatial information of the point cloud, easily leading to uneven spatial distribution of the upsampled point cloud.

[0005] In subsequent work, Wang et al. introduced MPU at the 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition, which uses densely connected dynamic edge convolutions to extract point cloud features, while artificially introducing differences in feature expansion using 2D mesh stitching. Qian et al. disclosed a geometry-based point cloud upsampling method (PU-Geo) at the European Conference on Computer Vision 2020; however, PU-Geo requires normal vectors at each point in the point cloud as input, while most point cloud datasets do not contain normal vector information. Qian et al. proposed a point cloud upsampling network based on dilated graph convolutional networks (PU-GCN) at the 2021 IEEE / CVF Conference on Computer Vision and Pattern Recognition, extracting multi-scale features of point clouds through the Inception-GCN network structure and dilated graph convolutional networks of different scales. However, similar to MPU's use of densely connected dynamic edge convolutional networks to extract point cloud features, the point cloud feature information in PU-GCN is mainly concentrated in low-pass information within receptive fields of different sizes. For sparse point cloud inputs, the information they contain is limited, and it is difficult to efficiently and comprehensively utilize point cloud information by only considering low-pass features within the receptive field. Summary of the Invention

[0006] To address the aforementioned shortcomings in the prior art, this invention provides a three-dimensional point cloud upsampling method and system based on dynamic scattering transformation, along with a corresponding computer terminal and computer-readable storage medium.

[0007] According to a first aspect of the present invention, a three-dimensional point cloud upsampling method based on dynamic scattering transformation is provided, comprising:

[0008] Obtain the 3D point cloud to be upsampled, normalize the spatial coordinates of each point to a unit sphere, and obtain the normalized point cloud;

[0009] A dense connectivity feature extractor based on dynamic graph scattering transform is provided. The normalized point cloud is input into the dense connectivity feature extractor based on dynamic graph scattering transform to extract multi-scale features.

[0010] The multi-scale features are concatenated by multi-scale heat map convolution, and the features are upsampled to reconstruct the normalized upsampled point cloud coordinates from the upsampled features.

[0011] Optionally, the dense connection feature extractor based on dynamic graph scattering transform mainly consists of K dynamic graph scattering feature extraction units, where K is a positive integer, and:

[0012] The input to the first feature extractor unit is the spatial coordinates of N points in the normalized point cloud. Output d1-dimensional point features in, For the real number field;;

[0013] The input of the k-th feature extraction unit (k = 2, ..., K) is the output of the (k-1)-th feature extraction unit. The features obtained through the k-th feature extraction unit and the input passed through the cross-layer connection Concatenate to obtain output features

[0014] Optionally, inputting the normalized point cloud into the dense connectivity feature extractor based on dynamic graph scattering transform to extract multi-scale features includes:

[0015] For the k-th feature extractor unit, the input feature X is... k-1 The number of feature channels is compressed to a specified dimension using a one-dimensional convolution to obtain a compressed feature matrix. X cnp Each row corresponds to the feature of each point;

[0016] A dynamic K-Nearest Neighbor (KNN) graph is constructed based on the features of each point. For a point with feature x... i node v i Filter the remaining nodes to match x i The K nearest features {x} in Euclidean distance j1 ,x j2 ,…,x jK The corresponding point v j1 ,v j2 ,…,v jK} as v i The neighborhood of and construct an undirected edge (v i ,v jh ), 1≤h≤k, generate a dynamic graph G and its corresponding adjacency matrix A and normalized graph Laplacian matrix L;

[0017] The compressed feature matrix X cmp By employing an M-layer graph scattering network and using a normalized graph Laplacian matrix L to perform spectral domain graph convolution in graph scattering transformation, the features output from different scattering paths in the scattering network are aggregated to obtain a multi-scale feature representation of the point cloud.

[0018] Based on the feature matrix of the input point cloud, construct the dynamic neighborhood of each point in the point cloud based on the feature space. By aggregating the features of each point and the central point in the dynamic neighborhood, use dynamic edge convolution to update the features of the central point and enhance the multi-scale features Obtain the output point cloud feature matrix of the dynamic edge convolution network

[0019] Through cross-layer connection, splice the input feature and the output feature of the dynamic edge convolution to obtain the output feature matrix of the k-th multi-scale feature extractor unit

[0020] Optionally, the compressed feature matrix X cmp Pass through an M-layer graph scattering network, and perform spectral domain graph convolution operation in the graph scattering transformation using the normalized graph Laplacian matrix L. Aggregate the features output by different scattering paths in the scattering network to obtain the multi-scale feature representation of the point cloud Include:

[0021] Select a graph wavelet function ψ and its corresponding scaling function φ, and calculate its Fourier transform and Among them, the graph wavelet function ψ can adopt functions such as diffusion wavelet and Mexican hat wavelet;

[0022] Set the lowest resolution scale Construct wavelet functions of different scales and their corresponding scaling functions Among them, is the integer domain, is the l-th eigenvalue of the graph shift operator S of the graph G constructed by the point cloud through feature KNN;

[0023] Use the normalized graph Laplacian matrix L as the graph shift operator in the spectral domain graph convolution operation of the graph scattering transformation. That is, the convolution operation of the input feature matrix X and the wavelet ψ is defined as:

[0024]

[0025] Among them, is the l-th eigenvalue of the normalized graph Laplacian matrix L;

[0026] Construct an M-layer graph scattering network based on the multi-scale wavelet filter bank and the graph shift operator L. This graph scattering network contains all scattering paths p=(p1,p2,…,p m )(0≤p i ≤J-1), and each scattering path uses a different combination of multi-scale wavelet filters, according to a series of graph wavelet functions in a fixed order Learn the multi-scale feature representation of point clouds and output the multi-scale features S[p][G]X of point clouds under different bandpass filters:

[0027]

[0028] Where, φ J The scaling function is defined at scale J, and the symbol ★ indicates spectral domain graph convolution.

[0029] For a scattering path of length 0 The scattering characteristics output by this path are

[0030] For each scattering path p of length m, the scattering transformation characteristics along that path are output. All scattering paths in the graph scattering network Output feature X p By concatenating the components, we obtain the multi-scale feature representation H of the point cloud based on a dynamically constructed graph scattering network. sct :

[0031] H sct =Concat({X p |p∈Ω M})

[0032] Among them, {X p |p∈Ω M} represents the features output by all scattering paths in the M-layer graph scattering network, and Concat(·) is the feature concatenation operation according to row alignment. Multi-scale features H sct Dimensions

[0033] By performing multi-scale eigenvalue compression on the graph scattering transform and concatenating it with the input features:

[0034]

[0035] We obtain a stable feature representation of the multi-scale point cloud, where Conv1d(·) is a 1D convolutional layer that converts the multi-scale features H... sct The channel is compressed, making

[0036] The concatenated features are further compressed using 1D convolutional layers, making the input features of the dynamic edge convolutional network more efficient. Channel equals d c .

[0037] Optionally, based on the feature matrix of the input point cloud, a dynamic neighborhood of each point in the point cloud is constructed in the feature space. By aggregating the features of each point in the dynamic neighborhood and the center point, dynamic edge convolution is used to update the features of the center point, thereby enhancing the multi-scale features. The point cloud feature matrix is ​​obtained from the output of the dynamic edge convolutional network. include:

[0038] Based on input features Construct a neighborhood in the feature space with each point as the center. For node v i and its corresponding feature x i Filter the remaining nodes to match x i The K nearest features {x} in Euclidean distance j1 ,x j2 ,…,x jK} Corresponding point As v i ;

[0039] Calculate the center point v i Each point v in the neighborhood jh Feature x jh With center point feature x i displacement x jh -x i By vertically concatenating all displacement vectors, a local characterization within the point cloud feature space is obtained.

[0040]

[0041] Enhance local spatial information of point clouds using one-dimensional convolutional layers. center point v i The features are copied k times as a global characterization of the point cloud feature space, and... By horizontally stitching together, the features of the center point are obtained.

[0042]

[0043] Merge the features of each center point Obtain the feature matrix of a point cloud with N points. feature matrix Inputting to a 2D convolutional layer to learn hidden features of a point cloud A point cloud feature matrix containing local and global information of the point cloud feature space is obtained by cross-layer stitching.

[0044]

[0045] Max pooling is performed on the neighborhood features of each center point to update the center node features, resulting in the point cloud feature matrix output by the dynamic edge convolutional network.

[0046] Optionally, the step of performing feature concatenation on the multi-scale features through multi-scale heat map convolution, performing feature upsampling, and reconstructing normalized upsampled point cloud coordinates from the upsampled features includes:

[0047] Point cloud features are expanded by concatenating multi-scale heat map convolutional features, and the multi-scale feature extractor outputs feature X. k Using r heat map convolutional networks of different scales

[0048]

[0049] Generate point cloud feature matrices at different heat kernel scales Where σ is a nonlinear activation function, s i Let L be the i-th scale of the heat map convolution, L be the normalized graph Laplacian matrix, r be the point cloud upsampling rate, and W be the value of the heat map convolution. t These are trainable parameters. The point cloud feature matrix is ​​vertically concatenated to obtain extended features.

[0050]

[0051] Each row of the feature matrix corresponds to a feature of a point in the upsampled point cloud.

[0052] A point cloud upsampling model is provided, comprising a coordinate residual connection network and a two-layer fully connected network; based on the extended feature X, the point cloud upsampling model is used to perform upsampling. out The spatial coordinates of the upsampled point cloud are reconstructed from the input point cloud's 3D coordinates X0:

[0053] Z=FC(σ(FC(X out )))+Duplicate(X0,r)

[0054] Obtain the normalized upsampled point cloud 3D coordinates output by the point cloud upsampled depth network. The number of points in the sampled point cloud is r times that of the original point cloud. Here, FC(·) represents a fully connected network, and Duplicate(X0,r) represents a three-dimensional coordinate matrix obtained by vertically stitching together the input point cloud coordinates X0 copied r times.

[0055] Optionally, the parameters of the point cloud upsampling model are obtained by training a loss function based on a weighted sum of the Chamfer Distance and the regularization term of the parameters, including:

[0056] The upsampled point cloud is obtained using the point cloud upsampling model. Then, the difference between the normalized upsampled point cloud spatial 3D coordinates and the normalized true value Q is calculated using the chamfer distance:

[0057]

[0058] Where CD(·) is the chamfer distance, dist(a,b) is the Euclidean distance between points a and b, and |Q| and Point cloud Q and The number of points in the point cloud. The training of the point cloud upsampling model uses a weighted sum of chamfer distance and parameter regularization term as the loss function Loss:

[0059]

[0060] Where α and β are hyperparameters, and Θ is a learnable parameter in the point cloud upsampling depth network;

[0061] The gradient of parameter Θ is calculated based on the loss function Loss, and the parameter Θ in the point cloud upsampling model is updated by backpropagation. The parameters of the point cloud upsampling model are obtained by iteratively optimizing and minimizing the above loss function.

[0062] A second aspect of the present invention provides a three-dimensional point cloud upsampling system based on dynamic scattering transformation, comprising:

[0063] Preprocessing module: acquires the 3D point cloud to be upsampled, normalizes the spatial coordinates of each point to a unit sphere, and obtains a normalized point cloud;

[0064] Feature extraction module: Provides a dense connection feature extractor based on dynamic graph scattering transform, inputs the normalized point cloud into the dense connection feature extractor based on dynamic graph scattering transform, and extracts multi-scale features;

[0065] Point cloud reconstruction module: The multi-scale features are spliced ​​by multi-scale heat map convolution, and the features are upsampled to reconstruct the normalized upsampled point cloud coordinates from the upsampled features.

[0066] According to a third aspect of the present invention, a computer terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to perform the method described in any one of the above-described inventions, or to run the system described in the above-described inventions.

[0067] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that, when executed by a processor, the program can be used to perform the method described in any one of the above-described inventions, or to run the system described in the above-described inventions.

[0068] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with existing point cloud upsampling techniques:

[0069] The present invention provides a three-dimensional point cloud upsampling method and system based on dynamic scattering transformation. It uses a feature space KNN dynamic graphing method to perform multi-scale feature extraction based on graph scattering transformation and edge convolution, so that node features can adaptively flow during network training. This method is more adaptable to complex tasks such as point cloud upsampling than the coordinate space KNN static graphing method.

[0070] The present invention provides a three-dimensional point cloud upsampling method and system based on dynamic scattering transformation. It uses graph scattering transformation to extract multi-scale bandpass information of point clouds, and at the same time uses dynamic edge convolutional network to enhance the multi-scale bandpass information output by graph scattering network. Compared with traditional graph convolutional network, its output features are more diversified, and it can take into account the low-frequency and mid-to-high-frequency information in the input point cloud, avoiding the feature oversmoothing problem that may occur in multi-layer graph convolutional network.

[0071] The three-dimensional point cloud upsampling method and system based on dynamic scattering transformation provided by this invention uses multiple densely connected feature extractor units as feature extractors, which can enhance the flow of features between different feature extractor units, increase the diversity of hidden layer features, and enhance the expressive power of feature extractors.

[0072] The present invention provides a three-dimensional point cloud upsampling method and system based on dynamic scattering transformation. It uses a multi-scale heat map convolutional network to expand the point cloud features. By using heat map convolution at different scales, it takes into account both the local and global features of the point cloud, making the spatial coordinate distribution of the dense point cloud after coordinate reconstruction more uniform and better depicting the details of the point cloud.

[0073] The three-dimensional point cloud upsampling method and system based on dynamic scattering transformation provided by this invention were tested on real three-dimensional objects with different geometric shapes, and achieved consistent performance improvement compared with existing methods (see embodiments). Attached Figure Description

[0074] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0075] Figure 1 This is a flowchart illustrating the workflow of a three-dimensional point cloud upsampling method based on dynamic scattering transformation in one embodiment of the present invention.

[0076] Figure 2 This is a schematic diagram of the operation of the three-dimensional point cloud upsampling method in a preferred embodiment of the present invention.

[0077] Figure 3 This is a schematic diagram of the operation of the dense connection feature extractor in a preferred embodiment of the present invention.

[0078] Figure 4 This is a schematic diagram of the operation of the graph scattering network in a preferred embodiment of the present invention.

[0079] Figure 5 This is a schematic diagram of the components of a three-dimensional point cloud upsampling system based on dynamic scattering transformation in one embodiment of the present invention. Detailed Implementation

[0080] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0081] One embodiment of the present invention provides a three-dimensional point cloud upsampling method based on dynamic scattering transformation. This method can perform higher quality representation learning and detail characterization on sparse input non-uniform point clouds, and generate denser point clouds with more uniform spatial distribution.

[0082] like Figure 1 As shown, the three-dimensional point cloud upsampling method based on dynamic scattering transformation provided in this embodiment may include:

[0083] S100: Obtain the 3D point cloud to be upsampled, normalize the spatial coordinates of each point to a unit sphere, and obtain the normalized point cloud.

[0084] S200 provides a densely connected feature extractor based on dynamic graph scattering transform. Normalized point clouds are input into the densely connected feature extractor based on dynamic graph scattering transform to extract multi-scale features.

[0085] S300 performs feature stitching on multi-scale features through multi-scale heat map convolution, upsamples the features, and reconstructs the normalized upsampled point cloud coordinates from the upsampled features.

[0086] In a preferred embodiment of S200, the densely connected feature extractor based on dynamic graph scattering transform mainly consists of K dynamic graph scattering feature extraction units, where K is a positive integer, and:

[0087] The input to the first feature extractor unit is the spatial coordinates of N points in the normalized point cloud. Output d1-dimensional point features in, For the real number field;;

[0088] The input of the k-th feature extraction unit (k = 2, ..., K) is the output of the (k-1)-th feature extraction unit. The features obtained through the k-th feature extraction unit and the input passed through the cross-layer connection Concatenate to obtain output features

[0089] Furthermore, in a preferred embodiment of S200, the normalized point cloud is input into a densely connected feature extractor based on dynamic graph scattering transform to extract multi-scale features, including:

[0090] S201, for the k-th feature extractor unit, the input feature X is... k-1 The number of feature channels is compressed to a specified dimension using a one-dimensional convolution to obtain a compressed feature matrix. X cmp Each row corresponds to the feature of each point;

[0091] S202, construct a graph using the dynamic K-Nearest Neighbor (KNN) algorithm based on the features of each point, for a point with feature x... i node v i Filter the remaining nodes to match x i The K nearest features {x} in Euclidean distance j1 ,x j2 ,…,x jK The corresponding point {v} j1 ,v j2 ,…,v jK} as v i The neighborhood of and construct an undirected edge (v i ,v jh ), 1≤h≤k, generate a dynamic graph G and its corresponding adjacency matrix A and normalized graph Laplacian matrix L;

[0092] S203, compress the feature matrix X cmp By employing an M-layer graph scattering network and using a normalized graph Laplacian matrix L to perform spectral domain graph convolution in graph scattering transformation, the features output from different scattering paths in the scattering network are aggregated to obtain a multi-scale feature representation of the point cloud.

[0093] S204: Based on the feature matrix of the input point cloud, a dynamic neighborhood of each point in the point cloud is constructed in the feature space. By aggregating the features of each point in the dynamic neighborhood and the center point, dynamic edge convolution is used to update the center point features, thereby enhancing multi-scale features. The point cloud feature matrix is ​​obtained from the output of the dynamic edge convolutional network.

[0094] S205 concatenates the input features with the output features of the dynamic side convolution through cross-layer connections to obtain the output feature matrix of the k-th multi-scale feature extractor unit.

[0095] In a preferred embodiment of S203, the compressed feature matrix X is...cmp Through an M-layer graph scattering network, the spectral domain graph convolution operation in the graph scattering transformation is performed using the normalized graph Laplacian matrix L, and the features output by different scattering paths in the scattering network are aggregated to obtain the multi-scale feature representation of the point cloud. It includes:

[0096] S2031, select a graph wavelet function ψ and its corresponding scaling function φ, and calculate their Fourier transforms and Among them, the graph wavelet function ψ can adopt functions such as diffusion wavelets and Mexican hat wavelets;

[0097] S2032, set the lowest resolution scale Construct wavelet functions of different scales and their corresponding scaling functions Among them, is the integer domain, is the l-th eigenvalue of the graph shift operator S of the graph G constructed by the point cloud through feature KNN;

[0098] S2033, use the normalized graph Laplacian matrix L as the graph shift operator for the spectral domain graph convolution operation in the graph scattering transformation, that is, the convolution operation of the input feature matrix X and the wavelet ψ is defined as:

[0099]

[0100] Among them, is the l-th eigenvalue of the normalized graph Laplacian matrix L;

[0101] S2034, construct an M-layer graph scattering network based on the multi-scale wavelet filter bank and the graph shift operator L. This graph scattering network contains all scattering paths p = (p1, p2,..., p m )(0 ≤ p i ≤ J - 1) with a length of m < M. Each scattering path adopts a different combination of multi-scale wavelet filters, and learns the multi-scale feature representation of the point cloud according to a series of graph wavelet functions in a fixed order to output the multi-scale feature S[p][G]X of the point cloud under different band-pass filters:

[0102]

[0103] Among them, φ J is the scaling function at the J scale, and the symbol ★ represents the spectral domain graph convolution;

[0104] S2035, for a scattering path with a length of 0 the scattering feature output by this path is

[0105] S2036, for each scattering path p of length m, the scattering transformation characteristics along that path are output. All scattering paths in the graph scattering network Output feature X p By concatenating the components, we obtain the multi-scale feature representation H of the point cloud based on a dynamically constructed graph scattering network. sct :

[0106] H sct =Concat({X p |p∈Ω M})

[0107] Among them, {X p |p∈Ω M} represents the features output by all scattering paths in the M-layer graph scattering network, and Concat(·) is the feature concatenation operation according to row alignment. Multi-scale features H sct Dimensions

[0108] S2037, through multi-scale eigenvalue compression of graph scattering transform and concatenation with input features:

[0109]

[0110] We obtain a stable feature representation of the multi-scale point cloud, where Conv1d(·) is a 1D convolutional layer that converts the multi-scale features H... sct The channel is compressed, making

[0111] S2038 further compresses the concatenated features through 1D convolutional layers, making the input features of the dynamic edge convolutional network more efficient. Channel equals d c .

[0112] In a preferred embodiment of S204, based on the feature matrix of the input point cloud, a dynamic neighborhood of each point in the point cloud is constructed based on the feature space. By aggregating the features of each point in the dynamic neighborhood and the center point, dynamic edge convolution is used to update the features of the center point, thereby enhancing multi-scale features. The point cloud feature matrix is ​​obtained from the output of the dynamic edge convolutional network. include:

[0113] S2041, based on input features Construct a neighborhood in the feature space with each point as the center. For node v i and its corresponding feature x i Filter the remaining nodes to match x iThe K nearest features {x} in Euclidean distance j1 ,x j2 ,…,x jK} Corresponding point As v i ;

[0114] S2042, Calculate the center point v i Each point v in the neighborhood jh Feature x jh With center point feature x i displacement x jh -x i By vertically concatenating all displacement vectors, a local characterization within the point cloud feature space is obtained.

[0115]

[0116] S2043 enhances the local spatial information of point clouds through one-dimensional convolutional layers. center point v i The features are copied k times as a global characterization of the point cloud feature space, and... By horizontally stitching together, the features of the center point are obtained.

[0117]

[0118] S2044, merge the features of each center point Obtain the feature matrix of a point cloud with N points. feature matrix Inputting to a 2D convolutional layer to learn hidden features of a point cloud A point cloud feature matrix containing local and global information of the point cloud feature space is obtained by cross-layer stitching.

[0119]

[0120] S2045, max pooling is performed on the neighborhood features of each center point to update the center node features, resulting in the point cloud feature matrix output by the dynamic edge convolutional network.

[0121] In a preferred embodiment of S300, multi-scale features are concatenated through multi-scale heat map convolution, and feature upsampling is performed. Normalized upsampled point cloud coordinates are then reconstructed from the upsampled features, including:

[0122] S301 expands point cloud features using multi-scale heat map convolutional feature concatenation, and outputs feature X from the multi-scale feature extractor. k Using r heat map convolutional networks of different scales

[0123]

[0124] Generate point cloud feature matrices at different heat kernel scales Where σ is a nonlinear activation function, s i Let L be the i-th scale of the heat map convolution, L be the normalized graph Laplacian matrix, r be the point cloud upsampling rate, and W be the value of the heat map convolution. t These are trainable parameters.

[0125] S302, vertically concatenate the point cloud feature matrix to obtain extended features.

[0126]

[0127] Each row of the feature matrix corresponds to a feature of a point in the upsampled point cloud.

[0128] S303 provides a point cloud upsampling model, which employs a coordinate residual connection network and a two-layer fully connected network. Based on the extended feature X, the point cloud upsampling model is used to... out The spatial coordinates of the upsampled point cloud are reconstructed from the input point cloud's 3D coordinates X0:

[0129] Z=FC(σ(FC(X out )))+Duplicate(X0,r)

[0130] Obtain the normalized upsampled point cloud 3D coordinates output by the point cloud upsampled depth network. The number of points in the sampled point cloud is r times that of the original point cloud. Here, FC(·) represents a fully connected network, and Duplicate(X0,r) represents a three-dimensional coordinate matrix obtained by vertically stitching together the input point cloud coordinates X0 copied r times.

[0131] In a preferred embodiment of S303, the parameters of the point cloud upsampling model are obtained by training a loss function based on a weighted sum of the Chamfer Distance and the regularization term of the parameters, including:

[0132] S3031, using a point cloud upsampling model to obtain the output upsampled point cloud. Then, the difference between the normalized upsampled point cloud spatial 3D coordinates and the normalized true value Q is calculated using the chamfer distance:

[0133]

[0134] Where CD(·) is the chamfer distance, dist(a,b) is the Euclidean distance between points a and b, and |Q| and Point cloud Q and The number of points in the point cloud. The training of the point cloud upsampling model uses a weighted sum of the chamfer distance and the parameter regularization term as the loss function.

[0135]

[0136] Where α and β are hyperparameters, and Θ is a learnable parameter in the point cloud upsampling depth network;

[0137] S3032, calculate the gradient of parameter Θ based on the loss function Loss, update parameter Θ in the point cloud upsampling model by backpropagation, and obtain the parameters of the point cloud upsampling model by iteratively optimizing and minimizing the above loss function.

[0138] The present invention will now be described in detail with reference to a preferred embodiment. These preferred embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way.

[0139] like Figure 2 As shown, the preferred embodiment of the three-dimensional point cloud upsampling method based on dynamic scattering transformation includes the following steps:

[0140] S1: Input the original point cloud Normalization is performed: the center point of the point cloud is calculated, the center point of the point cloud is translated to the far point, and the maximum radius of the point cloud is normalized to obtain a normalized point cloud dataset.

[0141] Computational point cloud The center position C of the point cloud Translate the point cloud so that its center point falls on the origin. Calculate the maximum radius of the point cloud after the translation. The `norm2(·)` function calculates the 2-norm of the horizontal vectors of the matrix. It then calculates the normalized point cloud spatial coordinate matrix based on the center position C and the maximum radius R.

[0142] S2, based on the feature space of each node in the point cloud, performs KNN dynamic mapping, and extracts multi-scale features of the point cloud by a series of densely connected feature extractor units (i.e., densely connected feature extractors).

[0143] Based on the input normalized point cloud spatial coordinates Multi-scale hierarchical features of point clouds are extracted using densely connected feature extractor units, where the input of the k-th feature extractor unit is the output X of the previous feature extractor unit. k-1Each feature extractor unit contains a graph scattering network with dynamic KNN mapping of the feature space and a dynamic edge convolutional network. The input of the first feature extractor unit is the normalized 3D spatial coordinates X0 of the point cloud. Each feature extractor unit uses an independent feature KNN mapping strategy based on the input feature matrix, extracts multi-scale bandpass features of the point cloud using a graph scattering network, and then enhances the multi-scale bandpass features through a dynamic edge convolutional network, fusing the low-pass features of the point cloud. k cascaded feature extractor units are used as multi-scale feature extractors. Due to the cross-layer connections within the feature extractor units, the multi-scale point cloud feature matrices output by the dynamic edge convolutions in the first k-1 feature extractors are all included in the input of the kth feature extractor unit. The 3D spatial coordinates of the point cloud are used as the initial feature X0 of the point cloud input to the feature extractor. The number of channels of the output feature increases by d with each multi-scale feature extractor unit. c +2d hd For a multi-scale point cloud feature extractor containing k of the above feature extractor units, its output point cloud feature matrix is: like Figure 3 As shown, each point cloud feature extractor unit includes the following steps:

[0144] S21, based on the features of each node in the point cloud, perform KNN mapping in the feature space (when the input features are point cloud spatial coordinates, this mapping method is equivalent to geometric space KNN mapping). The input feature matrix X... k Compress to a specified dimension Based on the compressed feature matrix X cmp A KNN dynamic graph is constructed in the feature space to generate a graph G and its corresponding adjacency matrix A and normalized graph Laplacian matrix L. For each point v in the point cloud... i and its corresponding feature x i Among the remaining points, select the points that are at a distance v in the feature space. i Feature x i The most recent k features {x j1 ,x j2 ,…,x jK The corresponding point {v} j1 ,v j2 ,…,v jK} as point v i neighborhood Based on each point v in the point cloud i neighborhood The edge indicator matrix E of the constructed graph,

[0145]

[0146] Meanwhile, a heat kernel function is used to define the edge weight matrix W.

[0147] W(i,j)=exp(-dist(x i ,x j ) 2 / d i 2 )

[0148] in, The weighted adjacency matrix A = maximum(E·W, (E·W)) is calculated using the edge indicator matrix and the edge weight matrix. T Let A be a symmetric matrix suitable for graph convolution in the spectral domain. `maximum(·,·)` represents the symmetrical maximum of the matrix. Calculate the normalized graph Laplacian matrix L = ID of graph G. -1 / 2 AD -1 / 2 D is the degree matrix of graph G based on the weighted adjacency matrix A.

[0149] S22, based on feature space KNN construction, extracts multi-scale bandpass features of the point cloud by passing the input features through a graph scattering network. Based on the normalized graph Laplacian matrix L, it compresses the feature matrix X. cmp Graph scattering networks are used to compute and learn multi-scale features of a point cloud in the current feature space. For example... Figure 4 As shown, the graph scattering network mainly includes the following operations:

[0150] S221, based on dynamic KNN graph construction in the feature space, performs graph scattering transformation on the input features. Based on the normalized graph Laplacian matrix L of graph G, graph scattering transformation is used to learn the input compressed feature matrix X. cmp Multi-scale bandpass features. Graph scattering transform features are aggregated from the output features of multiple graph scattering paths, each with a length of m, p = {p1, p2, ..., p...}. m} is a series of multi-scale graphical wavelets Define the graph scattering transform on path p as follows:

[0151]

[0152] Where ★ denotes spectral domain graph convolution, and the normalized graph Laplacian matrix L is used as the graph shift operator. For an M-layer graph scattering network, the graph scattering transformation features output by all paths p with lengths less than or equal to M are aggregated to obtain the input matrix X. k-1 Multi-scale feature representation in feature space.

[0153] S222 concatenates the input features of each path in the scattering transform to obtain a multi-scale bandpass feature representation of the input signal. Laterally concatenates the graph scattering transform features output from each scattering path with a length less than or equal to M-1. When the multi-scale wavelet has J different scales, the M-layer graph scattering network has a total of... A scattering path. Output characteristics by laterally stitching together different scattering paths. Output dimension is Multiscale graph scattering transformation feature H sct .

[0154] S223 compresses multi-scale features to reduce feature variance and increase the convergence speed of the point cloud upsampling model. Multi-scale endemic compression of the graph scattering transform is performed, concatenated with the input features, and channel compression is applied through a 1D convolutional layer.

[0155]

[0156] To obtain stable feature representations of multi-scale point clouds The first 1D convolutional layer incorporates multi-scale features H sct The channel is compressed, making By further compressing the concatenated features through 1D convolutional layers, the input features of the dynamic edge convolutional network are made more efficient. Channel equals d c .

[0157] S23 concatenates the multi-scale bandpass features obtained from graph scattering transform with the original features and then compresses them, fusing the original features and multi-scale bandpass features. The multi-scale features output from the dynamically constructed graph scattering network are then fed into a dynamic edge convolutional network for point cloud feature enhancement. Based on the input features... Construct a neighborhood in the feature space with each point as the center. For node v i and its corresponding feature x i Filter the remaining nodes to match x i The K nearest features {x} in Euclidean distance j1 ,x j2 ,…,x jK} Corresponding point As v i The neighborhood of v. Calculate the center point v. i Each point v in the neighborhood jh Feature x jh With center point feature x i displacement x jh -x i By vertically concatenating all displacement vectors, a local characterization within the point cloud feature space is obtained.

[0158]

[0159] Enhance local information in the local space of point clouds using 1D convolutional layers. center point v i The features are copied k times as a global characterization of the point cloud feature space, and... Horizontal splicing

[0160]

[0161] Merge the features of each center point Obtain the feature matrix of a point cloud with N points. feature matrix Inputting to a 2D convolutional layer to learn hidden features of a point cloud A point cloud feature matrix containing local and global information of the point cloud feature space is obtained by cross-layer stitching.

[0162]

[0163] S24 enhances feature compression through dynamic edge convolution, allowing node features to flow within the feature space, strengthening the connections between features and enhancing the expressive power of the feature extraction unit. Max pooling is performed on the neighborhood features of each center point to retain the main information of the center node, resulting in the point cloud feature matrix output by the dynamic edge convolution network. The input of the multi-scale feature extractor unit is connected through cross-layer connections. The feature matrix X′ output by the dynamic side convolutional network is concatenated laterally to obtain the output X of the k-th multi-scale feature extractor unit. k

[0164] X k =Concat(X) k-1 ,X′)

[0165] Cross-layer concatenation ensures that the input of each feature extractor unit takes into account the output features of all preceding feature extractor units, thereby achieving a dense connection effect and enhancing the fluidity of features.

[0166] S3 uses heat kernel map convolution to expand the multi-scale features of the point cloud, generating features equivalent to the number of nodes in the target point cloud.

[0167] The point cloud feature matrix X extracted from multi-scale features k Feature extension is performed based on a multi-scale heat map convolutional network. The maximum scale s is specified. max and minimum scale s min For point cloud upsampling at a magnification of r, with r different heat core scales s1, s2, ..., s r In s max and s min They are uniformly distributed on a logarithmic scale. For each thermonuclear scale s i The output features of the heat core diagram convolutional network

[0168]

[0169] Among them W t These are trainable parameters. The point cloud feature matrix is ​​vertically concatenated to obtain the extended feature X. out ,

[0170]

[0171] Each row of the feature matrix corresponds to a feature of a point in the upsampled point cloud.

[0172] S4 reconstructs the spatial coordinates of the upsampled point cloud based on the extended features generated by the heat map convolution.

[0173] A multi-layer fully connected network is used to reconstruct the 3D coordinates of the point cloud. This is achieved from the extended feature X... out Reconstructing the spatial coordinate residuals of upsampled dense point clouds

[0174] X res =FC(σ(FC(X) out )))

[0175] Copy the input point cloud coordinates r times and stitch them together vertically to obtain the sparse framework of the upsampled point cloud.

[0176]

[0177] The point cloud spatial coordinate residual X res With X dup Adding them together yields the spatial coordinates of the upsampled normalized point cloud. The final upsampled point cloud coordinates are obtained by scaling and translating the output normalized point cloud using the origin point cloud center C and the maximum radius R.

[0178] exist Figure 2 Building upon the previous method, this further includes parameter optimization for the 3D point cloud upsampling method. The 3D point cloud upsampling method obtains learnable parameters based on end-to-end training using training data, including:

[0179] Point cloud upsampling networks compute normalized upsampled point clouds. The chamfer distance between the output upsampled point cloud and the normalized true value Q measures the difference between the output and the true value.

[0180]

[0181] Where dist(a,b) is the Euclidean distance between points a and b, |Q| and Point cloud Q and The number of points in the model. The model training uses a weighted sum of the chamfer distance and the parameter regularization term as the loss function.

[0182]

[0183] Here, α and β are hyperparameters, and Θ is a learnable parameter in the point cloud upsampling depth network. The gradient of parameter Θ is calculated based on the loss function, and the parameter Θ in the point cloud upsampling depth network is updated by backpropagation. By optimizing the above loss function, the purpose of training the parameters in the point cloud upsampling depth model is achieved.

[0184] To better understand, the technical solution provided by the above embodiments of the present invention will be further described in detail below with reference to a specific application example.

[0185] In this specific application example, the 3D point cloud upsampling method based on dynamic graph scattering networks and dynamic edge convolutional networks mainly includes the following 5 steps:

[0186] Step 1: Input the original point cloud Normalization is performed. Point cloud calculation is then performed. The center position C of the point cloud Translate the point cloud so that its center point falls on the origin. Calculate the maximum radius of the point cloud after the translation. That is, the maximum L2 norm of the horizontal vector of the point cloud coordinate matrix after translation. The normalized point cloud spatial coordinate matrix is ​​calculated based on the center position C and the maximum radius R.

[0187] Step 2: Based on the input normalized point cloud coordinate matrix X0, extract point cloud features using the densely connected feature extractor units described above. The input to the first feature extractor unit is the normalized point cloud 3D spatial coordinates X0. Each feature extractor unit employs an independent feature KNN graph construction strategy based on the input feature matrix, uses a graph scattering network to extract multi-scale bandpass features of the point cloud, and then enhances the multi-scale bandpass features through a dynamic edge convolutional network, fusing the point cloud low-pass features. Using k cascaded feature extractor units as the multi-scale feature extractor, the 3D spatial coordinates of the point cloud are used as the initial feature X0 of the point cloud as input to the feature extractor, and the output is the point cloud feature matrix.

[0188] Step 3: Extract the point cloud feature matrix X based on multi-scale features. k Feature extension is performed based on a multi-scale heat map convolutional network. The maximum scale s is specified. max and minimum scale s min For point cloud upsampling at a magnification of r, with r different heat core scales s1, s2, ..., s r In s maxand s min They are uniformly distributed on a logarithmic scale. For each thermonuclear scale s i The output features of the heat core diagram convolutional network Extended feature X is obtained by horizontally concatenating feature matrices of different scales. out .

[0189] Step 4: Reconstruct the 3D coordinates of the point cloud using a multi-layer fully connected network. From the extended feature X... out Reconstructing the spatial coordinate residuals of upsampled dense point clouds The input point cloud coordinates are copied r times and vertically stitched together. The result is then added to the spatial coordinate residuals to obtain the normalized upsampled point cloud spatial coordinates.

[0190] Step 5: Calculate the normalized upsampled point cloud The chamfer distance between the output upsampled point cloud and the normalized true value Q measures the difference between the output upsampled point cloud and the true value. Simultaneously calculate the regularization term of the trainable parameters in the deep network and calculate the loss function.

[0191]

[0192] Step 6: Calculate the gradient of parameter Θ based on the loss function, and update parameter Θ in the point cloud upsampling depth model using the Adam optimizer.

[0193] Repeat steps two through six until the network converges. Update the learned parameters Θ for the upsampling task of the 3D point cloud.

[0194] Step 7: Scale and translate the output normalized point cloud using the original point cloud center C and the maximum radius R to obtain the final upsampled point cloud coordinates.

[0195] In this specific application example, the comparative evaluation methods for implementation effectiveness are PU-GAN, PU-Geo, MPU, and PU-GCN. All compared network models were ported from open-source code to the TensorFlow architecture and point cloud upsampling experiments were conducted on the PU1K dataset with default parameters. The training data consisted of 60,000 samples, with point clouds of 256 / 512 / 1024 / 2048 points input at different resolutions, and corresponding ground truth values ​​of 1024 / 2048 / 4096 / 8192 points for dense point clouds. The test data contained 127 samples, each also containing multi-resolution inputs of 256 / 512 / 1024 / 2048 points and corresponding ground truth values. The dataset also provided a grid of test samples for calculating P2F (point-to-surface) distances. Table 1 provides the results of 4x and 16x upsampling experiments with 256-point sparse input, using CD: Chamfer distance, HD: Hausdorff distance, EMD: ground motion distance, and P2F: point-to-surface distance.

[0196]

[0197]

[0198] It should be noted that PU-Geo performs poorly because the PU1K dataset does not contain point cloud normal vector information.

[0199] The 3D point cloud upsampling method provided in this invention, compared to methods like MPU, PU-GAN, and PU-GCN which use graph networks or graph convolutional networks to extract point cloud features, primarily focuses on low-pass graphs, making it difficult to efficiently utilize the effective information in sparse input point clouds. In contrast, the former introduces graph scattering networks to extract multi-scale bandpass features of the point cloud, ensuring feature diversity and enhancing the model's expressive power. Furthermore, this invention employs a dynamic feature space KNN graph construction method, enabling the network to adaptively construct graphs during training. This is more conducive to uncovering hidden structural information in the point cloud than static coordinate space KNN graph construction, allowing feature flow to be unrestricted by geometric space and better suited for advanced tasks like point cloud upsampling. Compared to PU-Geo, this invention's point cloud upsampling method only requires the normalized spatial coordinates of the sparse point cloud as input, without needing additional auxiliary data. Compared to existing methods, this invention shows performance improvements in all metric aspects for sparse point cloud inputs, with a more significant advantage in high-rate upsampling tasks with sparse inputs.

[0200] 3D point clouds are widely used in virtual modeling, autonomous driving, and other fields. However, due to limitations in hardware and communication costs, initial point clouds are generally highly sparse and unevenly distributed. The 3D point cloud upsampling method based on dynamic scattering transformation provided in the above embodiments of the present invention can generate denser point clouds with enhanced details and more uniform spatial distribution from sparse and non-uniformly distributed point clouds, which is of great significance for subsequent mesh modeling and 3D rendering. Therefore, the 3D point cloud upsampling method based on dynamic scattering transformation provided in the above embodiments of the present invention has significant implications for industrial point cloud data processing.

[0201] One embodiment of the present invention provides a three-dimensional point cloud upsampling system based on dynamic scattering transformation.

[0202] like Figure 5 As shown, the 3D point cloud upsampling system based on dynamic scattering transformation provided in this embodiment may include:

[0203] Preprocessing module: acquires the 3D point cloud to be upsampled, normalizes the spatial coordinates of each point to a unit sphere, and obtains a normalized point cloud;

[0204] Feature extraction module: Provides a densely connected feature extractor based on dynamic graph scattering transform. Input the normalized point cloud into the densely connected feature extractor based on dynamic graph scattering transform to extract multi-scale features;

[0205] Point cloud reconstruction module: Multi-scale features are stitched together by multi-scale heat map convolution, and features are upsampled to reconstruct normalized upsampled point cloud coordinates from the upsampled features.

[0206] It should be noted that the steps in the method provided by the present invention can be implemented using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system. That is, the embodiments in the method can be understood as preferred examples for building the system, and will not be elaborated here.

[0207] One embodiment of the present invention provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to perform any of the methods in the above embodiments of the present invention, or to run any of the systems in the above embodiments of the present invention.

[0208] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0209] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0210] A processor is used to execute computer programs stored in memory to implement the various steps of the methods or various modules of the systems involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method and system embodiments.

[0211] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0212] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform the method of any of the above embodiments of the present invention, or to run the system of any of the above embodiments of the present invention.

[0213] The 3D point cloud upsampling method and system based on dynamic scattering transform provided in the above embodiments of the present invention first obtains the 3D point cloud to be upsampled, normalizes the spatial coordinates of each point to a unit sphere, and obtains a normalized point cloud; then, the normalized point cloud is input into a densely connected feature extractor based on dynamic graph scattering transform to extract multi-scale features; finally, the multi-scale features are concatenated through multi-scale heat map convolution to perform feature upsampling, and the normalized upsampled point cloud coordinates are reconstructed from the upsampled features. Compared with existing methods, the present invention takes into account both local and global features of the point cloud to reconstruct dense point clouds, without requiring additional information other than point cloud coordinates, while ensuring accurate representation of the geometric structure of objects, more uniform spatial coordinate distribution, and better characterization of point cloud details. It achieves certain improvements in multiple indicators of point cloud upsampling, has greater advantages in processing sparsely acquired point clouds, and has practical application value in industrial applications such as virtual modeling and autonomous driving.

[0214] Those skilled in the art will understand that, in addition to implementing the system and its various devices provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices of this invention function as logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices provided by this invention can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0215] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0216] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A three-dimensional point cloud upsampling method based on dynamic scattering transformation, characterized in that, include: Obtain the 3D point cloud to be upsampled, normalize the spatial coordinates of each point to a unit sphere, and obtain the normalized point cloud; A dense connectivity feature extractor based on dynamic graph scattering transform is provided. The normalized point cloud is input into the dense connectivity feature extractor based on dynamic graph scattering transform to extract multi-scale features. The multi-scale features are concatenated by multi-scale heat map convolution, and the features are upsampled to reconstruct the normalized upsampled point cloud coordinates from the upsampled features. The dense connectivity feature extractor based on dynamic graph scattering transform mainly consists of It consists of a dynamic graph scattering feature extraction unit. Let be a positive integer, where: The input to the first feature extractor unit is the normalized point cloud. Spatial coordinates of each point Output 2D features ,in, For the real number field; No. The input to the feature extraction unit is the first... The output of each feature extraction unit Through the first The features obtained by each feature extraction unit and the input passed through cross-layer connections Concatenate to obtain output features ,in, ; The step of inputting the normalized point cloud into the dense connectivity feature extractor based on dynamic graph scattering transform to extract multi-scale features includes: For the Each feature extractor unit will extract the input features. The number of feature channels is compressed to a specified dimension using a one-dimensional convolution to obtain a compressed feature matrix. , Each row corresponds to the feature of each point; A dynamic K-nearest neighbor algorithm is used to construct a graph based on the features of each point. nodes Filter among the remaining nodes and The closest in Euclidean distance K Features Corresponding points As The neighborhood of the region is used to construct undirected edges. , Generate dynamic images and its corresponding adjacency matrix And normalized graph Laplace matrix ; The compressed feature matrix Through a Layered graph scattering network, using normalized graph Laplacian matrix By performing spectral domain graph convolution operations in graph scattering transform, the features output from different scattering paths in the scattering network are aggregated to obtain a multi-scale feature representation of the point cloud. ; Based on the feature matrix of the input point cloud, a dynamic neighborhood of each point in the point cloud is constructed in the feature space. By aggregating the features of each point in the dynamic neighborhood and the center point, dynamic edge convolution is used to update the features of the center point, thereby enhancing the multi-scale features. The point cloud feature matrix output by the dynamic edge convolutional network is obtained. ; By concatenating the input features with the output features of dynamic edge convolution through cross-layer connections, we obtain the first... The output feature matrix of each multi-scale feature extractor unit .

2. The three-dimensional point cloud upsampling method based on dynamic scattering transformation according to claim 1, characterized in that, The compressed feature matrix Through a Layered graph scattering network, using normalized graph Laplacian matrix By performing spectral domain graph convolution operations in graph scattering transform, the features output from different scattering paths in the scattering network are aggregated to obtain a multi-scale feature representation of the point cloud. ,include: Choose a graphical wavelet function and its corresponding scaling function And calculate its Fourier transform. and ; Set minimum resolution scale Construct wavelet functions of different scales and its corresponding scaling function ,in, For integer fields, The graph constructed from the point cloud using feature KNN Graph displacement operator The One characteristic root; Using the normalized graphical Laplacian matrix As the graph shift operator for the spectral domain graph convolution operation in graph scattering transform, i.e., the input feature matrix With Xiaobo The convolution operation is defined as: ， in, For the normalized graph Laplace matrix The One characteristic root; Based on multi-scale wavelet filter bank and graph shift operator Build a A graph scattering network of layers containing all lengths of scattering path in, Each scattering path employs a different combination of multi-scale wavelet filters, based on a series of graph wavelet functions in a fixed order. Learn multi-scale feature representations of point clouds and output multi-scale features of point clouds under different bandpass filters. for: ， in, for Scaling function at scale, sign Represents spectral domain graph convolution; For a scattering path of length 0 The scattering characteristics output by this path are ; Each length is scattering path All outputs the scattering transformation characteristics along this path. All scattering paths in the graph scattering network Output characteristics By concatenating the data, we obtain a multi-scale feature representation of the point cloud based on a graph scattering network with dynamic graph construction. : ， in, for Features of the outputs of all scattering paths in a layered graph scattering network. To perform feature concatenation operations based on row alignment, multi-scale features Dimensions ; By performing multi-scale eigenvalue compression on the graph scattering transform and concatenating it with the input features: ， A stable feature representation of a multi-scale point cloud is obtained, where For 1D convolutional layers, multi-scale features are incorporated. The channel is compressed, making ; The concatenated features are further compressed using 1D convolutional layers, making the input features of the dynamic edge convolutional network more efficient. Channel equals .

3. The three-dimensional point cloud upsampling method based on dynamic scattering transformation according to claim 1, characterized in that, The process involves constructing a dynamic neighborhood for each point in the point cloud based on the feature matrix of the input point cloud and the feature space. By aggregating the features of each point in the dynamic neighborhood and the center point, dynamic edge convolution is used to update the center point features, thereby enhancing the multi-scale features. The point cloud feature matrix output by the dynamic edge convolutional network is obtained. ,include: Based on input features In the feature space, a neighborhood is constructed with each point as the center; for each node... and their corresponding features Filter among the remaining nodes and The closest in Euclidean distance Features Corresponding points As The neighborhood; Calculate the center point Each point in the neighborhood Features Features of the center point displacement ; Concatenate all displacement vectors vertically to obtain a local characterization within the point cloud feature space. : ， Enhance local spatial information of point clouds using one-dimensional convolutional layers. , ; center point Feature replication As a global characterization of the point cloud feature space, and By horizontally stitching together, the features of the center point are obtained. : ， Merge the features of each center point The points obtained are Feature matrix of point cloud ; the feature matrix Inputting to a 2D convolutional layer to learn hidden features of a point cloud A point cloud feature matrix containing local and global information of the point cloud feature space is obtained by cross-layer stitching. : ， Max pooling is performed on the neighborhood features of each center point to update the center node features, resulting in the point cloud feature matrix output by the dynamic edge convolutional network. .

4. The three-dimensional point cloud upsampling method based on dynamic scattering transformation according to claim 1, characterized in that, The step of performing feature concatenation on the multi-scale features through multi-scale heat map convolution, upsampling the features, and reconstructing normalized upsampled point cloud coordinates from the upsampled features includes: Point cloud features are expanded by convolutional feature stitching of multi-scale heat map, and the output features of the multi-scale feature extractor are then used. pass Heat map convolutional networks at different scales: ， Generate point cloud feature matrices at different heat kernel scales Where σ is a nonlinear activation function, The first convolution of the heat map One scale, For the normalized graph Laplacian matrix, The upsampling ratio of the point cloud. These are trainable parameters; the point cloud feature matrix is ​​vertically concatenated to obtain extended features. : ， Each row of the feature matrix corresponds to a feature of a point in the upsampled point cloud; A point cloud upsampling model is provided, comprising a coordinate residual connection network and a two-layer fully connected network; based on the point cloud upsampling model, extended features are used to perform... Input point cloud 3D coordinates Reconstructing the spatial coordinates of the upsampled point cloud: ， Obtain the normalized upsampled point cloud 3D coordinates output by the point cloud upsampled depth network. The number of points in the sampled point cloud is equal to that in the original point cloud. times; among which, Indicates a fully connected network. This indicates that the input point cloud coordinates will be displayed. copy The three-dimensional coordinate matrix obtained by vertical stitching is then obtained.

5. The three-dimensional point cloud upsampling method based on dynamic scattering transformation according to claim 4, characterized in that, The parameters of the point cloud upsampling model are obtained by training based on a weighted sum of the chamfer distance and the regularization term of the parameters as a loss function, including: The upsampled point cloud is obtained using the point cloud upsampling model. Then, the normalized upsampled point cloud spatial 3D coordinates and normalized true values ​​are calculated using the chamfer distance. Differences: ， in, This is the chamfer distance. For point and Euclidean distance, and Point clouds and The number of points in the point cloud; the training of the point cloud upsampling model uses a weighted sum of chamfer distance and parameter regularization term as the loss function. : ， in, and For hyperparameters, These are the learnable parameters in a point cloud upsampling deep network; According to the loss function Calculation parameters The gradient is used to update the parameters in the point cloud upsampling model via backpropagation. The parameters of the point cloud upsampling model are obtained by iteratively optimizing and minimizing the above loss function.

6. A three-dimensional point cloud upsampling system based on dynamic scattering transformation, characterized in that, include: Preprocessing module: acquires the 3D point cloud to be upsampled, normalizes the spatial coordinates of each point to a unit sphere, and obtains a normalized point cloud; Feature extraction module: Provides a dense connectivity feature extractor based on dynamic graph scattering transform, inputting the normalized point cloud into the dense connectivity feature extractor based on dynamic graph scattering transform to extract multi-scale features; wherein: The dense connectivity feature extractor based on dynamic graph scattering transform mainly consists of It consists of a dynamic graph scattering feature extraction unit. Let be a positive integer, where: The input to the first feature extractor unit is the normalized point cloud. Spatial coordinates of each point Output 2D features ,in, For the real number field; No. The input to the feature extraction unit is the first... The output of each feature extraction unit Through the first The features obtained by each feature extraction unit and the input passed through cross-layer connections Concatenate to obtain output features ,in, ; The step of inputting the normalized point cloud into the dense connectivity feature extractor based on dynamic graph scattering transform to extract multi-scale features includes: For the Each feature extractor unit will extract the input features. The number of feature channels is compressed to a specified dimension using a one-dimensional convolution to obtain a compressed feature matrix. , Each row corresponds to the feature of each point; A dynamic K-nearest neighbor algorithm is used to construct a graph based on the features of each point. nodes Filter among the remaining nodes and The closest in Euclidean distance K Features Corresponding points As The neighborhood of the region is used to construct undirected edges. , Generate dynamic images and its corresponding adjacency matrix And normalized graph Laplace matrix ; The compressed feature matrix Through a Layered graph scattering network, using normalized graph Laplacian matrix By performing spectral domain graph convolution operations in graph scattering transform, the features output from different scattering paths in the scattering network are aggregated to obtain a multi-scale feature representation of the point cloud. ; Based on the feature matrix of the input point cloud, a dynamic neighborhood of each point in the point cloud is constructed in the feature space. By aggregating the features of each point in the dynamic neighborhood and the center point, dynamic edge convolution is used to update the features of the center point, thereby enhancing the multi-scale features. The point cloud feature matrix output by the dynamic edge convolutional network is obtained. ; By concatenating the input features with the output features of dynamic edge convolution through cross-layer connections, we obtain the first... The output feature matrix of each multi-scale feature extractor unit ; Point cloud reconstruction module: The multi-scale features are spliced ​​by multi-scale heat map convolution, and the features are upsampled to reconstruct the normalized upsampled point cloud coordinates from the upsampled features.

7. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method of any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1-5.

Images