Three-dimensional point cloud encoding and decoding method, compression method and device based on graph dictionary learning

Abstract

The present invention provides a three-dimensional point cloud graph dictionary learning method and a coding and decoding method, a compression method and a device based on graph dictionary learning, including: obtaining N training set point cloud data; voxelizing the point cloud data to obtain voxelized training set point cloud data; dividing the training set point cloud data into voxel blocks, selecting several voxel blocks as training sets, and constructing a graph dictionary learning model according to the training set; iteratively optimizing the objective function of graph dictionary learning to obtain a graph dictionary for coding and decoding of three-dimensional point cloud signals. The present invention effectively utilizes the spatial correlation between point cloud signals to progressively and optimally remove the redundancy between point cloud signals; utilizes the data correlation between signals at different levels to perform predictive coding from the low level to the top down, effectively improving the compression efficiency of 3D point cloud attribute signals, effectively reducing coding overhead, flexibly meeting the requirements for decoding signals of different qualities in actual needs, and having scalability.

Description

Technical Field

[0001] The present invention relates to a solution in the technical field of 3D point cloud data compression, and in particular to a 3D point cloud encoding and decoding method, a compression method and a device based on graph dictionary learning. Background Art

[0002] In recent years, with the rapid development of 3D acquisition and perception equipment, irregular data such as 3D point clouds have been widely used in many emerging technology fields such as autonomous driving, virtual reality, augmented reality, 3D distant video communication, and ruins reconstruction. A 3D point cloud image usually contains millions of points containing geometric and attribute information, so its data size is often huge, requiring a lot of computing resources and storage space. Constrained by the actual network bandwidth and limited storage space, the existing storage and transmission requirements in this way have far exceeded the resource limitations of nodes and local computing. Therefore, exploring effective 3D point cloud compression methods has become an important issue that needs to be solved urgently.

[0003] After searching the literature on the prior art, it was found that the early 3D point cloud attribute compression schemes used fixed analytical basis functions to represent and encode point cloud signals. In the paper "Point Cloud Attribute Compression with Graph Transform" published at the 2014 IEEE International Conference on Image Processing (ICIP2014), C. Zhang et al. pioneered the use of graph Fourier transform to remove the correlation between the attribute signals of point cloud voxel blocks. It uses the spatial correlation between point cloud voxel blocks to construct a nearest neighbor graph, and calculates the graph Laplacian matrix as the transformation basis matrix to encode the point cloud signal. However, since the calculation of the graph Laplacian matrix requires eigenvalue decomposition, the computational complexity is very high, which is not conducive to real-time transmission.

[0004] In order to reduce the computational complexity, R.L. de Queiroz et al. proposed a standard orthogonal region-adaptive Haar transform (RAHT) in the article "Compression of 3D Point Clouds Using a Region-Adaptive Hierarchical Transform" published in the journal "IEEE Transactions on Image Processing" (TIP2016) in 2016. The method uses the geometric information of the 3D point cloud to adaptively design the Haar transformation basis and encode the point cloud. Subsequently, S. Gu et al. proposed a method based on "3D point cloud attribute compression using geometry-guided sparse representation" in the article "3D point cloud attribute compression using geometry-guided sparse representation" published in the journal "IEEE Transactions on Image Processing" (TIP2020) in 2020. The norm-constrained geometric adaptive sparse coding method uses the geometric information of the point cloud signal to design a fixed variation basis and removes the redundancy between the point cloud signals based on the sparse coding method to perform coding compression. However, the above non-Euclidean space sparse representation method relies heavily on geometric information, ignores the statistical characteristics of the signal itself, and requires the use of a fixed transformation basis derived from a strict mathematical expression, which cannot adaptively express complex spatial irregular multidimensional signals.

[0005] In the paper "Multi-scale structured dictionary learning for 3-D point cloud attribute compression" published in the 2020 IEEE Transactions on Circuits and Systems for Video Technology (TCSVT2020), Y. Shen et al. proposed a structured sparse representation coding scheme based on multi-scale dictionary learning, which uses the statistical characteristics of the 3D point cloud signal itself to learn a multi-scale dictionary and obtain a structured sparse representation result to improve the compression representation performance. However, this scheme does not take into account the inherent potential structural characteristics of the signal itself, and therefore cannot effectively express irregular multidimensional signals distributed in complex topological spaces, which limits its representation compression performance.

[0006] In addition, the MPEG 3DG PCC standardization organization has been soliciting proposals for 3D point cloud compression technology since 2017, and has now entered the final draft writing stage. It divides 3D point clouds into three different categories: static point clouds (Category 1), dynamic time-varying point clouds (Category 2), and dynamically acquired point clouds (Category 3). For these three different types of 3D point clouds, it has released three different test models (TMs), including TMC1 for static point clouds using regional adaptive wavelet transform, TMC2 for dynamic time-varying point clouds based on image and video coding schemes, and TMC3 for dynamically acquired point clouds based on predictive lifting transform. Since TMC1 and TMC3 are both transform coding schemes based on geometric information, they were eventually merged together and called the Geometry-based Point Cloud Compression (G-PCC) standard technology; while TMC2 is called the Video-based Point Cloud Compression (V-PCC) standard technology by projecting 3D point clouds into 2D plane sequences and compressing them using existing video coding standards such as HEVC. However, V-PCC will introduce inevitable distortion during the projection process, and the attribute space transformation scheme of G-PCC only relies on geometric information without considering the statistical characteristics of the attribute signal itself, thus reducing the compression quality of the attribute signal. Summary of the invention

[0007] In view of the above problems, the present invention proposes a graph dictionary learning method for three-dimensional point cloud and an encoding and decoding method, a compression method and a device based on graph dictionary learning.

[0008] According to a first aspect of the present invention, a method for learning a graph dictionary of a three-dimensional point cloud is provided, comprising:

[0009] Obtain N training set point cloud data;

[0010] Performing voxelization processing on the N training set point cloud data to obtain voxelized training set point cloud data;

[0011] Dividing the voxelized training set point cloud data into voxel blocks, selecting a number of voxel blocks as training sets for graph dictionary learning, constructing a graph dictionary learning model based on the training sets, and establishing a graph dictionary learning objective function;

[0012] The objective function of the graph dictionary learning is iteratively optimized to obtain a graph dictionary for encoding and decoding of three-dimensional point cloud signals.

[0013] Preferably, voxelizing the N training set point cloud data to obtain voxelized training set point cloud data includes:

[0014] Divide the point cloud data into regular voxels aligned with the spatial coordinate system;

[0015] The geometric coordinates and attribute signals of each voxel are calculated to obtain voxelized point cloud data.

[0016] Preferably, the voxelized training set point cloud data is divided into voxel blocks, and a number of voxel blocks are selected as training sets for graph dictionary learning, including:

[0017] The bounding cube where all the voxelized training set point cloud data are located is evenly divided into m×m×m voxel blocks, where m is a preset side length;

[0018] All voxel blocks of each training set point cloud data are sorted in descending order according to the number of voxels they contain;

[0019] Select the first r voxel blocks containing the largest number of voxels from each training set point cloud data, calculate the average value of the attribute signals of the voxels contained in each voxel block in the first r voxel blocks, and use it as the DC attribute signal of the voxel block, where r is a preset positive integer;

[0020] The DC attribute signal is subtracted from each voxel to obtain a residual attribute signal as a training signal to form a training set for graph dictionary learning.

[0021] Preferably, the graph dictionary learning model is constructed according to the training set, and a graph dictionary learning objective function is established, wherein the graph dictionary learning model is constructed based on a nonlinear p-Laplace operator, including:

[0022] The training set signal is used as a graph signal to construct a graph topology connection structure Where V represents m 3 A set of nodes, ε represents the connection m 3 The set of edges of nodes, Represents the edge weight matrix;

[0023] The characteristic basis function of the p-Lap-Pull operator for constructing the graph topological connection structure is used to construct a graph dictionary learning model, and a graph dictionary learning objective function is established:

[0024]

[0025] in, represents the i-th training collective pixel block, is a mask matrix whose elements are either 0 or 1 and is used to transform x i Extract the corresponding m i Individual elements, Represents a complete sub-dictionary consisting of S The overcomplete graph dictionary composed of S ], Used to constrain the corresponding complete sub-dictionary D s p s The characteristic basis of the Laplap operator, w ij represents the element in the i-th row and j-th column of the weight matrix W, d sk represents the kth atom of the sth sub-dictionary, d ski Indicates d sk The i-th element of represents a sparse coefficient matrix, The column vector of the sparse coefficient matrix A represents the signal x i The sparse coefficients on the graph dictionary D, represents the identity matrix, n = Nr is the number of training blocks, p s represents the power parameter corresponding to the s-th sub-dictionary, and β, γ, and λ represent the corresponding regularization parameters.

[0026] Preferably, iteratively optimizing the objective function of the graph dictionary learning model to obtain a graph dictionary for encoding and decoding of three-dimensional point cloud signals includes:

[0027] The graph dictionary learning objective function is iteratively optimized to obtain a graph dictionary, wherein in each iteration, the dictionary D is fixed to solve the geometric adaptive sparse coding subproblem and the sparse coefficient matrix A is fixed to solve the graph dictionary update subproblem, until a predetermined number of iterations are reached, to obtain a graph dictionary for encoding and decoding of three-dimensional point cloud signals.

[0028] Preferably, iteratively optimizing the graph dictionary learning objective function to obtain the graph dictionary includes:

[0029] According to the fixed graph dictionary D obtained by solving the graph dictionary update subproblem, for each training set of pixels, the geometric adaptive sparse coding subproblem is solved one by one to obtain the sparse coefficient matrix A:

[0030]

[0031] According to the fixed sparse coefficient matrix A obtained by solving the geometric adaptive sparse coding subproblem, the graph dictionary update subproblem is solved by using the alternating direction multiplier method to obtain the graph dictionary D:

[0032]

[0033] in, represents the Hadamard product of element-wise multiplication, represents the set of training set signals, Represents the set of mask matrices for all training set signals, Represents a vector whose elements are all 1.

[0034] Preferably, the alternating direction multiplier method comprises:

[0035] By introducing an auxiliary variable Z instead of D, the non-convex second and third terms in the dictionary update optimization subproblem are separated from the convex first term and transformed into a constrained optimization problem:

[0036]

[0037] Where Z = [Z1, ..., Z S ],Z s =D s , s = 1, ..., S represents the sub-auxiliary variable corresponding to each dictionary;

[0038] Introducing a dual variable Transforming the constrained optimization problem into its augmented Lagrangian function, we obtain three sub-optimization problems:

[0039]

[0040]

[0041] U + =U+DZ

[0042] Among them, ρ represents the introduced penalty parameter;

[0043] The three sub-optimization problems are solved alternately to update the three variables alternately until convergence, and a graph dictionary is obtained.

[0044] According to a second aspect of the present invention, a three-dimensional point cloud signal encoding method based on graph dictionary learning is provided, and the three-dimensional point cloud signal is encoded according to the graph dictionary obtained by the above method to generate a three-dimensional point cloud signal code stream.

[0045] Preferably, the encoding of the three-dimensional point cloud signal to generate a three-dimensional point cloud signal code stream includes:

[0046] Dividing the three-dimensional point cloud signal into quality layers to obtain l quality layers;

[0047] Encode the point cloud signal of a single quality layer to obtain the point cloud signal code stream of the quality layer;

[0048] Decode the point cloud signal of a single quality layer to obtain the reconstructed point cloud signal of the quality layer;

[0049] Encode and decode l quality layers in turn, merge l quality layer point cloud signal code streams, and obtain a three-dimensional point cloud signal code stream.

[0050] Preferably, dividing the three-dimensional point cloud signal into quality layers to obtain l quality layers includes:

[0051] Divide the three-dimensional point cloud data into l quality layers, from low to high quality layers, the first layer is the basic layer, and the l-1 layers thereafter are enhanced layers, from the 1st enhanced layer to the l-1th enhanced layer, respectively, and obtain point cloud signals of the basic layer and each of the enhanced layers respectively;

[0052] For the point cloud signals of the obtained l quality layers, the lowest quality layer finally obtained is used as the basic layer, and the other l-1 layers are used as the 1st to l-1th enhancement layers in order of quality from low to high.

[0053] Preferably, obtaining the point cloud signals of the base layer and each enhancement layer includes:

[0054] Voxelize point cloud data and output voxelized point cloud signals as point cloud signals of the highest quality layer;

[0055] Performing l-1 voxel block merging on the voxelized point cloud signal to obtain l-1 point cloud signals of different quality layers as point cloud signals of the basic layer and the first to l-2th enhancement layers;

[0056] The input of each voxel block merging is the point cloud signal of the high-quality layer obtained by the previous voxel block merging, and the point cloud signal of the low-quality layer is output as the input of the next voxel block merging.

[0057] Preferably, the voxel block merging comprises:

[0058] Divide the boundary cube where the point cloud signal of the high-quality layer is located into evenly distributed c×c×c voxel blocks;

[0059] Merging the voxels contained in each voxel block as a voxel of the point cloud signal of the low-quality layer;

[0060] The maximum value of the geometric coordinates of the voxels contained in each voxel block is reduced by c times and rounded off to an integer as the geometric coordinates of the corresponding voxels in the point cloud signal of the low-quality layer;

[0061] Taking the average value of the attribute signals of the voxels contained in each voxel block as the value of the attribute signal of the corresponding voxel in the point cloud signal of the low-quality layer;

[0062] A point cloud signal of the low-quality layer is obtained.

[0063] Preferably, the single quality layer point cloud signal encoding to obtain the point cloud signal code stream of the quality layer includes:

[0064] Geometric information encoding: the geometric coordinates of the point cloud signal are divided according to the octree structure, and an eight-bit byte unit is allocated to each branch node in the octree structure to indicate whether there is point cloud point data in the spatial unit corresponding to the child node of the branch node. The octree structure is traversed in depth-breadth priority order to obtain a byte unit sequence representing the geometric coordinates of all spatial units, and the geometric information code stream is obtained by entropy encoding the byte unit sequence;

[0065] Inter-layer attribute signal prediction: for the enhancement layer only, the geometric coordinates of the decoded previous layer reconstructed point cloud signal and the current layer point cloud signal are used to obtain the inter-layer prediction signal of the current layer attribute signal for attribute signal preprocessing;

[0066] Attribute signal preprocessing: divide the boundary cube where the point cloud signal of the current layer is located into m×m×m voxel blocks evenly according to the preset side length m; if the input quality layer is a basic layer, average the values ​​of the attribute signals of the voxels contained in each voxel block to obtain a DC attribute signal, and subtract the DC attribute signal from the attribute signals of the voxels contained in each voxel block to obtain a residual attribute signal; if the input quality layer is an enhanced layer, subtract the inter-layer prediction signal from the attribute signal of the current layer to obtain a residual attribute signal of the current layer;

[0067] DC attribute signal encoding: uniformly quantize only the DC attribute signal of the base layer, make a difference between the quantized DC attribute signal of each voxel block and the adjacent voxel block, and entropy encode to obtain the DC attribute signal code stream;

[0068] Residual attribute signal encoding, using the graph dictionary and the geometric coordinates of the current layer to perform geometric adaptive sparse encoding on the residual attribute signal of each cubic block to obtain a sparse coefficient matrix, uniformly quantizing the sparse coefficient matrix, and entropy encoding the values ​​and positions of non-zero elements in the quantized sparse coefficient matrix to obtain a residual attribute signal code stream;

[0069] The point cloud signal code stream is merged to merge the geometric information code stream and the attribute signal code stream of the l′th quality layer to obtain the point cloud signal code stream of the l′th quality layer.

[0070] Preferably, the inter-layer attribute signal prediction includes:

[0071] Determine the quality layer of the input, if it is the basic layer, no operation;

[0072] If it is an enhanced layer, the bounding cube where the point cloud signal of the current layer is located is divided into evenly distributed c×c×c voxel blocks using geometric coordinates, and the attribute signals of the voxels contained in each voxel block are predicted;

[0073] The geometric coordinates of each voxel of the previous layer point cloud signal are enlarged by c times and rounded to the nearest integer. The voxel of the previous layer point cloud signal corresponding to the 19 connected voxel blocks with the closest spatial distance to each voxel block is found, including itself. The spatial distance between the voxel contained in each voxel block and the 19 voxels connected to the voxel block in the previous layer point cloud signal is calculated one by one. The 7 connected voxels with the closest spatial distance are selected, and the attribute signals of the 7 connected voxels are weighted averaged with the inverse of the spatial distance of the 7 connected voxels as the weight to obtain the inter-layer prediction signal of the voxel contained in the voxel block.

[0074] Preferably, the residual attribute signal encoding includes:

[0075] Using the geometric coordinates of the voxels contained in each voxel block, the corresponding rows are extracted from the graph dictionary to form a sub-dictionary;

[0076] Normalizing the sub-dictionary corresponding to each extracted voxel block;

[0077] Using the normalized sub-dictionary, the residual attribute signals of the voxels contained in each voxel block are calculated based on Regularize the sparse terms to perform geometric adaptive sparse coding and obtain a sparse coefficient matrix:

[0078] The sparse coefficient matrix is ​​uniformly quantized, and the values ​​and positions (row coordinates and column coordinates) of the non-zero elements of the quantized sparse coefficient matrix are entropy encoded respectively, and the residual attribute signal code stream is obtained by merging.

[0079] Preferably, decoding the single quality layer point cloud signal to obtain the reconstructed point cloud signal of the quality layer includes:

[0080] Splitting the point cloud signal code stream, splitting the point cloud signal code stream of the l′th quality layer to obtain the geometric information code stream and the attribute signal code stream of the l′th quality layer;

[0081] Geometric information decoding, obtaining an octree structure and a byte unit of each branch node from the geometric information code stream entropy decoding, and then obtaining the geometric coordinates of each voxel;

[0082] Inter-layer attribute signal prediction, obtaining the inter-layer prediction signal from the reconstructed point cloud signal of the previous quality layer;

[0083] DC attribute signal decoding: only entropy decoding is performed on the DC attribute signal bit stream of the base layer. For each voxel block, the decoded DC attribute signal of the adjacent voxel block is added, and inverse uniform quantization is performed to obtain a reconstructed DC attribute signal.

[0084] Residual attribute signal decoding, entropy decoding and inverse quantization of the residual attribute signal code stream to obtain a reconstructed sparse coefficient matrix; using the geometric coordinates of the voxels contained in each voxel block, extract the corresponding row composition sub-dictionary from the graph dictionary and normalize it; multiply the normalized sub-dictionary and the sparse vector corresponding to the voxel block in the reconstructed coefficient matrix to obtain the reconstructed residual attribute signal of the voxels contained in the corresponding voxel block;

[0085] Three-dimensional point cloud reconstruction, determine the quality layer of the input, if it is a basic layer, add the basic layer DC attribute signal and the basic layer residual attribute signal to obtain the reconstructed attribute signal of the basic layer; if it is an enhanced layer, add the inter-layer prediction signal and the enhanced layer residual attribute signal to obtain the reconstructed attribute signal of the enhanced layer; align the reconstructed attribute signal and the geometric coordinates to obtain the reconstructed point cloud signal of the current layer, which is used for single quality layer point cloud signal encoding and point cloud signal decoding of the next enhanced layer.

[0086] According to a third aspect of the present invention, a three-dimensional point cloud signal decoding method based on graph dictionary learning is provided, which is used to decode the three-dimensional point cloud signal code stream obtained by the three-dimensional point cloud signal encoding method based on graph dictionary learning, comprising:

[0087] Extract the point cloud signal bitstreams of the first l′ quality layers from the 3D point cloud signal bitstream, where l′ is any positive integer not less than l, and corresponds to the base layer, the first enhancement layer, …, the l′-1th enhancement layer in order of the number of layers;

[0088] According to the graph dictionary, single quality layer point cloud signals are decoded for l′ quality layers in sequence starting from the basic layer to obtain reconstructed three-dimensional point cloud signals corresponding to the first l′ layers, where l′≤l.

[0089] According to a fourth aspect of the present invention, a three-dimensional point cloud compression method based on graph dictionary learning is provided, comprising:

[0090] Obtaining point cloud data to be encoded, wherein the point cloud data to be encoded is a three-dimensional point cloud signal;

[0091] The three-dimensional point cloud signal is encoded into a three-dimensional point cloud signal code stream by using the three-dimensional point cloud signal encoding method based on graph dictionary learning.

[0092] The three-dimensional point cloud signal decoding method based on graph dictionary learning is adopted to decode and reconstruct the three-dimensional point cloud signal from the three-dimensional point cloud signal code stream.

[0093] According to a fifth aspect of the present invention, there is provided a three-dimensional point cloud processing device based on graph dictionary learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; the processor may execute any of the following methods when executing the program:

[0094] - the above-mentioned 3D point cloud dictionary learning method, or,

[0095] - the above-mentioned three-dimensional point cloud signal encoding method based on graph dictionary learning, or,

[0096] - the above-mentioned three-dimensional point cloud signal decoding method based on graph dictionary learning, or,

[0097] -The above-mentioned 3D point cloud compression method based on graph dictionary learning.

[0098] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0099] The three-dimensional point cloud graph dictionary learning method in the embodiment of the present invention can effectively utilize the spatial correlation between point cloud signals and gradually and optimally remove the redundancy between point cloud signals;

[0100] The 3D point cloud encoding and decoding scheme, compression method and device based on graph dictionary learning in the embodiment of the present invention are suitable for lossless geometry and lossy attribute compression of point cloud signals, and use the data correlation between signals at different levels to perform predictive coding from the low level to the top, which effectively improves the compression efficiency of 3D point cloud attribute signals, and can not only effectively reduce the encoding overhead, but also flexibly meet the requirements of different quality decoding signals in actual needs, and have scalability;

[0101] The three-dimensional point cloud encoding and decoding scheme, compression method and device based on graph dictionary learning in the embodiments of the present invention can adaptively utilize the statistical prior information of the training set data compared with the analytical transformation based on graph Fourier, or region adaptive Haar wavelet and geometry-guided sparse coding methods based on graph Fourier basis, and can effectively utilize the spatial topological correlation of the training set signal compared with the structured sparse representation coding scheme based on multi-scale dictionary learning, so the reconstruction quality of the point cloud signal can be effectively improved; compared with the MPEG PCC international standard, it does not only rely on geometric information to construct the attribute transformation basis, and the learned graph dictionary can effectively utilize the statistical prior information and topological structure information between the attribute signals at the same time, thereby achieving significant performance gains. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0103] Figure 1 It is a structural block diagram of a preferred embodiment of the present invention;

[0104] Figure 2 It is a structural block diagram of single quality layer point cloud signal encoding according to an embodiment of the present invention;

[0105] Figure 3 A structural block diagram of single quality layer point cloud signal decoding according to an embodiment of the present invention;

[0106] Figure 4 This is a structural block diagram of inter-layer attribute signal prediction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0107] The present invention is described in detail below in conjunction with specific embodiments and accompanying drawings. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0108] The present invention uses the inherent topological structure information of the point cloud signal to learn a graph dictionary to perform asymptotically optimal sparse representation of the point cloud signal, effectively removes redundant information between signals, and divides the point cloud signal into different quality levels. The data correlation between signals of different levels is used to perform predictive coding from the low level to the top down, which can not only effectively reduce the coding overhead, but also flexibly meet the actual needs for different quality decoding signals. The compression effect exceeds the existing optimal transform coding scheme and MPEG 3DG G-PCC international standard scheme. Based on the above invention concept, the present invention provides an embodiment, such as Figure 1 As shown, a graph dictionary learning method for three-dimensional point cloud includes:

[0109] S100 obtains N training set point cloud data, including the geometric coordinates of each point as well as color, normal vector, etc. as attribute signals;

[0110] S200 voxelizes point cloud data, divides the point cloud data into voxels that are aligned in the spatial coordinate system and evenly distributed, calculates the geometric coordinates and attribute signals of each voxel, and obtains voxelized point cloud data;

[0111] S300 voxel block division: divide the boundary cube where the voxelized point cloud data is located into m×m×m voxel blocks evenly, where m is a preset side length; and sort the voxel blocks to obtain a training set;

[0112] S400 builds a graph dictionary learning model and optimizes it to obtain a graph dictionary for encoding and decoding three-dimensional point cloud signals.

[0113] In this embodiment, m is set to 8 to balance performance and computational complexity, r is set to 50, the total number of point cloud training sets N is 2000, and a total of 10,000 blocks are extracted. In other embodiments, other parameters may also be selected, and the present invention is not limited to the above parameters.

[0114] In order to effectively utilize the spatial correlation between point cloud signals and gradually and optimally remove the redundancy between point cloud signals, the present invention provides a preferred embodiment to perform S200 voxelization of point cloud data, including:

[0115] S201 uses the octree structure to recursively divide the boundary cube where the point cloud data is located. Starting from the boundary cube, each cube is divided into 8 small cubes from top to bottom, and it is iterated in sequence until the specified division level L is reached, and a plurality of leaf node cube blocks that are evenly distributed and aligned with the coordinate axis are obtained, and each non-empty leaf node cube block containing a point is regarded as a voxel; usually, the division level number L of the octree is set to 9 or 10. Of course, in other embodiments, other division times can also be selected, and the present invention is not limited to the above parameters;

[0116] S202 merges the points contained in each voxel. Calculate the geometric coordinates and attribute signals of each voxel, wherein the geometric coordinates of the center position of the leaf node cube block corresponding to each voxel are rounded and quantized to integers as the geometric coordinates of the merged voxel, and the average value of the attribute signals of the points contained in the leaf node cube block corresponding to each voxel is used as the attribute signal of the merged voxel.

[0117] In one embodiment of the present invention, the voxel blocks are sorted in S300 to obtain a training set, including: sorting all voxel blocks of the point cloud data in descending order according to the number of voxels contained, selecting the first r voxel blocks containing the largest number of voxels, calculating the average value of the attribute signals of the voxels contained in each voxel block as the DC attribute signal of the voxel block, and subtracting the DC attribute signal from each voxel to obtain a residual attribute signal as a training signal to form a training set for graph dictionary learning. Wherein, r is a pre-set positive integer.

[0118] The present invention provides a preferred embodiment to execute S400 to construct a graph dictionary learning model and perform optimization and solution to obtain a graph dictionary for encoding and decoding of three-dimensional point cloud signals, wherein the graph dictionary learning model is constructed based on a nonlinear p-Laplacian operation operator, including:

[0119] S401 uses the training set signal as a graph signal to construct a graph topology connection structure Where V represents m 3 A set of nodes, ε represents the connection m 3 The set of edges of nodes, Represents the edge weight matrix; in this embodiment, the ∈-neighborhood graph connection structure is constructed according to the spatial position relationship That is, given a threshold ∈, if the distance between two nodes of the graph signal is less than the threshold, the two nodes are connected to form an edge of the graph structure, and a weight of 1 is assigned to the edge, otherwise, the two nodes are not connected, where ∈ is 1. Of course, in other embodiments, other composition schemes or other threshold parameters may be selected, and the present invention is not limited to the above-mentioned composition schemes and parameter settings;

[0120] S402 constructs a graph dictionary learning model using the characteristic basis function of the p-Lap-Pull operator of the constructed graph structure, and optimizes the graph based on Objective function of norm sparse penalty term:

[0121]

[0122] in, represents the i-th training collective pixel block, is a mask matrix whose elements are either 0 or 1 and is used to transform x i Extract the corresponding m i Individual elements, Represents a complete sub-dictionary consisting of S The overcomplete graph dictionary composed of S ], Used to constrain the corresponding complete sub-dictionary D s p s The characteristic basis of the Laplap operator, w ij represents the element in the i-th row and j-th column of the weight matrix W, d sk represents the kth atom of the sth sub-dictionary, d ski Indicates d sk The i-th element of represents a sparse coefficient matrix, The column vector of the sparse coefficient matrix A represents the signal x i The sparse coefficients on the graph dictionary D, represents the identity matrix, n = Nr is the number of training blocks, p s represents the power parameter corresponding to the s-th sub-dictionary, and β, γ, and λ represent the corresponding regularization parameters.

[0123] In the graph dictionary learning optimization objective function, the first term represents the data fitting term, which makes the graph dictionary and the corresponding sparse code fit the original data; the second term represents the constraint on the p-Laplacian operator characteristic basis, which is used to constrain the topological connection structure of the graph dictionary, so that the learned graph dictionary can keep the same topological change law as the original signal as close as possible; the third term represents the constraint on the orthogonality of the graph dictionary, which is used to remove the correlation between the atoms in the graph dictionary; the fourth term is the sparse regularization term, which is used to constrain the sparsity of the obtained coefficients;

[0124] S403 iteratively optimizes the graph dictionary learning objective function, and alternately fixes the dictionary D to solve the geometric adaptive sparse coding subproblem and fixes the sparse coefficient matrix A to solve the graph dictionary update subproblem in each iteration, until a predetermined number of iterations are reached to obtain the graph dictionary. Specifically, it includes:

[0125] S4031 solves the geometric adaptive sparse coding subproblem based on the graph dictionary D obtained by solving the graph dictionary update subproblem to obtain the sparse coefficient matrix A:

[0126]

[0127] Here, the geometry-adaptive sparse coding subproblem can be solved using the commonly used The norm constrained sparse coding method is used for solving, including but not limited to least angle regression (LARS), basis pursuit (BP), iterative shrinkage threshold method (ISTA), alternating direction multiplier method (ADMM), etc.; in this embodiment, the least angle regression (LARS) method is selected for solving;

[0128] S4032 solves the graph dictionary update subproblem using the alternating direction multiplier method based on the fixed sparse coefficient matrix A obtained by solving the geometric adaptive sparse coding subproblem to obtain the graph dictionary D:

[0129]

[0130] in, represents the Hadamard product of element-wise multiplication, represents the set of training set signals, Represents the set of mask matrices for all training set signals, Represents a vector whose elements are all 1.

[0131] Furthermore, the alternating direction multiplier method includes:

[0132] By introducing an auxiliary variable Z instead of D, the non-convex second and third terms in the dictionary update optimization subproblem are separated from the convex first term and transformed into a constrained optimization problem:

[0133]

[0134] Where Z = [Z1, ..., Z S ],Z s =D s , s=1,..., S represents the sub-auxiliary variables corresponding to each dictionary.

[0135] By introducing a dual variable Transform the above constrained optimization problem into its augmented Lagrangian function, and then obtain three sub-optimization problems:

[0136]

[0137]

[0138] U + =U+DZ

[0139] Among them, ρ represents the introduced penalty parameter.

[0140] The three sub-optimization problems are solved alternately to update the three variables alternately until convergence, and a graph dictionary is obtained.

[0141] In this embodiment, in order to eliminate the complex calculation of the Hadamard product, D can be solved in parallel by directly obtaining the analytical solution for the elements of each row thereof. Since the Z sub-optimization problem is non-convex, the stochastic gradient descent algorithm is selected for solving it.

[0142] The above-mentioned embodiment of the present invention effectively utilizes the topological structure relationship between point cloud signals to learn a graph dictionary, which progressively and optimally removes the redundancy between point cloud signals and improves the coding efficiency.

[0143] Based on the same inventive concept, an embodiment of the present invention further provides a three-dimensional point cloud coding method based on graph dictionary learning. According to the graph dictionary, the three-dimensional point cloud signal is encoded to generate a three-dimensional point cloud signal code stream. The three-dimensional point cloud signal code stream can be truncated into l quality layers, l is any positive integer, and any first l' (l'≤l) quality layers can be decoded to obtain a reconstructed three-dimensional point cloud signal of different quality, including:

[0144] S1 quality layer division, divide the 3D point cloud data into l quality layers, from low to high quality layers, the first layer is the basic layer, and the l-1 layers are the enhancement layers, from the 1st enhancement layer to the l-1th enhancement layer, and obtain the point cloud signals of the basic layer and each enhancement layer respectively;

[0145] S2 single quality layer point cloud signal encoding, encoding the point cloud signal of the l′th quality layer (1≤l′≤l) of the input, wherein the geometric coordinates are losslessly encoded to generate the geometric coordinate code stream of the l′th quality layer; the attribute signal is losslessly encoded using the graph dictionary and the geometric coordinates to generate the attribute signal code stream of the l′th quality layer; the geometric information code stream and the attribute signal code stream of the l′th quality layer are combined to obtain the point cloud signal code stream of the l′th quality layer;

[0146] S3 single quality layer point cloud signal decoding, split the l′th (1≤l′≤l) quality layer point cloud signal code stream, obtain the geometry information code stream and attribute signal code stream of the l′th quality layer; decode the geometry information code stream of the l′th quality layer to obtain the geometry coordinates; use the geometry coordinates of the l′th quality layer and the graph dictionary to decode the attribute signal code stream of the l′th quality layer to obtain the reconstructed attribute signal of the l′th quality layer; combine the geometry coordinates and the reconstructed attribute signal of the l′th quality layer as the reconstructed point cloud signal of the l′th quality layer, which is used for the encoding and decoding of the l′+1th quality layer point cloud signal;

[0147] S4 sequentially encodes and decodes the l quality layers, merges the l quality layer point cloud signal code streams, and obtains a three-dimensional point cloud signal code stream. In this embodiment, l is 3. Of course, in other embodiments, other levels can also be selected. The present invention is not limited to the above-mentioned level setting.

[0148] The above-mentioned embodiments of the present invention effectively improve the compression efficiency of 3D point cloud attribute signals, which can not only effectively reduce the encoding overhead, but also flexibly meet the requirements for decoding signals of different qualities in actual needs.

[0149] like Figure 1 As shown, in a preferred embodiment, the S1 quality layer division process is:

[0150] S11 voxelizes the point cloud data and outputs the voxelized point cloud signal as the point cloud signal of the highest quality layer; specifically, the boundary cube where the point cloud signal of the high quality layer is located is divided into evenly distributed c×c×c voxel blocks.

[0151] S12 performs l-1 voxel block merging on the voxelized point cloud signal to obtain l-1 point cloud signals of different quality layers. The input of each voxel block merging is the point cloud signal of the high-quality layer obtained by the previous voxel block merging, and the point cloud signal of the low-quality layer is output as the input of the next voxel block merging. Specifically,

[0152] The voxels contained in each voxel block are merged as a voxel of the point cloud signal of the low-quality layer, the maximum value of the geometric coordinates of the voxels contained in each voxel block is reduced by c times and rounded off as the geometric coordinates of the corresponding voxels in the point cloud signal of the low-quality layer, and the average value of the attribute signals of the voxels contained in each voxel block is used as the value of the attribute signal of the corresponding voxel in the point cloud signal of the low-quality layer, so as to obtain the point cloud signal of the low-quality layer.

[0153] S13 obtains the point cloud signals of the l quality layers, uses the lowest quality layer finally obtained as the basic layer, and uses the other l-1 layers as the 1st to l-1th enhancement layers in order of quality from low to high.

[0154] Among them, different sizes of c determine the difference between adjacent quality layer point cloud signals. The larger c is, the greater the difference between quality layer signals is, the worse the inter-layer prediction effect will be, and the larger the value of the residual attribute signal will be. In this embodiment, c is preferably set to 2 to reduce the prediction residual between adjacent quality layers and improve compression performance.

[0155] like Figure 2 As shown, in a preferred embodiment, S2 single quality layer point cloud signal encoding encodes the point cloud signal of the l′th (1≤l′≤l) quality layer of the input, wherein: the geometric coordinates are losslessly encoded to generate a geometric coordinate code stream of the l′th quality layer; the attribute signal is losslessly encoded using a graph dictionary and geometric coordinates to generate an attribute signal code stream of the l′th quality layer; the geometric information code stream and the attribute signal code stream of the l′th quality layer are merged to obtain the l′th quality layer point cloud signal code stream.

[0156] Specifically, the single quality layer point cloud signal encoding includes: geometric information encoding, inter-layer attribute signal prediction, attribute signal preprocessing, DC attribute signal encoding and residual attribute signal encoding, and point cloud signal code stream merging.

[0157] S21 geometric information encoding: the geometric coordinates of the point cloud signal are divided according to the octree structure, and an eight-bit byte unit is allocated to each branch node in the octree structure to indicate whether there is point cloud point data in the spatial unit corresponding to the child node of the branch node. The octree structure is traversed in depth-breadth priority order to obtain a byte unit sequence representing the geometric coordinates of all spatial units, and the geometric information code stream is obtained by entropy encoding the byte unit sequence;

[0158] S22 inter-layer attribute signal prediction, which does not operate on the input basic layer, but uses the geometric coordinates of the decoded previous layer reconstructed point cloud signal and the current layer point cloud signal to obtain the inter-layer prediction signal of the current layer attribute signal for attribute signal preprocessing;

[0159] S23 attribute signal preprocessing, dividing the boundary cube where the point cloud signal of the current layer is located into m×m×m voxel blocks evenly according to the preset side length m; if the input quality layer is a basic layer, averaging the values ​​of the attribute signals of the voxels contained in each voxel block to obtain a DC attribute signal, and subtracting the DC attribute signal from the attribute signals of the voxels contained in each voxel block to obtain a residual attribute signal; if the input quality layer is an enhanced layer, subtracting the inter-layer prediction signal from the current layer attribute signal to obtain a prediction residual of the current layer attribute signal;

[0160] S24 DC attribute signal coding, which operates only on the base layer, uniformly quantizes the DC attribute signal of the base layer, subtracts the quantized DC attribute signal of each voxel block from that of the adjacent voxel block, and entropy codes it to obtain a DC attribute signal code stream;

[0161] S25 residual attribute signal coding, using the graph dictionary and the geometric coordinates of the current layer to perform geometric adaptive sparse coding on the residual attribute signal of each cubic block, obtain a sparse coefficient matrix, uniformly quantize the sparse coefficient matrix, and entropy code the values ​​and positions of the non-zero elements in the quantized sparse coefficient matrix to obtain a residual attribute signal code stream;

[0162] S26: merging the point cloud signal code streams, merging the geometric information code stream and the attribute signal code stream of the l′th quality layer to obtain the point cloud signal code stream of the l′th quality layer.

[0163] Furthermore, the process of S22 inter-layer attribute signal prediction is as follows:

[0164] S221 determines the quality layer of the input, and if it is the basic layer, no operation is performed;

[0165] If it is an enhanced layer, S222 uses geometric coordinates to divide the boundary cube where the point cloud signal of the current layer is located into evenly distributed c×c×c voxel blocks, and predicts the attribute signals of the voxels contained in each voxel block;

[0166] S223 calculates the spatial distances between the voxels contained in each voxel block and the 19 voxel blocks connected to the voxel block in the previous layer of point cloud signals, selects the 7 connected voxel blocks with the closest spatial distances, and uses the inverse of their spatial distances as weights to perform weighted averaging on the attribute signals of the 7 connected voxel blocks to obtain the inter-layer prediction signals of the voxels contained in the voxel block.

[0167] In this embodiment, c is preferably set to 2 to reduce the prediction residual between adjacent quality layers and improve compression performance.

[0168] Specifically, the specific operation of S25 residual attribute signal encoding is:

[0169] S251 uses the geometric coordinates of the voxels contained in each voxel block to extract the corresponding rows from the graph dictionary to form a sub-dictionary;

[0170] S252 normalizes the sub-dictionary corresponding to each extracted voxel block;

[0171] S253 uses the normalized sub-dictionary to calculate the residual attribute signal of the voxels contained in each voxel block based on Regularize the sparse terms to perform geometric adaptive sparse coding and obtain a sparse coefficient matrix:

[0172]

[0173] Here, you can use the commonly used The norm-constrained sparse coding method is used for solving, including but not limited to matching pursuit (MP), orthogonal matching pursuit (OMP), and iterative hard threshold method (IHT); in this embodiment, the solution method adopts the orthogonal matching pursuit method. Of course, in other embodiments, other solution methods can also be selected, and the present invention is not limited to the above solution methods;

[0174] S254 uniformly quantizes the sparse coefficient matrix, entropy codes the values ​​and positions (row coordinates and column coordinates) of the non-zero elements of the quantized sparse coefficient matrix respectively, and combines them to obtain a residual attribute signal code stream.

[0175] like Figure 3 As shown, in a preferred embodiment, S3 single quality layer point cloud signal decoding is performed to obtain the reconstructed point cloud signal of the quality layer, including: splitting the point cloud signal code stream of the l′th quality layer to obtain the geometric information code stream and the attribute signal code stream of the l′th quality layer; decoding the geometric coordinates from the geometric information code stream of the l′th quality layer; using the geometric coordinates of the l′th quality layer and the graph dictionary to decode the attribute signal code stream of the l′th quality layer to obtain the reconstructed attribute signal of the l′th quality layer; merging the geometric coordinates and the reconstructed attribute signal of the l′th quality layer as the reconstructed point cloud signal of the l′th quality layer, which is used for encoding and decoding the point cloud signal of the l′+1th quality layer.

[0176] Specifically, it includes: point cloud signal code stream splitting, geometric information decoding, inter-layer attribute signal prediction, DC attribute signal decoding, residual attribute signal decoding and 3D point cloud reconstruction.

[0177] S31 splits the point cloud signal code stream, splitting the point cloud signal code stream of the l′th quality layer to obtain the geometric information code stream and the attribute signal code stream of the l′th quality layer;

[0178] S32 geometric information decoding, obtain the octree structure and the byte unit of each branch node from the geometric information code stream entropy decoding, and then obtain the geometric coordinates of each voxel;

[0179] S33 inter-layer attribute signal prediction, same as above, obtains inter-layer prediction signal from the reconstructed point cloud signal of the previous quality layer;

[0180] S34 DC attribute signal decoding, only operates on the basic layer, performs entropy decoding on the basic layer DC attribute signal code stream, adds the signal values ​​of adjacent sequences obtained by decoding in sequence, and then performs inverse uniform quantization to obtain a reconstructed DC attribute signal;

[0181] S35 residual attribute signal decoding, entropy decoding and inverse quantization of the residual attribute signal code stream to obtain a reconstructed sparse coefficient matrix; using the geometric coordinates of the voxels contained in each voxel block, extract the corresponding row composition sub-dictionary from the graph dictionary and normalize it; multiply the normalized sub-dictionary and the sparse vector corresponding to the voxel block in the reconstructed coefficient matrix to obtain the reconstructed residual attribute signal of the voxels contained in the corresponding voxel block;

[0182] S36 three-dimensional point cloud reconstruction determines the quality layer of the input. If it is a basic layer, the basic layer DC attribute signal and the basic layer residual attribute signal are added to obtain the reconstructed attribute signal of the basic layer; if it is an enhanced layer, the inter-layer prediction signal and the enhanced layer residual attribute signal are added to obtain the reconstructed attribute signal of the enhanced layer; the reconstructed attribute signal and the geometric coordinates are aligned to obtain the reconstructed point cloud signal of the current layer, which is used for the single quality layer point cloud signal encoding and point cloud signal decoding of the next enhanced layer.

[0183] There are l quality layers for the point cloud data to be encoded, which are the basic layer, enhanced layer 1, ..., enhanced layer l-1 from low to high quality. The lowest quality reconstruction result of the 3D point cloud signal can be obtained by decoding the point cloud signal code stream of the basic layer. After that, each enhancement layer can be introduced from low to high to obtain a higher quality reconstruction result of the 3D point cloud signal, until all l quality layers are used to obtain the highest quality reconstruction result of the 3D point cloud signal. In practical applications, according to the transmission bit rate limit, according to the bit rate of each quality layer, the first l′ quality layers can be selected for transmission and decoding to provide the best reconstruction quality under the transmission bit rate.

[0184] Based on the above inventive concept, an embodiment of the present invention further provides a 3D point cloud decoding method based on graph dictionary learning, which extracts the first l' quality layers of point cloud signal code streams from the 3D point cloud signal code stream, where l' is any positive integer not less than l, and decodes the l' quality layers of point cloud signal code streams according to the graph dictionary to obtain a reconstructed 3D point cloud signal with different qualities, including:

[0185] S01 intercepts the first l′ layers (l′≤l) of the three-dimensional point cloud signal code stream, which correspond to the basic layer, the first enhancement layer, ..., the l′-1th enhancement layer in order according to the number of layers;

[0186] S02 decodes single quality layer point cloud signals of l′ quality layers in sequence starting from the basic layer to obtain reconstructed three-dimensional point cloud signals corresponding to the first l′ (l′≤l) layers.

[0187] The technologies used in the modules of the above system can refer to the implementation technologies corresponding to the steps in the above-mentioned embodiment of a three-dimensional point cloud encoding method based on graph dictionary learning, and will not be repeated here.

[0188] Based on the same inventive concept, Figure 1 As shown, an embodiment of the present invention further provides a three-dimensional point cloud compression method based on graph dictionary learning, including:

[0189] S001 obtains point cloud data to be encoded, wherein the point cloud data to be encoded is a three-dimensional point cloud signal;

[0190] S002 adopts the three-dimensional point cloud signal encoding method based on graph dictionary learning in the above embodiment to encode the three-dimensional point cloud signal into a three-dimensional point cloud signal code stream;

[0191] S003 adopts the three-dimensional point cloud signal decoding method based on graph dictionary learning in the above embodiment to decode and reconstruct the three-dimensional point cloud signal from the three-dimensional point cloud signal code stream.

[0192] The techniques used in each step of the above compression method can refer to the implementation techniques corresponding to the steps in the above-mentioned three-dimensional point cloud graph dictionary learning method and the three-dimensional point cloud encoding and decoding method embodiment based on graph dictionary learning, and will not be repeated here.

[0193] Based on the same inventive concept, an embodiment of the present invention further provides a three-dimensional point cloud compression device based on graph dictionary learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; the processor may execute any of the following methods when executing the program:

[0194] The above three-dimensional point cloud dictionary learning method, or,

[0195] The above three-dimensional point cloud signal encoding method based on graph dictionary learning, or,

[0196] The above three-dimensional point cloud signal decoding method based on graph dictionary learning, or,

[0197] The above is a three-dimensional point cloud compression system based on graph dictionary learning.

[0198] Parts not particularly described in the above embodiments of the present invention may be implemented using existing technologies.

[0199] Based on the three-dimensional point cloud compression system of the above embodiment, the following is explained in combination with a specific application embodiment: The key parameters in this embodiment are set as follows: According to the general test conditions of MPEG PCC point cloud compression, the test point cloud data used in the experiment includes 5 half-body target point cloud data with a geometric resolution of 512×512×512 from the MUVB data set. The training data is obtained from a large number of data sets, and the training data and the test data set do not overlap. Since the human eye is more sensitive to changes in the brightness information of the image, the original point cloud is converted from the RGB color space to the YUV color space. The number of sub-dictionaries is set to S=2, the training parameter p corresponding to each sub-dictionary is set to 1.2 and 2.1, the regularization parameters β, γ and λ are set to 10, 2 and 60 respectively, and the initial dictionary is the graph Fourier basis of the corresponding graph structure. The graph dictionary learning method performs 20 iteration cycles on the training data set, and each cycle is 500 iterations of the alternating direction multiplier method.

[0200] The system of the above embodiment of the present invention is used to compress the test point cloud data, and the average BD-PSNR and BD-Rate are calculated as measurement indicators of compression performance. The larger the BD-PSNR, the better the reconstruction quality of the compression system, and the smaller the BD-Rate, the more bit rate the compression system saves.

[0201] Compared with the method proposed by RL de Queiroz (TIP2016), the system of this embodiment obtained an average BD-PSNR gain of 4.50dB on all test data, and the average BD-Rate was -59.04%, which means that the method saved 59.04% of the bit rate; compared with the method proposed by S.Gu (TIP2020), the system of this embodiment obtained an average BD-PSNR gain of 1.99dB on all test data, and the average BD-Rate was -28.14%, which means that the method saved 28.14% of the bit rate; compared with the method proposed by Y.Shen (TCSVT2020), the system of this embodiment obtained an average BD-PSNR gain of 2.22dB on all test data, and the average BD-Rate was -36.62%, which means that the method saved 36.62% of the bit rate; compared with the current best MPEG Compared with the 3DG international point cloud compression standard G-PCC, the average BD-PSNR gain obtained by the system of this embodiment on all test data is 0.14dB, and the average BD-Rate is -3.88%, which means that this method saves 3.88% of the bit rate.

[0202] Experiments show that the compression efficiency of the above-mentioned embodiment system of the present invention is significantly better than the method proposed by RL de Queiroz, S. Gu and Y. Shen. Compared with the international point cloud compression standard, it can significantly improve the reconstruction performance and save the encoding bit rate.

[0203] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A graph dictionary learning method for three-dimensional point clouds, characterized in that: include: Obtain N training set point cloud data; Performing voxelization processing on the N training set point cloud data to obtain voxelized training set point cloud data; Dividing the voxelized training set point cloud data into voxel blocks, selecting a number of voxel blocks as training sets for graph dictionary learning, constructing a graph dictionary learning model based on the training sets, and establishing a graph dictionary learning objective function; Iteratively optimizing the objective function of the graph dictionary learning to obtain a graph dictionary for encoding and decoding of three-dimensional point cloud signals; The step of dividing the voxelized training set point cloud data into voxel blocks and selecting a number of voxel blocks as training sets for graph dictionary learning includes: The bounding cube where all the voxelized training set point cloud data are located is evenly divided into m×m×m voxel blocks, where m is a preset side length; All voxel blocks of each training set point cloud data are sorted in descending order according to the number of voxels they contain; Select the first r voxel blocks containing the largest number of voxels from each training set point cloud data, calculate the average value of the attribute signals of the voxels contained in each voxel block in the first r voxel blocks, and use it as the DC attribute signal of the voxel block, where r is a preset positive integer; The DC attribute signal is subtracted from each voxel to obtain a residual attribute signal as a training signal to form a training set for graph dictionary learning.

2. A method for learning a three-dimensional point cloud dictionary according to claim 1, characterized in that: The N training set point cloud data are voxelized to obtain voxelized training set point cloud data, including: Divide the point cloud data into regular voxels aligned with the spatial coordinate system; The geometric coordinates and attribute signals of each voxel are calculated to obtain voxelized point cloud data.

3. The method for learning a three-dimensional point cloud dictionary according to claim 1, characterized in that: The step of constructing a graph dictionary learning model based on the training set and establishing a graph dictionary learning objective function includes: The training set signal is used as a graph signal to construct a graph topology connection structure Where V represents m 3 A set of nodes, ε represents the connection m 3 The set of edges of nodes, Represents the edge weight matrix; The graph dictionary learning model is constructed by using the characteristic basis function of the p-Lap-Pull operator that constructs the graph topological connection structure, and the graph dictionary learning objective function is established: in, represents the i-th training collective pixel block, is a mask matrix whose elements are either 0 or 1 and is used to transform x i Extract the corresponding m i Individual elements, Represents a complete sub-dictionary consisting of S The overcomplete graph dictionary composed of S ], Used to constrain the corresponding complete sub-dictionary D s p s The characteristic basis of the Laplap operator, w ij represents the element in the i-th row and j-th column of the weight matrix W, d sk represents the kth atom of the sth sub-dictionary, d ski Indicates d sk The i-th element of represents a sparse coefficient matrix, The column vector of the sparse coefficient matrix A represents the signal x i The sparse coefficients on the graph dictionary D, represents the identity matrix, n = Nr is the number of training blocks, p s represents the power parameter corresponding to the s-th sub-dictionary, and β, γ, and λ represent the corresponding regularization parameters.

4. The method for learning a three-dimensional point cloud dictionary according to claim 3, characterized in that: Iteratively optimizing the objective function of the graph dictionary learning model to obtain a graph dictionary for encoding and decoding of three-dimensional point cloud signals, including: The graph dictionary learning objective function is iteratively optimized to obtain a graph dictionary, wherein in each iteration, the dictionary D is fixed to solve the geometric adaptive sparse coding subproblem and the sparse coefficient matrix A is fixed to solve the graph dictionary update subproblem, until a predetermined number of iterations are reached, to obtain a graph dictionary for encoding and decoding of three-dimensional point cloud signals.

5. The method for learning a three-dimensional point cloud dictionary according to claim 4, characterized in that: The graph dictionary learning objective function is iteratively optimized to obtain a graph dictionary, including: According to the fixed graph dictionary D obtained by solving the graph dictionary update subproblem, for each training set of pixels, the geometric adaptive sparse coding subproblem is solved one by one to obtain the sparse coefficient matrix A: According to the fixed sparse coefficient matrix A obtained by solving the geometric adaptive sparse coding subproblem, the graph dictionary update subproblem is solved using the alternating direction multiplier method to obtain the graph dictionary D: in, represents the Hadamard product of element-wise multiplication, represents the set of training set signals, Represents the set of mask matrices for all training set signals, Represents a vector whose elements are all 1.

6. A method for learning a three-dimensional point cloud graph dictionary according to claim 5, characterized in that: The alternating direction multiplier method comprises: By introducing an auxiliary variable Z instead of D, the non-convex second and third terms in the dictionary update optimization subproblem are separated from the convex first term and transformed into a constrained optimization problem: Where Z=[Z1,…,Z S ],Z s =D s ,s=1,…,S represents the sub-auxiliary variable corresponding to each dictionary; Introducing a dual variable Transforming the constrained optimization problem into its augmented Lagrangian function, we obtain three sub-optimization problems: U + =U+D-Z Among them, ρ represents the introduced penalty parameter; The three sub-optimization problems are solved alternately to update the three variables alternately until convergence, and a graph dictionary is obtained.

7. A three-dimensional point cloud signal encoding method based on graph dictionary learning, characterized in that: According to the graph dictionary obtained by the method described in any one of claims 1 to 6, the three-dimensional point cloud signal is encoded to generate a three-dimensional point cloud signal code stream.

8. The three-dimensional point cloud signal encoding method based on graph dictionary learning according to claim 7 is characterized in that: The step of encoding the three-dimensional point cloud signal to generate a three-dimensional point cloud signal code stream includes: Dividing the three-dimensional point cloud signal into quality layers to obtain l quality layers; Encode the point cloud signal of a single quality layer to obtain the point cloud signal code stream of the quality layer; Decode the point cloud signal of a single quality layer to obtain the reconstructed point cloud signal of the quality layer; Encode and decode l quality layers in turn, merge l quality layer point cloud signal code streams, and obtain a three-dimensional point cloud signal code stream.

9. The three-dimensional point cloud signal encoding method based on graph dictionary learning according to claim 8, characterized in that: The step of dividing the three-dimensional point cloud signal into quality layers to obtain l quality layers includes: The three-dimensional point cloud data is divided into l quality layers, and the first layer is the basic layer, and the l-1th layers are enhanced layers, namely the 1st enhanced layer to the l-1th enhanced layer, and the point cloud signals of the basic layer and each of the enhanced layers are obtained respectively.

10. The three-dimensional point cloud signal encoding method based on graph dictionary learning according to claim 9, characterized in that: The step of obtaining the point cloud signals of the base layer and each enhancement layer includes: Voxelize point cloud data and output voxelized point cloud signals as point cloud signals of the highest quality layer; Performing l-1 voxel block merging on the voxelized point cloud signal to obtain l-1 point cloud signals of different quality layers as point cloud signals of the basic layer and the first to l-2th enhancement layers; The input of each voxel block merger is the point cloud signal of the high-quality layer obtained by the previous voxel block merger, and the point cloud signal of the low-quality layer is output as the input of the next voxel block merger.

11. The three-dimensional point cloud signal encoding method based on graph dictionary learning according to claim 10, characterized in that: The voxel block merging comprises: Divide the boundary cube where the point cloud signal of the high-quality layer is located into evenly distributed c×c×c voxel blocks; Merging the voxels contained in each voxel block as a voxel of the point cloud signal of the low-quality layer; The maximum value of the geometric coordinates of the voxels contained in each voxel block is reduced by c times and rounded off to an integer as the geometric coordinates of the corresponding voxels in the point cloud signal of the low-quality layer; Taking the average value of the attribute signals of the voxels contained in each voxel block as the value of the attribute signal of the corresponding voxel in the point cloud signal of the low-quality layer; A point cloud signal of the low-quality layer is obtained.

12. The three-dimensional point cloud signal encoding method based on graph dictionary learning according to claim 8, characterized in that: The single quality layer point cloud signal encoding to obtain the point cloud signal code stream of the quality layer includes: Geometric information encoding: the geometric coordinates of the point cloud signal are divided according to the octree structure, and an eight-bit byte unit is allocated to each branch node in the octree structure to indicate whether there is point cloud point data in the spatial unit corresponding to the child node of the branch node. The octree structure is traversed in depth-breadth priority order to obtain a byte unit sequence representing the geometric coordinates of all spatial units, and the geometric information code stream is obtained by entropy encoding the byte unit sequence; Inter-layer attribute signal prediction: for the enhancement layer only, the geometric coordinates of the decoded previous layer reconstructed point cloud signal and the current layer point cloud signal are used to obtain the inter-layer prediction signal of the current layer attribute signal for attribute signal preprocessing; Attribute signal preprocessing: divide the boundary cube where the point cloud signal of the current layer is located into m×m×, voxel blocks evenly according to the preset side length m; if the input quality layer is the basic layer, average the values ​​of the attribute signals of the voxels contained in each voxel block to obtain the DC attribute signal, and subtract the DC attribute signal from the attribute signals of the voxels contained in each voxel block to obtain the residual attribute signal; if the input quality layer is the enhancement layer, subtract the inter-layer prediction signal from the attribute signal of the current layer to obtain the residual attribute signal of the current layer; DC attribute signal encoding: uniformly quantize only the DC attribute signal of the base layer, make a difference between the quantized DC attribute signal of each voxel block and the adjacent voxel block, and entropy encode to obtain the DC attribute signal code stream; Residual attribute signal encoding, using the graph dictionary and the geometric coordinates of the current layer to perform geometric adaptive sparse encoding on the residual attribute signal of each cubic block to obtain a sparse coefficient matrix, uniformly quantizing the sparse coefficient matrix, and entropy encoding the values ​​and positions of non-zero elements in the quantized sparse coefficient matrix to obtain a residual attribute signal code stream; The point cloud signal code stream is merged, and the geometric information code stream and the attribute signal code stream of the l'th quality layer are merged to obtain the point cloud signal code stream of the l'th quality layer.

13. The three-dimensional point cloud signal encoding method based on graph dictionary learning according to claim 12, characterized in that: The inter-layer attribute signal prediction includes: Determine the quality layer of the input, if it is the basic layer, no operation; If it is an enhanced layer, the bounding cube where the point cloud signal of the current layer is located is divided into evenly distributed c×c×c voxel blocks using geometric coordinates, and the attribute signals of the voxels contained in each voxel block are predicted; The geometric coordinates of each voxel of the previous layer point cloud signal are enlarged by c times and rounded to the nearest integer. The voxel of the previous layer point cloud signal corresponding to the 19 connected voxel blocks with the closest spatial distance to each voxel block is found, including itself. The spatial distance between the voxel contained in each voxel block and the 19 voxels connected to the voxel block in the previous layer point cloud signal is calculated one by one. The 7 connected voxels with the closest spatial distance are selected, and the attribute signals of the 7 connected voxels are weighted averaged with the inverse of the spatial distance of the 7 connected voxels as the weight to obtain the inter-layer prediction signal of the voxel contained in the voxel block.

14. The three-dimensional point cloud signal encoding method based on graph dictionary learning according to claim 12, characterized in that: The residual attribute signal encoding comprises: Using the geometric coordinates of the voxels contained in each voxel block, the corresponding rows are extracted from the graph dictionary to form a sub-dictionary; Normalizing the sub-dictionary corresponding to each extracted voxel block; Using the normalized sub-dictionary, the residual attribute signals of the voxels contained in each voxel block are subjected to geometric adaptive sparse coding based on the l0 regularized sparse term to obtain a sparse coefficient matrix: The sparse coefficient matrix is ​​uniformly quantized, and the values ​​and positions of the non-zero elements of the quantized sparse coefficient matrix, that is, the row coordinates and column coordinates, are entropy encoded respectively, and combined to obtain the residual attribute signal code stream.

15. The three-dimensional point cloud signal encoding method based on graph dictionary learning according to claim 8, characterized in that: The decoding of the single quality layer point cloud signal to obtain the reconstructed point cloud signal of the quality layer includes: Splitting the point cloud signal code stream, splitting the point cloud signal code stream of the l'th quality layer to obtain the geometric information code stream and the attribute signal code stream of the l'th quality layer; Geometric information decoding, obtaining an octree structure and a byte unit of each branch node from the geometric information code stream entropy decoding, and then obtaining the geometric coordinates of each voxel; Inter-layer attribute signal prediction, obtaining the inter-layer prediction signal from the reconstructed point cloud signal of the previous quality layer; DC attribute signal decoding: only entropy decoding is performed on the DC attribute signal bit stream of the base layer. For each voxel block, the decoded DC attribute signal of the adjacent voxel block is added, and inverse uniform quantization is performed to obtain a reconstructed DC attribute signal. Residual attribute signal decoding, entropy decoding and inverse quantization of the residual attribute signal code stream to obtain a reconstructed sparse coefficient matrix; using the geometric coordinates of the voxels contained in each voxel block, extract the corresponding row composition sub-dictionary from the graph dictionary and normalize it; multiply the normalized sub-dictionary and the sparse vector corresponding to the voxel block in the reconstructed coefficient matrix to obtain the reconstructed residual attribute signal of the voxels contained in the corresponding voxel block; Three-dimensional point cloud reconstruction, determine the quality layer of the input, if it is a basic layer, add the basic layer DC attribute signal and the basic layer residual attribute signal to obtain the reconstructed attribute signal of the basic layer; if it is an enhanced layer, add the inter-layer prediction signal and the enhanced layer residual attribute signal to obtain the reconstructed attribute signal of the enhanced layer; align the reconstructed attribute signal and the geometric coordinates to obtain the reconstructed point cloud signal of the current layer, which is used for single quality layer point cloud signal encoding and point cloud signal decoding of the next enhanced layer.

16. A three-dimensional point cloud signal decoding method based on graph dictionary learning, used for decoding a three-dimensional point cloud signal code stream obtained by the three-dimensional point cloud signal encoding method based on graph dictionary learning according to any one of claims 7 to 15, characterized in that: include: Extract the point cloud signal bitstreams of the first l' quality layers from the 3D point cloud signal bitstream, where l' is any positive integer not less than l, and corresponds to the base layer, the first enhancement layer, ..., the l'-1th enhancement layer in order of the number of layers; According to the graph dictionary, single quality layer point cloud signals are decoded for l' quality layers in sequence starting from the basic layer to obtain reconstructed three-dimensional point cloud signals corresponding to the first l' layers, where l'≤l.

17. A three-dimensional point cloud compression method based on graph dictionary learning, characterized in that: include: Obtaining point cloud data to be encoded, wherein the point cloud data to be encoded is a three-dimensional point cloud signal; Adopting the three-dimensional point cloud signal encoding method based on graph dictionary learning as described in any one of claims 7 to 15, encoding the three-dimensional point cloud signal into a three-dimensional point cloud signal code stream; The three-dimensional point cloud signal decoding method based on graph dictionary learning as described in claim 16 is adopted to decode and reconstruct the three-dimensional point cloud signal from the three-dimensional point cloud signal code stream.

18. A three-dimensional point cloud processing device based on graph dictionary learning, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor; wherein the processor executes any one of the following methods when executing the program: - The three-dimensional point cloud dictionary learning method according to any one of claims 1 to 6, or, - the three-dimensional point cloud signal encoding method based on graph dictionary learning as described in any one of claims 7 to 15, or, - The three-dimensional point cloud signal decoding method based on graph dictionary learning as described in claim 16, or, -The three-dimensional point cloud compression method based on graph dictionary learning as described in claim 17.

Images