Geometric point cloud compression method based on multi-scale multi-level Transform
The multi-scale and multi-level Transformer-based point cloud compression method addresses the loss of local detail information by enhancing feature extraction and representation, resulting in improved reconstruction quality and efficiency.
Patent Information
- Application Number
- CN202411168398.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-17
AI Technical Summary
Existing point cloud compression methods fail to effectively extract local features from multiple scales, leading to loss of local detail information and limiting the ability to capture local context in point clouds, while also being computationally inefficient.
A geometric point cloud compression method utilizing a multi-scale and multi-level Transformer architecture, comprising a feature encoder with multi-scale local and global feature extraction modules, and a feature decoder, trained with a rate-distortion optimization loss function to enhance feature representation and retention of spatial details.
The method improves reconstruction quality and retains more spatial details in point clouds with low computational cost, achieving better compression efficiency compared to existing methods.
Smart Images
Figure CN119152050A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of point cloud compression technology and mainly discusses the compression technology of geometric point clouds. Background Art
[0002] Point clouds are three-dimensional point sets generated by lidar to represent the geometric coordinates and related attributes (such as color and normal vectors) of static and dynamic three-dimensional objects. Due to the huge amount of data in point clouds, compression is required in fields such as cultural heritage, VR / AR, and autonomous driving. However, how to compress sparse, unstructured three-dimensional point clouds is a challenging problem. In sparse point clouds, there may be fewer points in a local area, which makes it difficult for feature extraction algorithms to accurately capture local feature information. Therefore, accurate extraction of local features is crucial in the point cloud compression process. The core goal of point cloud geometry compression is to reduce the number of points while maintaining the original geometric information and quality of the point cloud as much as possible.
[0003] Deep learning-based methods can be divided into two categories: voxel-based point cloud compression and point-based point cloud compression. Voxel-based point cloud compression methods usually convert unstructured point clouds into voxels with a regular volume grid structure, and then use a three-dimensional convolutional neural network to extract the intrinsic features of the point cloud. However, these voxel-based methods ignore the sparsity of point clouds, and inevitably lead to the loss of structural details of point clouds during the voxel conversion process. In addition, the required computational effort and memory usage increase exponentially with the increase of voxel size. Point-based point cloud compression methods do not require any voxelization and directly use points as input to the feature extraction network. Recently, Transformer has made some preliminary attempts in point cloud compression. The self-attention mechanism of Transformer is consistent with the permutation invariance of point clouds and can capture long-range dependencies from irregular and disordered points.
[0004] Existing point-based point cloud compression methods always directly extract single-scale features from the original input point cloud, without extracting local features of the point cloud from multiple scales, resulting in the loss of local detail information of the point cloud, thereby limiting the Transformer from learning local context-related representations in the point cloud and failing to efficiently capture the local context of the point cloud. Summary of the invention
[0005] The main technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a geometric point cloud compression method based on a multi-scale and multi-level Transformer with good reconstruction quality, high compression efficiency and low computational cost.
[0006] The technical solution adopted to solve the above technical problems consists of the following steps:
[0007] (1) Dataset preprocessing
[0008] Select the ShapeNetCorev2 point cloud dataset and divide it into a training set and a test set at a ratio of 4:1.
[0009] (2) Construct a point cloud geometric compression network
[0010] The point cloud geometric compression network is composed of a feature encoder, an entropy bottleneck layer, and a feature decoder connected in series in sequence.
[0011] The described feature encoder is composed of encoding block 1 and encoding block 2 connected in series.
[0012] The described encoding block 1 is composed of a multi-scale local feature extraction module and a global feature extraction module connected in parallel.
[0013] The described encoding block 2 is composed of a multi-scale Transformer module and a multi-level Transformer module connected in series.
[0014] The described feature decoder is composed of decoding block 1 and decoding block 2 connected in series.
[0015] (3) Train the point cloud geometric compression network
[0016] 1) Construct a rate-distortion optimization loss function
[0017] Construct a rate-distortion optimization loss function according to the following formula:
[0018] J = R + λD
[0019]
[0020] Among them, R is the bit rate, the value range of λ is 50 to 1000, D is the chamfer distance, S 1 is the original point cloud, S 2 is the reconstructed point cloud, N 1 represents the total number of points of the point cloud S 1 , N 2 represents the total number of points of the point cloud S 2 , N 1 , N 2 is a finite positive integer, x is the three-dimensional coordinate vector of any point in the original point cloud S 1 , x ∈ S 1 , y is the three-dimensional coordinate vector of any point in the reconstructed point cloud S 2 , y ∈ S 2 .
[0021] 2) Train the point cloud geometric compression network
[0022] The training set is input into the point cloud geometry compression network for training. The entropy bottleneck layer size is set to 256, the batch size is 32, the initial learning rate is 0.001, the training cycle is 200, and the training is carried out until the loss function converges.
[0023] (4) Testing point cloud geometry compression network
[0024] The test set is input into the trained point cloud geometry compression network to obtain the compressed output point cloud.
[0025] In step (2) of constructing a point cloud geometry compression network of the present invention, the multi-scale local feature extraction module is composed of a local feature extraction branch 1, a local feature extraction branch 2, and a local feature extraction branch 3 connected in parallel.
[0026] The local feature extraction branch 1 of the present invention is composed of a downsampling layer, a shared multi-layer perceptron, and a maximum pooling layer connected in series in sequence; the structures of the local feature extraction branch 2 and the local feature extraction branch 3 are the same as those of the local feature extraction branch 1;
[0027] The construction method of the local feature extraction branch 1 is as follows:
[0028] Determine the Euclidean distance d of neighboring points according to formula (1):
[0029]
[0030] Among them, n represents the number of points, and the value range of n is 32 to 80. k ,y k for any two points;
[0031] According to formula (2), the features of K neighboring points are concatenated and fused:
[0032]
[0033] Among them, f i is the fusion feature of the i-th point, It is the feature of the Kth neighboring point of the ith point, i ranges from 256 to 2048, K ranges from 32 to 80, concate is feature concatenation, and MLP is a shared multi-layer perceptron.
[0034] According to formula (3), the maximum pooling layer is determined as:
[0035]
[0036] Where F is the pooling window size, which is 2, and S is the step size of the window movement, which is 1;
[0037] The construction method of local feature extraction branch 2 and local feature extraction branch 3 is the same as the construction method of local feature extraction branch 1.
[0038] In (2) constructing a point cloud geometry compression network of the present invention, the global feature extraction module is composed of a shared multi-layer perceptron and a maximum pooling layer connected in series.
[0039] In (2) constructing a point cloud geometry compression network of the present invention, the multi-scale Transformer module is composed of an offset attention block 1, an offset attention block 2, an offset attention block 3, and an offset attention block 4 connected in series in sequence.
[0040] The construction method of the offset attention block 1 is as follows:
[0041] Determine the query matrix Q, key matrix K, and value matrix V as follows:
[0042] Q=W q ×F 0
[0043] K=W k ×F 0
[0044] V=W v ×F 0
[0045] Among them, W q , W k , W v is a linear layer, F 0 It is the output feature of the multi-scale feature extraction module;
[0046] Generate the attention map A according to formula (4):
[0047] A=(α i,j ) (4)
[0048]
[0049] Among them, M and C are the dimensions of A, M ranges from 1024 to 8192, and C ranges from 128 to 512. is the element in the i-th row and j-th column of A, i∈(1,M), j∈(1,C), and exp(·) is an exponential function with e as the base.
[0050] Offset attention output feature F out As follows:
[0051] F ou =MLP(F 0 -F a )+F 0
[0052] F a =A×V
[0053] Among them, MLP is a shared multi-layer perceptron, F 0 It is the output feature of the multi-scale feature extraction module.
[0054] The construction method of offset attention block 2, offset attention block 3, and offset attention block 4 is the same as that of offset attention block 1.
[0055] In (2) of the present invention, in constructing a point cloud geometry compression network, the method for constructing the multi-level Transformer module is as follows:
[0056] Construct a multi-level Transformer module as follows:
[0057] F mlt =concate(SA 1 , S.A. 2 , …, SA z )+F mst
[0058]
[0059] Q m =F mst ×W qm
[0060] K m =F mst ×W km
[0061] V m =F mst ×W vm
[0062] Among them, concate is the concatenation operation, F mlt is the output feature of the multi-level Transformer module, F mst is the output feature of the multi-scale Transformer module; softmax is a normalization operation, Z is the number of attention heads, Z ranges from 2 to 6, m is the mth head, m∈(1,Z), W qm、 W km , W vm is a linear layer.
[0063] Since the present invention uses a multi-scale local feature extraction module to extract the global local feature information of point clouds at multiple scales, and uses a global feature extraction module to extract geometric features, the network can effectively learn the local area information of point clouds at multiple scales, retaining rich local detail information; the multi-scale Transformer module is used to fuse features of multiple scales, enhancing the representation ability of features; and the multi-level Transformer is used to learn the cross-level feature relationships, further fusing the geometric and semantic information of point clouds. The comparative experimental results show that the method of the present invention improves the reconstruction quality of point clouds, retains more spatial details of point clouds, and has low computational cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] Figure 1 This is a flow chart of Example 1 of the present invention.
[0065] Figure 2 It is a structural diagram of the feature encoder.
[0066] Figure 3 It is a structural diagram of the feature decoder.
[0067] Figure 4 yes Figure 2 Schematic diagram of the structure of the multi-scale local feature extraction module.
[0068] Figure 5 yes Figure 2 Schematic diagram of the structure of the global feature extraction module.
[0069] Figure 6 yes Figure 2 Schematic diagram of the structure of the multi-scale Transformer module. DETAILED DESCRIPTION
[0070] The present invention is further described below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.
[0071] Example 1
[0072] The geometric point cloud compression method based on multi-scale and multi-level Transformer in this embodiment consists of the following steps (see Figure 1 ):
[0073] (1) Dataset preprocessing
[0074] Select the ShapeNetCorev2 point cloud dataset and divide it into training set and test set in a ratio of 4:1.
[0075] (2) Constructing a point cloud geometry compression network
[0076] The point cloud geometry compression network of this embodiment is composed of a feature encoder, an entropy bottleneck layer, and a feature decoder connected in series.
[0077] Figure 2 The structural diagram of the feature encoder of this embodiment is given. Figure 2 In the embodiment, the feature encoder is composed of encoding block 1 and encoding block 2 connected in series.
[0078] The encoding block 1 of this embodiment is composed of a multi-scale local feature extraction module and a global feature extraction module in parallel.
[0079] The encoding block 2 of this embodiment is composed of a multi-scale Transformer module and a multi-level Transformer module connected in series.
[0080] Figure 3 The feature decoder of this embodiment is composed of a decoding block 1 and a decoding block 2 connected in series.
[0081] Figure 4 Given Figure 2 Schematic diagram of the structure of the multi-scale local feature extraction module. Figure 4 In the embodiment, the multi-scale local feature extraction module is composed of a local feature extraction branch 1, a local feature extraction branch 2, and a local feature extraction branch 3 connected in parallel.
[0082] The local feature extraction branch 1 of this embodiment is composed of a downsampling layer, a shared multi-layer perceptron, and a maximum pooling layer connected in series. The structures of the local feature extraction branch 2 and the local feature extraction branch 3 are the same as those of the local feature extraction branch 1.
[0083] The construction method of the local feature extraction branch 1 of this embodiment is as follows:
[0084] Determine the Euclidean distance d of neighboring points according to formula (1):
[0085]
[0086] Where n represents the number of points, and the value range of n is 32 to 80. In this embodiment, the value of n is 64. k ,y k For any two points.
[0087] According to formula (2), the features of K neighboring points are concatenated and fused:
[0088]
[0089] Among them, f i is the fusion feature of the i-th point, i ranges from 256 to 2048, and the value of i in this embodiment is 896. is the feature of the Kth neighboring point of the ith point, the value range of K is 32 to 80, and the value of K in this embodiment is 64, concate is feature concatenation, and MLP is a shared multi-layer perceptron.
[0090] According to formula (3), the maximum pooling layer is determined as:
[0091]
[0092] Among them, F is the pooling window size, which is 2, and S is the step size of the window movement, which is 1.
[0093] The construction method of local feature extraction branch 2 and local feature extraction branch 3 is the same as the construction method of local feature extraction branch 1.
[0094] Figure 5 Given Figure 2 Schematic diagram of the structure of the global feature extraction module. Figure 5 In the present embodiment, the global feature extraction module is composed of a shared multi-layer perceptron and a maximum pooling layer connected in series.
[0095] Figure 6 Given Figure 2 Schematic diagram of the structure of the multi-scale Transformer module. Figure 6 In the embodiment, the multi-scale Transformer module is composed of offset attention block 1, offset attention block 2, offset attention block 3, and offset attention block 4 connected in series in sequence.
[0096] The construction method of the offset attention block 1 of this embodiment is as follows:
[0097] Determine the query matrix Q, key matrix K, and value matrix V as follows:
[0098] Q=W q ×F 0
[0099] K=W k ×F 0
[0100] V=W v ×F 0
[0101] Among them, W q , W k , W v is a linear layer, F 0 It is the output feature of the multi-scale feature extraction module.
[0102] Generate the attention map A according to formula (4):
[0103] A=(α i，j) (4)
[0104]
[0105] Wherein, M and C are the dimensions of A respectively, the value range of M is 1024-8192, and the value of M in this embodiment is 2048, and the value range of C is 128-512, and the value of C in this embodiment is 256. is the element in the i-th row and j-th column of A, i∈(1,M), j∈(1,C), and exp(·) is an exponential function with e as the base.
[0106] Offset attention output feature F out As follows:
[0107] F out＝ MLP(F 0 -F a )+F 0
[0108] F a =A×V
[0109] Among them, MLP is a shared multi-layer perceptron, F 0 It is the output feature of the multi-scale feature extraction module.
[0110] The construction method of offset attention block 2, offset attention block 3, and offset attention block 4 is the same as that of offset attention block 1.
[0111] The construction method of the multi-level Transformer module of this embodiment is as follows:
[0112] Determine the multi-level Transformer module as follows:
[0113] F mlt =concate(SA 1 , S.A. 2 , …, SA z )+F mst
[0114]
[0115] Q m =F mst ×W qm
[0116] K m =F mst ×W km
[0117] V m =F mst ×W vm
[0118] Among them, concate is the concatenation operation, F mlt is the output feature of the multi-level Transformer module, F mst is the output feature of the multi-scale Transformer module; softmax is a normalization operation, Z is the number of attention heads, the value range of Z is 2 to 6, and the value of Z in this embodiment is 4, m is the mth head, m∈(1,Z), W qm , W km , W vm is a linear layer.
[0119] (3) Training point cloud geometry compression network
[0120] 1) Construct rate-distortion optimization loss function
[0121] The rate-distortion optimization loss function is constructed as follows:
[0122] J=R+λD
[0123]
[0124] Wherein, R is the bit rate, λ is in the range of 50 to 1000, and is 500 in this embodiment, D is the chamfer distance, N is 1 Represents the point cloud S 1 The total number of points, N 2 Represents the point cloud S 2 The total number of points, S 1 is the original point cloud, S 2 is the reconstructed point cloud, N 1 、N 2 is a finite positive integer, x is the original point cloud S 1 The three-dimensional coordinate vector of any point in S, x∈S 1 , y is the reconstructed point cloud S 2 The three-dimensional coordinate vector of any point in , y∈S 2 .
[0125] 2) Training point cloud geometry compression network
[0126] The training set is input into the point cloud geometry compression network for training. The entropy bottleneck layer size is set to 256, the batch size is 32, the initial learning rate is 0.001, the training cycle is 200, and the training is carried out until the loss function converges.
[0127] (4) Testing point cloud geometry compression network
[0128] The test set is input into the trained point cloud geometry compression network to obtain the compressed output point cloud.
[0129] Complete the geometric point cloud compression method based on multi-scale and multi-level Transformer.
[0130] Example 2
[0131] The geometric point cloud compression method based on multi-scale and multi-level Transformer of this embodiment consists of the following steps:
[0132] (1) Dataset preprocessing
[0133] This step is the same as in Example 1.
[0134] (2) Constructing a point cloud geometry compression network
[0135] The structure of the point cloud geometry compression network is the same as that of Example 1.
[0136] The construction method of the local feature extraction branch 1 of this embodiment is as follows:
[0137] Determine the Euclidean distance d of neighboring points according to formula (1):
[0138]
[0139] Where n represents the number of points, and the value range of n is 32 to 80. In this embodiment, the value range of n is 32. k ,y k For any two points.
[0140] According to formula (2), the features of K neighboring points are concatenated and fused:
[0141]
[0142] Among them, f i is the fusion feature of the i-th point, i ranges from 256 to 2048, and in this embodiment, i is 256. is the feature of the Kth neighboring point of the ith point, the value range of K is 32 to 80, the value of K in this embodiment is 32, concate is feature concatenation, and MLP is a shared multi-layer perceptron.
[0143] According to formula (3), the maximum pooling layer is determined as:
[0144]
[0145] Among them, F is the pooling window size, which is 2, and S is the step size of the window movement, which is 1.
[0146] The construction method of local feature extraction branch 2 and local feature extraction branch 3 is the same as the construction method of local feature extraction branch 1.
[0147] The construction method of the offset attention block 1 of this embodiment is as follows:
[0148] Determine the query matrix Q, key matrix K, and value matrix V as follows:
[0149] Q=W q ×F 0
[0150] K=W k ×F 0
[0151] V=W v ×F 0
[0152] Among them, W q , W k , W v is a linear layer, F 0 It is the output feature of the multi-scale feature extraction module.
[0153] Generate the attention map A as follows:
[0154] A=(α i，j )
[0155]
[0156] Wherein, M and C are the dimensions of A respectively, the value range of M is 1024-8192, and the value of M in this embodiment is 1024, and the value range of C is 128-512, and the value of C in this embodiment is 128. is the element in the i-th row and j-th column of A, i∈(1,M), j∈(1,C), and exp(·) is an exponential function with e as the base.
[0157] Offset attention output feature F out As follows:
[0158] F out =MLP(F 0 -F a )+F 0
[0159] F a =A×V
[0160] Among them, MLP is a shared multi-layer perceptron, F 0 It is the output feature of the multi-scale feature extraction module.
[0161] The construction method of offset attention block 2, offset attention block 3, and offset attention block 4 is the same as that of offset attention block 1.
[0162] The construction method of the multi-level Transformer module of this embodiment is as follows:
[0163] Construct a multi-level Transformer module as follows:
[0164] F mlt =concate(SA 1 ,SA 2 , …, SA z )+F mst
[0165]
[0166] Q m =F mst ×W qm
[0167] K m =F mst ×W km
[0168] V m =F mst ×W vm
[0169] Among them, concate is the concatenation operation, F mlt is the output feature of the multi-level Transformer module, F mst is the output feature of the multi-scale Transformer module; softmax is a normalization operation, Z is the number of attention heads, the value range of Z is 2 to 6, and the value of Z in this embodiment is 2, m is the mth head, m∈(1,Z), W qm , W km , W vm is a linear layer.
[0170] (3) Training point cloud geometry compression network
[0171] 1) Construct rate-distortion optimization loss function
[0172] The expression of the rate-distortion optimization loss function is the same as that in Example 1.
[0173] In the expression of the rate-distortion optimization loss function, R is the bit rate, λ is in the range of 50 to 1000, and the value of λ in this embodiment is 50, D is the chamfer distance, S 1 is the original point cloud, S 2 is the reconstructed point cloud, N 1 Represents the point cloud S 1 The total number of points, N 2 Represents the point cloud S 2 The total number of points, N 1 、N 2 is a finite positive integer, x is the original point cloud S1 The three-dimensional coordinate vector of any point in S, x∈S 1 , y is the reconstructed point cloud S 2 The three-dimensional coordinate vector of any point in , y∈S 2 .
[0174] 2) Training point cloud geometry compression network
[0175] This step is the same as in Example 1.
[0176] (4) Testing point cloud geometry compression network
[0177] This step is the same as in Example 1.
[0178] Complete the geometric point cloud compression method based on multi-scale and multi-level Transformer.
[0179] Example 3
[0180] The geometric point cloud compression method based on multi-scale and multi-level Transformer of this embodiment consists of the following steps:
[0181] (1) Dataset preprocessing
[0182] This step is the same as in Example 1.
[0183] (2) Constructing a point cloud geometry compression network
[0184] The structure of the point cloud geometry compression network is the same as that of Example 1.
[0185] The construction method of the local feature extraction branch 1 of this embodiment is as follows:
[0186] Determine the Euclidean distance d of neighboring points according to formula (1):
[0187]
[0188] Where n represents the number of points, and the value range of n is 32 to 80. In this embodiment, the value range of n is 80. k ,y k For any two points.
[0189] According to formula (2), the features of K neighboring points are concatenated and fused:
[0190]
[0191] Among them, f i is the fusion feature of the i-th point, is the feature of the Kth neighboring point of the ith point, i ranges from 256 to 2048, and the value of i in this embodiment is 2048, K ranges from 32 to 80, and the value of k in this embodiment is 80, concate is feature concatenation, and MLP is a shared multi-layer perceptron.
[0192] According to formula (3), the maximum pooling layer is determined as:
[0193]
[0194] Among them, F is the pooling window size, which is 2, and S is the step size of the window movement, which is 1.
[0195] The construction method of local feature extraction branch 2 and local feature extraction branch 3 is the same as the construction method of local feature extraction branch 1.
[0196] The construction method of the offset attention block 1 of this embodiment is as follows:
[0197] Determine the query matrix Q, key matrix K, and value matrix V as follows:
[0198] Q=W q ×F 0
[0199] K=W k ×F 0
[0200] V=W v ×F 0
[0201] Among them, W q , W k , W v is a linear layer, F 0 It is the output feature of the multi-scale feature extraction module.
[0202] Generate the attention map A as follows:
[0203] A=(α i，j )
[0204]
[0205] Wherein, M and C are the dimensions of A respectively, the value range of M is 1024 to 8192, and the value of M in this embodiment is 8192, and the value range of C is 128 to 512, and the value of C in this embodiment is 512. is the element in the i-th row and j-th column of A, i∈(1,M), j∈(1,C), and exp(·) is an exponential function with e as the base.
[0206] Offset attention output feature F out As follows:
[0207] F out =MLP(F 0 -F a )+F 0
[0208] F a =A×V
[0209] Among them, MLP is a shared multi-layer perceptron, F 0 It is the output feature of the multi-scale feature extraction module.
[0210] The construction method of offset attention block 2, offset attention block 3, and offset attention block 4 is the same as that of offset attention block 1.
[0211] The construction method of the multi-level Transformer module of this embodiment is as follows:
[0212] Determine the multi-level Transformer module as follows:
[0213] F mlt =concate(SA 1 , S.A. 2 , …, SA z )+F mst
[0214]
[0215] Q m =F mst ×W qm
[0216] K m =F mst ×W km
[0217] V m =F mst ×W vm
[0218] Among them, concate is the concatenation operation, F mlt is the output feature of the multi-level Transformer module, F mst is the output feature of the multi-scale Transformer module; softmax is a normalization operation, Z is the number of attention heads, the value range of Z is 2 to 6, and the value of Z in this embodiment is 6, m is the mth head, m∈(1,Z), W qm , W km , W vm is a linear layer.
[0219] (3) Training point cloud geometry compression network
[0220] 1) Construct rate-distortion optimization loss function
[0221] The expression of the rate-distortion optimization loss function is the same as that in Example 1.
[0222] In the expression of the rate-distortion optimization loss function, R is the bit rate, λ is in the range of 50 to 1000, and in this embodiment, λ is 1000, D is the chamfer distance, S 1 is the original point cloud, S 2 is the reconstructed point cloud, N 1 Represents the point cloud S 1 The total number of points, N 2 Represents the point cloud S 2 The total number of points, N 1 、N 2 is a finite positive integer, x is the original point cloud S 1 The three-dimensional coordinate vector of any point in S, x∈S 1 , y is the reconstructed point cloud S 2 The three-dimensional coordinate vector of any point in , y∈S 2 .
[0223] 2) Training point cloud geometry compression network
[0224] This step is the same as in Example 1.
[0225] (4) Testing point cloud geometry compression network
[0226] This step is the same as in Example 1.
[0227] Complete the geometric point cloud compression method based on multi-scale and multi-level Transformer.
[0228] In order to verify the beneficial effects of the present invention, a comparative experiment was carried out using the method of Example 1 of the present invention and the existing point cloud geometry compression method based on deep learning (hereinafter referred to as Comparative Experiment 1), the point cloud geometry compression method based on neural graph sampling (hereinafter referred to as Comparative Experiment 2), and the point cloud geometry compression method based on Transformer and upsampling (hereinafter referred to as Comparative Experiment 3), and the bit rate gain BD-Rate and the bit rate-peak signal-to-noise ratio gain BD-PSNR were determined as comprehensive indicators for evaluating the point cloud compression performance.
[0229] The bit rate gain BD-Rate is determined as follows:
[0230]
[0231] Among them, a and b are the lower and upper limits of the PSNR range, respectively, and R 1 and R 2They are the bit rates of the two encoders at corresponding PSNR values.
[0232] The bit rate-peak signal-to-noise ratio gain BD-PSNR is determined by the following formula:
[0233]
[0234] Among them, the peak signal-to-noise ratio PSNR 1 and peak signal-to-noise ratio PSNR 2 They are the peak signal-to-noise ratio (PSNR) values of the two encoders at bit rate R.
[0235] The experimental and calculation results are shown in Table 1.
[0236] Table 1 Experimental results of the method of the present invention and the comparative experimental method
[0237] Experimental methods BD-Rate(%) BD-PSNR(dB) Comparative experiment 1 -56.78 1.95 Comparative experiment 2 -59.33 2.52 Comparative experiment 3 -62.35 2.86 Example 1 -64.59 3.34
[0238] As can be seen from Table 1, compared with the comparative experiment 1 method, the BD-Rate of the method of the present invention decreased by 7.81%, and the BD-PSNR increased by 1.39dB; compared with the comparative experiment 2 method, the BD-Rate decreased by 5.26%, and the BD-PSNR increased by 0.82dB; compared with the comparative experiment 3 method, the BD-Rate decreased by 2.24%, and the BD-PSNR increased by 0.48dB.
Claims
1. A geometric point cloud compression method based on multi-scale and multi-level Transformer, characterized in that It consists of the following steps: (1) Dataset preprocessing Select the ShapeNetCorev2 point cloud dataset and divide it into training set and test set at a ratio of 4:1; (2) Constructing a point cloud geometry compression network The point cloud geometry compression network is composed of a feature encoder, an entropy bottleneck layer, and a feature decoder connected in series. The feature encoder is composed of encoding block 1 and encoding block 2 in series; The encoding block 1 is composed of a multi-scale local feature extraction module and a global feature extraction module in parallel; The encoding block 2 is composed of a multi-scale Transformer module and a multi-level Transformer module connected in series; The feature decoder is composed of a decoding block 1 and a decoding block 2 connected in series; (3) Training point cloud geometry compression network 1) Construct rate-distortion optimization loss function The rate-distortion optimization loss function is constructed as follows: J=R+λD Where R is the bit rate, λ ranges from 50 to 1000, D is the chamfer distance, S1 is the original point cloud, S2 is the reconstructed point cloud, N1 represents the total number of points in point cloud S1, N2 represents the total number of points in point cloud S2, N1 and N2 are finite positive integers, x is the three-dimensional coordinate vector of any point in the original point cloud S1, x∈S1, y is the three-dimensional coordinate vector of any point in the reconstructed point cloud S2, y∈S2; 2) Training point cloud geometry compression network The training set is input into the point cloud geometry compression network for training. The entropy bottleneck layer size is set to 256, the batch size is 32, the initial learning rate is 0.001, the training cycle is 200, and the training is performed until the loss function converges. (4) Testing point cloud geometry compression network The test set is input into the trained point cloud geometry compression network to obtain the compressed output point cloud.
2. The geometric point cloud compression method based on multi-scale and multi-level Transformer according to claim 1, characterized in that: In step (2) of constructing the point cloud geometry compression network, the multi-scale local feature extraction module is composed of a local feature extraction branch 1, a local feature extraction branch 2, and a local feature extraction branch 3 connected in parallel.
3. The geometric point cloud compression method based on multi-scale and multi-level Transformer according to claim 2, characterized in that: The local feature extraction branch 1 is composed of a downsampling layer, a shared multi-layer perceptron, and a maximum pooling layer connected in series in sequence; the structures of the local feature extraction branch 2 and the local feature extraction branch 3 are the same as those of the local feature extraction branch 1; The construction method of the local feature extraction branch 1 is as follows: Determine the Euclidean distance d of neighboring points according to formula (1): Among them, n represents the number of points, and the value range of n is 32 to 80. k ,y k for any two points; According to formula (2), the features of K neighboring points are concatenated and fused: Among them, f i is the fusion feature of the i-th point, is the feature of the Kth neighbor of the i-th point, i ranges from 256 to 2048, K ranges from 32 to 80, concate is feature concatenation, and MLP is a shared multi-layer perceptron; According to formula (3), the maximum pooling layer is determined as: Among them, F is the pooling window size, the value is 2, S is the step size of the window movement, the value is 1; The construction method of local feature extraction branch 2 and local feature extraction branch 3 is the same as the construction method of local feature extraction branch 1.
4. The geometric point cloud compression method based on multi-scale and multi-level Transformer according to claim 1, characterized in that: In step (2) of constructing the point cloud geometry compression network, the global feature extraction module is composed of a shared multi-layer perceptron and a maximum pooling layer connected in series.
5. The geometric point cloud compression method based on multi-scale and multi-level Transformer according to claim 1, characterized in that: In step (2) of constructing a point cloud geometry compression network, the multi-scale Transformer module is composed of an offset attention block 1, an offset attention block 2, an offset attention block 3, and an offset attention block 4 connected in series in sequence. The construction method of the offset attention block 1 is as follows: Determine the query matrix Q, key matrix K, and value matrix V as follows: Q=W q ×F0 K=W k ×F0 V=W v ×F0 Among them, W q , W k , W v is a linear layer, and F0 is the output feature of the multi-scale feature extraction module; Generate the attention map A according to formula (4): A＝(α i，j ) (4) Among them, M and C are the dimensions of A respectively. The value range of M is 1024~8192, and the value range of C is 128~512. is the element in the i-th row and j-th column of A, i ∈ (1, M), j ∈ (1, C), exp(·) is an exponential function with base e; Offset attention output feature F out As follows: F out =MLP(F0-F a )+F0 F a =A×V Among them, MLP is a shared multi-layer perceptron, and F0 is the output feature of the multi-scale feature extraction module; The construction method of offset attention block 2, offset attention block 3, and offset attention block 4 is the same as that of offset attention block 1.
6. The geometric point cloud compression method based on multi-scale and multi-level Transformer according to claim 1, characterized in that: In step (2) of constructing the point cloud geometry compression network, the construction method of the multi-level Transformer module is as follows: Construct a multi-level Transformer module as follows: F mlt =concate(TO 1 ,IN 2 ,…,IN z )+F mst Q m =F mst ×W qm K m =F mst ×W km V m =F mst ×W vm Among them, concate is the concatenation operation, F mlt is the output feature of the multi-level Transformer module, F mst is the output feature of the multi-scale Transformer module; softmax is a normalization operation, Z is the number of attention heads, Z ranges from 2 to 6, m is the mth head, m∈(1,Z), W qm , W km , W vm is a linear layer.