Point cloud compression method, apparatus and device with variable bitrate, and storage medium

Abstract

The present application discloses a point cloud compression method, apparatus and device with a variable bitrate, and a storage medium. The point cloud compression method with a variable bitrate comprises: acquiring a point cloud to be compressed and a required bitrate; determining a scaling factor corresponding to the required bitrate; and on the basis of the point cloud to be compressed, by means of a preset point cloud compression model and a scale linear transformation module of the point cloud compression model, performing point cloud compression at a scale of the corresponding scaling factor to obtain a compressed point cloud meeting the required bitrate, wherein training configuration parameters of the point cloud compression model include scaling factors, the point cloud compression model comprises the scale linear transformation module, and the scale linear transformation module is used for realizing feature scaling of different scales. According to the present application, by means of the scale linear transformation module of the point cloud compression model, the point cloud to be compressed is scaled on the basis of the scaling factor to obtain the compressed point cloud meeting the bitrate required by a user, and only one point cloud compression model needs to be trained, thereby reducing the resource consumption.

Description

Point cloud compression method, device, equipment and storage medium with variable bit rate Technical Field

[0001] The present application relates to the technical field of point cloud data processing, and in particular to a point cloud compression method, apparatus, device, and storage medium with variable bit rate. Background Art

[0002] Three-dimensional point clouds are an important form of digital representation of the real world and are currently being widely used in fields such as autonomous driving and virtual reality. With the development of point cloud technology, the resolution of point clouds has gradually increased, leading to an increase in the amount of point cloud data. Therefore, how to compress point cloud data and reduce its size is an urgent problem that needs to be solved.

[0003] In related technologies, point cloud compression frameworks based on neural network structures are often used to achieve point cloud compression. These neural network point cloud compression frameworks use convolutional neural networks to form encoders and decoders, and then train the entire framework by jointly optimizing bit rate and distortion to achieve point cloud compression and restoration. However, this method can only achieve one compression bit rate for a trained neural network model. To achieve different compression rates, the method requires training multiple neural network models to meet the requirements of different scenarios. Training multiple neural network models consumes a large amount of computing resources and occupies a large amount of storage resources, resulting in high resource consumption.

[0004] Summary of the Invention

[0005] The main purpose of this application is to provide a point cloud compression method, device, equipment and storage medium with variable bit rate, aiming to solve the technical problem of high resource consumption in the existing technology.

[0006] To achieve the above objectives, the present application provides a point cloud compression method with variable bit rate, the point cloud compression method with variable bit rate comprising:

[0007] Get the point cloud to be compressed and the required bit rate;

[0008] Determining a scaling factor corresponding to the required bit rate;

[0009] Based on the point cloud to be compressed, point cloud compression of the scale corresponding to the scaling factor is performed through a preset point cloud compression model and a scale linear transformation module of the point cloud compression model to obtain a compressed point cloud that meets the required bit rate, wherein the training configuration parameters of the point cloud compression model include the scaling factor, and the point cloud compression model includes a scale linear transformation module, and the scale linear transformation module is used to achieve feature scaling of different scales.

[0010] Optionally, the step of performing point cloud compression on the point cloud to be compressed at a scale corresponding to the scaling factor using a preset point cloud compression model and a scale linear transformation module of the point cloud compression model to obtain a compressed point cloud meeting the required bit rate includes:

[0011] Based on the point cloud to be compressed, feature extraction is performed using a preset point cloud compression model to obtain point cloud features;

[0012] Based on the scale linear transformation module of the point cloud compression model, the point cloud features are compressed at a scale corresponding to the scaling factor to obtain a potential representation of the point cloud to be compressed;

[0013] Based on a preset first encoding module, the potential representation is compressed to obtain a compressed point cloud that meets the required bit rate.

[0014] Optionally, after the step of performing point cloud compression on the point cloud features at a scale corresponding to the scaling factor by the scale linear transformation module based on the point cloud compression model to obtain a potential representation of the point cloud to be compressed, the method includes:

[0015] Based on a preset super-prior network, parallel feature convolution is performed on the potential representation to obtain point cloud features after the first convolution;

[0016] Based on the scale linear transformation module of the point cloud compression model, the point cloud features after the first convolution are compressed at a scale corresponding to the scaling factor, and a first probability matrix of the potential representation is output, wherein the first probability matrix reflects the side information of the point cloud to be compressed.

[0017] Optionally, the step of compressing the potential representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate includes:

[0018] Based on a preset first encoding module, the potential representation and the first probability matrix are compressed to obtain a compressed point cloud that meets the required bit rate.

[0019] Optionally, after the step of performing point cloud compression on the first convolved point cloud features at a scale corresponding to the scaling factor by the scale linear transformation module based on the point cloud compression model and outputting the first probability matrix of the potential representation, the method includes:

[0020] Compressing the first probability matrix based on a preset second encoding module to obtain a compressed first probability matrix;

[0021] Based on a preset first decoding module, the compressed first probability matrix is ​​decompressed to obtain a decompressed first probability matrix, so as to transmit the first probability matrix to a preset first encoding module.

[0022] Optionally, the step of compressing the potential representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate includes:

[0023] Based on a preset first encoding module, the potential representation and the decompressed first probability matrix are compressed to obtain a compressed point cloud that meets the required bit rate.

[0024] Optionally, after the step of performing point cloud compression on the convolved point cloud features at a scale corresponding to the scaling factor by the scale linear transformation module based on the point cloud compression model and outputting the first probability matrix of the potential representation, the method includes:

[0025] Inputting the first probability matrix into a preset decomposition density entropy model, performing probability calculation on the first probability matrix based on the decomposition density entropy model and a preset number of entropy parameters of the decomposition density entropy model, and obtaining a corresponding number of probability matrix sets of the first probability matrices;

[0026] The transmission code rate value of each probability matrix under the probability matrix set is calculated, and the probability matrix corresponding to the minimum transmission code rate value is selected as the target probability matrix.

[0027] Optionally, the step of compressing the first probability matrix based on a preset second encoding module to obtain a compressed first probability matrix includes:

[0028] Based on a preset second encoding module, the first probability matrix and the target probability matrix are compressed to obtain a compressed first probability matrix.

[0029] Optionally, after the step of performing point cloud compression on the point cloud features at a scale corresponding to the scaling factor by the scale linear transformation module based on the point cloud compression model to obtain a potential representation of the point cloud to be compressed, the method includes:

[0030] Based on a preset channel context conversion network, performing serial iterative feature convolution on the potential representation to obtain a point cloud feature after the second convolution;

[0031] Based on the scale linear transformation module of the point cloud compression model, perform point cloud compression on the point cloud features after the second convolution at a scale corresponding to the scaling factor, and output a second probability matrix of the potential representation, wherein the second probability matrix reflects channel context information of the point cloud to be compressed;

[0032] Determining a first probability matrix of the latent representation based on a preset hyper-prior network, wherein the first probability matrix reflects side information of the point cloud to be compressed, and the channel context conversion network and the hyper-prior network process the latent representation synchronously;

[0033] Based on a preset aggregation module, the first probability matrix and the second probability matrix are aggregated to obtain a third probability matrix.

[0034] Optionally, the step of compressing the potential representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate includes:

[0035] Based on a preset first encoding module, the potential representation and the third probability matrix are compressed to obtain a compressed point cloud that meets the required bit rate.

[0036] Optionally, the aggregation module includes a hybrid reference mean aggregation module and a hybrid reference variance aggregation module, and the step of aggregating the first probability matrix and the second probability matrix based on the preset aggregation module to obtain a third probability matrix includes:

[0037] Based on a preset hybrid reference mean aggregation module, extracting a first mean of the first probability matrix and a second mean of the second probability matrix respectively, and aggregating the first mean and the second mean to obtain an aggregated target mean;

[0038] Based on a preset mixed reference variance aggregation module, extracting a first variance entropy parameter of the first probability matrix and a second variance entropy parameter of the second probability matrix respectively, and aggregating the first variance entropy parameter and the second variance entropy parameter to obtain an aggregated target variance entropy parameter;

[0039] A third probability matrix is ​​formed based on the aggregated target mean and the aggregated target variance entropy parameter.

[0040] Optionally, the step of determining a scaling factor corresponding to the required bit rate includes:

[0041] Get a set of scaling factors, where the scaling factors are training configuration parameters obtained after the point cloud compression model training is completed;

[0042] Determining a target scaling factor closest to the required bit rate based on the scaling factor set;

[0043] Based on a preset interpolation, the target scaling factor is adjusted and calculated to obtain an adjusted scaling factor, and the adjusted scaling factor is used as the scaling factor corresponding to the required bit rate.

[0044] The present application also provides a point cloud compression device with a variable bit rate, the point cloud compression device with a variable bit rate comprising:

[0045] The acquisition module is used to obtain the point cloud to be compressed and the required bit rate;

[0046] A determination module, configured to determine a scaling factor corresponding to the required bit rate;

[0047] A compression module is used to compress the point cloud to a scale corresponding to the scaling factor based on the point cloud to be compressed through a preset point cloud compression model and a scale linear transformation module of the point cloud compression model, so as to obtain a compressed point cloud that meets the required bit rate, wherein the training configuration parameters of the point cloud compression model include the scaling factor, and the point cloud compression model includes a scale linear transformation module, and the scale linear transformation module is used to achieve feature scaling of different scales.

[0048] The present application also provides a point cloud compression device with a variable bit rate, the point cloud compression device with a variable bit rate comprising: a memory, a processor, and a program stored in the memory for implementing the point cloud compression method with a variable bit rate.

[0049] The memory is used to store a program for implementing a point cloud compression method with variable bit rate;

[0050] The processor is used to execute a program for implementing the variable bit rate point cloud compression method to implement the steps of the variable bit rate point cloud compression method.

[0051] The present application also provides a storage medium, on which is stored a program for implementing a point cloud compression method with a variable bit rate. The program for implementing a point cloud compression method with a variable bit rate is executed by a processor to implement the steps of the point cloud compression method with a variable bit rate.

[0052] The present application provides a scale linear transformation module on the structure of the pre-trained point cloud compression model, and sets multiple scaling factors on the training configuration parameters of the point cloud compression model, so that the scale linear transformation module of the point cloud compression model is used to scale the compressed point cloud by the scaling factor to obtain a compressed point cloud with a bit rate that meets the user's required bit rate. This method only requires training one point cloud compression model, and there is no need to train multiple neural network models, thereby reducing resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the embodiments or the description of the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without inventive work.

[0054] FIG1 is a flow chart of a first embodiment of a variable bit rate point cloud compression method of the present application;

[0055] FIG2 is a schematic diagram of the structure of a point cloud compression model of the variable bit rate point cloud compression method of the present application;

[0056] FIG3 is a flow chart of a second embodiment of the variable bit rate point cloud compression method of the present application;

[0057] FIG4 is a schematic diagram of the structure of each module in the point cloud compression model of the variable bit rate point cloud compression method of the present application;

[0058] FIG5 is a schematic diagram of a module of a point cloud compression device with a variable bit rate according to the present application;

[0059] FIG6 is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application.

[0060] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0062] 1 , which is a flow chart of a first embodiment of a point cloud compression method with variable bit rate according to the present application.

[0063] In a first embodiment, the variable bit rate point cloud compression method includes the following steps:

[0064] Step S100, obtaining the point cloud to be compressed and the required bit rate;

[0065] It should be noted that the executor of the variable bit rate point cloud compression method is a variable bit rate point cloud compression device. Preferably, the variable bit rate point cloud compression device is a software client, or it can be other terminals with data transmission and data processing functions, and no specific restrictions are made here.

[0066] It can be understood that the point cloud to be compressed refers to the point cloud / or point cloud set that needs to be compressed to a specified bit rate, and the required bit rate refers to the bit rate value required by the user for point cloud compression. For example, if the user's required bit rate is 0.3bpp, the device needs to convert the point cloud to be compressed into a compressed point cloud with a bit rate of 0.3bpp.

[0067] In a specific implementation, the device may obtain the point cloud to be compressed by uploading it through a user, or by receiving the point cloud to be compressed collected by a preset point cloud collection device; the device may obtain the required bit rate by usually obtaining the required bit rate through the user uploading the required bit rate information according to their own needs, or by other methods. The above methods of obtaining the point cloud to be compressed and the required bit rate are not specifically limited here.

[0068] Step S200, determining a scaling factor corresponding to the required bit rate;

[0069] It should be noted that the scaling factor is a training configuration parameter in the neural network model (specifically, the point cloud compression model in this embodiment), which is used to control the bit rate value of the compressed point cloud output by the point cloud compression model. Usually, a plurality of scaling factors corresponding to different bit rates are set in the point cloud compression model. However, in order to control the computing resources consumed by model training, the number of scaling factors will not be very large, and usually the scaling factors with the highest bit rate required by the user are used. Specifically, although the device can set the number of vectors to a very large number to achieve a bit rate with fine intervals, this will increase the parameters of the SLT and ISLT modules and consume a large amount of training resources and storage resources. Therefore, a small number of scaling factors are usually set.

[0070] For example, four code rate scaling factors of 0.1, 0.3, 0.5, and 0.7 are trained in the point cloud compression model. When the required code rate is 0.3, the device selects the scaling factor of 0.3.

[0071] Furthermore, since there is a certain interval between scaling factors, there is a situation where the required bit rate and the scaling factor do not completely correspond. Usually, the scaling factor closest to the required bit rate is used as the scaling factor of the current point cloud to be compressed.

[0072] For example, four code rate scaling factors of 0.1, 0.3, 0.5, and 0.7 are trained in the point cloud compression model. When the required code rate is 0.45, the device selects the scaling factor of 0.5.

[0073] In a specific implementation, the method for determining the scaling factor corresponding to the required bit rate by the device further includes the following steps:

[0074] The device obtains a set of scaling factors, wherein the scaling factors are training configuration parameters obtained after the point cloud compression model training is completed; based on the scaling factor set, a target scaling factor closest to the required bit rate is determined; based on a preset interpolation, the target scaling factor is adjusted and calculated to obtain an adjusted scaling factor, and the adjusted scaling factor is used as the scaling factor corresponding to the required bit rate.

[0075] It should be noted that the scaling factor set refers to the set of scaling factors configured in the neural network model, that is, it includes scaling factors of multiple different bit rates. Since the device only sets a small number of scaling factors and there is a certain interval between the scaling factors, there is a situation where the required bit rate and the scaling factor do not completely correspond. Based on this, the application proposes a training-free interpolation method to solve this problem. Based on the pre-trained bit rate, in order to achieve a bit rate that is not at the pre-trained bit rate point, an interpolation method can be used to approximate the given bit rate.

[0076] Specifically, the device first determines the target scaling factor closest to the required bit rate from the scaling factor set. The target scaling factor is specifically two scaling factors closest to the required bit rate, one of which is the scaling factor closest to the required bit rate and less than the required bit rate, and the other is the scaling factor closest to the required bit rate and greater than the required bit rate. For example, four scaling factors with bit rates of 0.1, 0.3, 0.5 and 0.7 are trained in the point cloud compression model. When the required bit rate is 0.45, the target scaling factors include 0.3 and 0.5.

[0077] In a specific implementation, the interpolation is a manually set calculation parameter for adjusting the target scaling factor, ranging from 0 to 1. In order to approximate a given bit rate, the device needs to convert the target scaling factor into the following interpolation formula: Y = (1-β)v1 + βv2

[0078] Where Y represents the adjusted scaling factor, v1 and v2 represent the two target scaling factors, and β represents the interpolation value.

[0079] Furthermore, the interpolation adjustment method can also be used in the feature scaling process. Specifically, the features need to be transformed as follows:

[0080] in represents channel-level multiplication, V′ represents the transformed feature, V represents the feature before transformation, v1 and v2 represent two target scaling factors, and β represents interpolation.

[0081] It is understandable that the entire process is equivalent to generating a new scale factor to scale the features. To more accurately estimate the interpolated features, the super-prior network and the hybrid channel prior module also perform interpolation to ensure that the interpolated features match the interpolated entropy parameters.

[0082] In the specific implementation, this application designs a training-free interpolation method that covers a wide range of bit rates and achieves arbitrary bit rate compression to meet the requirements of different scenarios. The interpolation method interpolates the trained bit rate to generate a new bit rate and simultaneously interpolates the entropy parameter to match the interpolated features.

[0083] Furthermore, interpolation can achieve the required bitrate for any environment. For a given bitrate, this method can get very close to the required bitrate without requiring retraining. Compared to other methods, these methods can only select a limited set of pre-trained parameters to meet the given bitrate requirement and cannot get very close to the given bitrate, which wastes bits and is less flexible.

[0084] Step S300: Based on the point cloud to be compressed, point cloud compression of the scale corresponding to the scaling factor is performed through a preset point cloud compression model and a scale linear transformation module of the point cloud compression model to obtain a compressed point cloud that meets the required bit rate, wherein the training configuration parameters of the point cloud compression model include the scaling factor, and the point cloud compression model includes a scale linear transformation module, and the scale linear transformation module is used to achieve feature scaling of different scales.

[0085] In a specific implementation, the present application sets a scale linear transformation module on the model structure of the point cloud compression model for realizing feature scaling of different scales. The scale linear transformation module can realize point cloud scaling of the point cloud to be compressed at a scale corresponding to the scaling factor, thereby obtaining a compressed point cloud that meets the required bit rate.

[0086] Specifically, referring to the schematic diagram of the point cloud compression model structure in Figure 2, the encoder is a downsampling network including a feature extraction module (FEM) and a scale linear transformation module (SLT), wherein the feature extraction module is used to eliminate the redundancy in the features of the point cloud to be compressed and retain the key information for reconstruction; the decoder is an upsampling network including a feature reconstruction module (FRM) and an inverse scale linear transformation module (ISLT), wherein the feature reconstruction module gradually generates a point cloud based on the retained features; Q in the figure represents the quantization process, Conv represents the convolutional neural network, and ReLU is an activation function for nonlinear transformation; x in Figure 2 represents the input point cloud, Represents the reconstructed point cloud; y and z represent the extracted features. The extracted point cloud features are divided into two parts. One part of the geometric features uses octree lossless encoding, where OE represents octree encoder and OD represents octree decoder; the other part of the features uses lossy entropy coding for compression, where AE represents arithmetic encoder and AD represents arithmetic decoder.

[0087] In a specific implementation, the device compresses the point cloud to a scale corresponding to the scaling factor based on the point cloud to be compressed by using a preset point cloud compression model and a scale linear transformation module of the point cloud compression model, and the method for obtaining a compressed point cloud that meets the required bit rate further includes the following steps:

[0088] The device extracts features based on the point cloud to be compressed through a preset point cloud compression model to obtain point cloud features; based on a scale linear transformation module of the point cloud compression model, the device performs point cloud compression on the point cloud features at a scale corresponding to the scaling factor to obtain a potential representation of the point cloud to be compressed; based on a preset first encoding module, the device compresses the potential representation to obtain a compressed point cloud that meets the required bit rate.

[0089] In a specific implementation, referring to FIG2 , the feature extraction module (FEM) in the preset point cloud compression model is used to extract features from the point cloud to be compressed, thereby obtaining point cloud features. The point cloud features are then compressed to a scale corresponding to the scaling factor by the scale linear transformation module (SLT) of the point cloud compression model, thereby obtaining a potential representation of the point cloud to be compressed, wherein the potential representation is a partial feature containing important features of the point cloud to be compressed. Furthermore, the first encoding module includes an octree encoder OE and an arithmetic encoder AE, which compresses a portion of the geometric features of the potential representation of the point cloud to be compressed using octree lossless coding, and compresses another portion of the features using lossy entropy coding, thereby obtaining a compressed point cloud that meets the required bit rate.

[0090] In a specific implementation, after the device performs point cloud compression on the point cloud features at a scale corresponding to the scaling factor based on the scale linear transformation module of the point cloud compression model to obtain the potential representation of the point cloud to be compressed, the method includes:

[0091] Based on a preset hyper-prior network, the device performs parallel feature convolution on the potential representation to obtain the point cloud features after the first convolution; based on the scale linear transformation module of the point cloud compression model, the point cloud features after the first convolution are compressed with a scale corresponding to the scaling factor, and output a first probability matrix of the potential representation, wherein the first probability matrix reflects the side information of the point cloud to be compressed; based on the preset second encoding module, the first probability matrix is ​​compressed to obtain a compressed first probability matrix, and based on the preset first decoding module, the compressed first probability matrix is ​​decompressed to obtain a decompressed first probability matrix, so as to transmit the first probability matrix to the preset first encoding module; based on the preset first encoding module, the potential representation and the decompressed first probability matrix are compressed to obtain a compressed point cloud that meets the required bit rate.

[0092] It should be noted that when performing arithmetic coding and arithmetic decoding on the latent representation, the probability matrix of the latent representation is required. The calculation of the probability matrix of the point cloud to be compressed in this application is mainly through a preset super-prior network to determine the side information of the point cloud to be compressed, and then obtain the first probability matrix of the latent representation. In the process of determining the side information of the point cloud to be compressed, the super-prior network also needs to perform compression of the scale linear transformation module.

[0093] In a specific implementation, after determining the first probability matrix of the potential representation, the device also needs to transmit the first probability matrix to the first encoding module, so that the device can compress the potential representation and the decompressed first probability matrix based on the preset first encoding module to obtain a compressed point cloud that meets the required bit rate.

[0094] Based on this, referring to Figure 2, the device compresses the first probability matrix z through the second encoding module to obtain a compressed first probability matrix, and then decompresses the compressed first probability matrix based on the preset first decoding module to obtain a decompressed first probability matrix, so as to transmit the first probability matrix to the preset first encoding module, wherein the second encoding module has the same structure as the above-mentioned first encoding module. Based on the above transmission, the present application reduces the computing resources and storage resources for transmitting the first probability matrix.

[0095] In a specific implementation, the device performs point cloud compression on the convolved point cloud features at a scale corresponding to the scaling factor based on a scale linear transformation module of the point cloud compression model, and outputs a first probability matrix of the potential representation. The method then includes:

[0096] The device inputs the first probability matrix into a preset decomposition density entropy model, performs probability calculation on the first probability matrix based on the decomposition density entropy model and a preset number of entropy parameters of the decomposition density entropy model, and obtains a probability matrix set of the corresponding number of first probability matrices; calculates the transmission code rate value of each probability matrix under the probability matrix set, and selects the probability matrix corresponding to the minimum transmission code rate value as the target probability matrix; based on a preset second encoding module, compresses the first probability matrix and the target probability matrix to obtain a compressed first probability matrix.

[0097] In the specific implementation, the scale linear transformation module SLT module and the inverse scale linear transformation module ISLT module are designed to achieve discrete multiple bit rates in one network. Directly scaling the data range of the potential representation will severely reduce the rate-distortion performance. Therefore, the device scales the intermediate features, potential representations and entropy parameters to match the features with the probability estimation parameters. The SLT and ISLT modules are matrices composed of multiple vectors, and different vectors correspond to different bit rates. When compressing the point cloud, only one vector is used to scale the activation value on the channel dimension. ISLT and SLT are also embedded in the super-prior network and the combined channel prior module network to achieve the scaling of the entropy parameter. And the potential representation generated by the super-prior network also consumes bits for transmitting data.

[0098] Based on this, this application designs a factorized density entropy model to estimate the probability of the potential representation generated by the super-prior network. The multi-factor decomposition entropy model contains multiple sets of parameters for estimating the probability of the potential representation at different bit rates. It is worth noting that the factorized density model has very few parameters, so multiple sets of parameters do not consume too much storage resources. The goal of the end-to-end training process is to reduce the consumed bit rate and the distortion of the reconstructed point cloud, and the binary cross entropy loss is used as the loss function in the gradient descent process.

[0099] Furthermore, for the first probability matrix at different bit rates, multiple sets of decomposition density entropy models will have multiple sets of parameters to generate multiple probabilities for the first probability matrix. The required bit rate for transmitting the first probability matrix under each probability is then calculated, and the probability with the lowest bit rate consumption is selected, namely the target probability matrix. This target probability matrix and the first probability matrix are ultimately input into the automatic encoding (AE). The decomposition density entropy model can provide more accurate probability estimates for different first probability matrices, saving more bits, saving bit rate, and reducing transmission and storage resources.

[0100] In the specific implementation, this application designs a scale transformation module to embed multiple bit rates into a set of neural network parameters. This scale transformation module is embedded in the main encoding network, main decoding network, and super-prior network of the point cloud coding network. It not only changes the extracted latent features but also better achieves entropy estimation that matches the features.

[0101] The present application provides a scale linear transformation module on the structure of the pre-trained point cloud compression model, and sets multiple scaling factors on the training configuration parameters of the point cloud compression model, so that the scale linear transformation module of the point cloud compression model is used to scale the compressed point cloud by the scaling factor to obtain a compressed point cloud with a bit rate that meets the user's required bit rate. This method only requires training one point cloud compression model, and there is no need to train multiple neural network models, thereby reducing resource consumption.

[0102] Based on the above first embodiment, the present application also provides another embodiment, referring to FIG3 , wherein the variable bit rate point cloud compression method includes:

[0103] After the device performs point cloud compression on the point cloud features at a scale corresponding to the scaling factor based on the scale linear transformation module of the point cloud compression model to obtain the potential representation of the point cloud to be compressed, the method includes:

[0104] Step A100, performing serial iterative feature convolution on the latent representation based on a preset channel context conversion network to obtain a point cloud feature after the second convolution;

[0105] Step A200: Based on the scale linear transformation module of the point cloud compression model, perform point cloud compression on the second convolved point cloud features at a scale corresponding to the scaling factor, and output a second probability matrix of the potential representation, wherein the second probability matrix reflects channel context information of the point cloud to be compressed;

[0106] In a specific implementation, referring to FIG2 , the present application proposes determining the channel context information of the point cloud to be compressed through a preset channel context transformation network CTT, and then estimating a second probability matrix of the potential representation.

[0107] Step A300: Determine a first probability matrix of the potential representation based on a preset super-prior network, wherein the first probability matrix reflects side information of the point cloud to be compressed, and the channel context transfer network and the super-prior network process the potential representation synchronously;

[0108] In a specific implementation, reference is made to the step of estimating the first probability matrix of the potential representation based on the preset super-prior network in the first embodiment above, which will not be repeated here.

[0109] Step A400: Aggregating the first probability matrix and the second probability matrix based on a preset aggregation module to obtain a third probability matrix;

[0110] In a specific implementation, the device aggregates the first probability matrix and the second probability matrix based on a preset aggregation module to obtain a third probability matrix, further comprising the following steps:

[0111] The device extracts the first mean of the first probability matrix and the second mean of the second probability matrix based on a preset hybrid reference mean aggregation module, and aggregates the first mean and the second mean to obtain an aggregated target mean; based on the preset hybrid reference variance aggregation module, the device extracts the first variance entropy parameter of the first probability matrix and the second variance entropy parameter of the second probability matrix, and aggregates the first variance entropy parameter and the second variance entropy parameter to obtain an aggregated target variance entropy parameter; and based on the aggregated target mean and the aggregated target variance entropy parameter, a third probability matrix is ​​formed.

[0112] In the specific implementation, the present application also designs a hybrid prior network to capture side information and channel context information to estimate the probability of the potential representation. Specifically, the hybrid prior network includes a hybrid channel prior module (HCPM) to mix side information and channel context information to obtain better probability estimation. The side information is captured by the super prior network. The hybrid channel prior module consists of a channel context conversion module (CCT), a hybrid reference mean aggregation module (MRMA) and a hybrid reference variance aggregation module (MRSA). CCT captures channel prior information to better eliminate the probability of occurrence without consuming bit rate. MRMA and MRSA aggregate the mean and variance parameters, respectively, and aggregate the mean and variance entropy parameters generated by the super prior network and the mean and variance entropy parameters generated by the HCPM module, respectively. Specifically, the composition of all modules can be referred to Figure 4.

[0113] Step A500: compress the potential representation and the third probability matrix based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate.

[0114] The present application also provides a point cloud compression device with a variable bit rate. Referring to FIG. 5 , the point cloud compression device with a variable bit rate includes:

[0115] An acquisition module 10 is used to obtain the point cloud to be compressed and the required bit rate;

[0116] A determination module 20, configured to determine a scaling factor corresponding to the required bit rate;

[0117] The compression module 30 is used to compress the point cloud to a scale corresponding to the scaling factor based on the point cloud to be compressed through a preset point cloud compression model and a scale linear transformation module of the point cloud compression model to obtain a compressed point cloud that meets the required bit rate, wherein the training configuration parameters of the point cloud compression model include the scaling factor, and the point cloud compression model includes a scale linear transformation module, and the scale linear transformation module is used to achieve feature scaling of different scales.

[0118] Optionally, the compression module 30 includes:

[0119] An extraction module, configured to extract features from the point cloud to be compressed using a preset point cloud compression model to obtain point cloud features;

[0120] a feature compression module, configured to perform point cloud compression on the point cloud features at a scale corresponding to the scaling factor based on the scale linear transformation module of the point cloud compression model, to obtain a potential representation of the point cloud to be compressed;

[0121] The first encoding module is used to compress the potential representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate.

[0122] Optionally, the variable bit rate point cloud compression device further includes:

[0123] A parallel convolution module is used to perform parallel feature convolution on the potential representation based on a preset super-prior network to obtain point cloud features after the first convolution;

[0124] a first probability matrix calculation module, configured to perform point cloud compression on the first convolved point cloud features at a scale corresponding to the scaling factor based on the scale linear transformation module of the point cloud compression model, and output a first probability matrix of the potential representation, wherein the first probability matrix reflects side information of the point cloud to be compressed;

[0125] a matrix compression module, configured to compress the first probability matrix based on a preset second encoding module to obtain a compressed first probability matrix;

[0126] a decompression module, configured to decompress the compressed first probability matrix based on a preset first decoding module to obtain a decompressed first probability matrix, so as to transmit the first probability matrix to a preset first encoding module;

[0127] The second encoding module is used to compress the potential representation and the decompressed first probability matrix based on the preset first encoding module to obtain a compressed point cloud that meets the required bit rate.

[0128] Optionally, the variable bit rate point cloud compression device further includes:

[0129] a probability calculation module, configured to input the first probability matrix into a preset decomposition density entropy model, perform probability calculation on the first probability matrix based on the decomposition density entropy model and a preset number of entropy parameters of the decomposition density entropy model, and obtain a probability matrix set of a corresponding number of the first probability matrices;

[0130] A selection module is used to calculate the transmission code rate value of each probability matrix under the probability matrix set, and select the probability matrix corresponding to the minimum transmission code rate value as the target probability matrix;

[0131] The third encoding module is used to compress the first probability matrix and the target probability matrix based on the preset second encoding module to obtain a compressed first probability matrix.

[0132] Optionally, the variable bit rate point cloud compression device further includes:

[0133] A serial convolution module, configured to perform serial iterative feature convolution on the potential representation based on a preset channel context conversion network to obtain a point cloud feature after the second convolution;

[0134] a second probability matrix calculation module, configured to perform point cloud compression on the second convolved point cloud features at a scale corresponding to the scaling factor based on the scale linear transformation module of the point cloud compression model, and output a second probability matrix of the potential representation, wherein the second probability matrix reflects channel context information of the point cloud to be compressed;

[0135] a first probability matrix determination module, configured to determine a first probability matrix of the potential representation based on a preset super-prior network, wherein the first probability matrix reflects side information of the point cloud to be compressed, and the channel context conversion network and the super-prior network process the potential representation synchronously;

[0136] an aggregation module, configured to aggregate the first probability matrix and the second probability matrix based on a preset aggregation module to obtain a third probability matrix;

[0137] The third encoding module is used to compress the potential representation and the third probability matrix based on the preset first encoding module to obtain a compressed point cloud that meets the required bit rate.

[0138] Optionally, the aggregation module includes:

[0139] a mean aggregation module, configured to extract, based on a preset hybrid reference mean aggregation module, a first mean of the first probability matrix and a second mean of the second probability matrix, respectively, and aggregate the first mean and the second mean to obtain an aggregated target mean;

[0140] A variance aggregation module is used to extract a first variance entropy parameter of the first probability matrix and a second variance entropy parameter of the second probability matrix based on a preset hybrid reference variance aggregation module, and aggregate the first variance entropy parameter and the second variance entropy parameter to obtain an aggregated target variance entropy parameter;

[0141] A combining module is used to form a third probability matrix based on the aggregated target mean and the aggregated target variance entropy parameter.

[0142] Optionally, the determining module 20 includes:

[0143] A scaling factor set acquisition module is used to obtain a scaling factor set, wherein the scaling factor is a training configuration parameter obtained after the point cloud compression model training is completed;

[0144] a target scaling factor determining module, configured to determine a target scaling factor closest to the required bit rate based on the scaling factor set;

[0145] The interpolation adjustment module is used to adjust and calculate the target scaling factor based on a preset interpolation to obtain an adjusted scaling factor, and use the adjusted scaling factor as the scaling factor corresponding to the required bit rate.

[0146] The specific implementation of the variable bit rate point cloud compression device of the present application is basically the same as the various embodiments of the variable bit rate point cloud compression method mentioned above, and will not be repeated here.

[0147] Refer to Figure 6, which is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0148] As shown in Figure 6, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0149] Optionally, the variable bit rate point cloud compression device may further include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. The rectangular user interface may include a display and an input submodule such as a keyboard. Optionally, the rectangular user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WiFi interface).

[0150] Those skilled in the art will understand that the variable bit rate point cloud compression device structure shown in Figure 5 does not constitute a limitation on the variable bit rate point cloud compression device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0151] As shown in FIG6 , memory 1005 , a storage medium, may include an operating system, a network communication module, and a variable bit rate point cloud compression program. The operating system manages and controls the hardware and software resources of the variable bit rate point cloud compression device, supporting the operation of the variable bit rate point cloud compression program and other software and / or programs. The network communication module is used to enable communication between components within memory 1005 , as well as with other hardware and software in the variable bit rate point cloud compression system.

[0152] In the variable bit rate point cloud compression device shown in Figure 6, the processor 1001 is used to execute the variable bit rate point cloud compression program stored in the memory 1005 to implement the steps of the variable bit rate point cloud compression method described in any one of the above items.

[0153] The specific implementation of the variable bit rate point cloud compression device of the present application is basically the same as the various embodiments of the variable bit rate point cloud compression method mentioned above, and will not be repeated here.

[0154] The present application further provides a storage medium, wherein a program for implementing a point cloud compression method with a variable bit rate is stored on the storage medium. The program for implementing a point cloud compression method with a variable bit rate is executed by a processor to implement the following point cloud compression method with a variable bit rate:

[0155] Get the point cloud to be compressed and the required bit rate;

[0156] Determining a scaling factor corresponding to the required bit rate;

[0157] Based on the point cloud to be compressed, point cloud compression of the scale corresponding to the scaling factor is performed through a preset point cloud compression model and a scale linear transformation module of the point cloud compression model to obtain a compressed point cloud that meets the required bit rate, wherein the training configuration parameters of the point cloud compression model include the scaling factor, and the point cloud compression model includes a scale linear transformation module, and the scale linear transformation module is used to achieve feature scaling of different scales.

[0158] Optionally, the step of performing point cloud compression on the point cloud to be compressed at a scale corresponding to the scaling factor using a preset point cloud compression model and a scale linear transformation module of the point cloud compression model to obtain a compressed point cloud meeting the required bit rate includes:

[0159] Based on the point cloud to be compressed, feature extraction is performed using a preset point cloud compression model to obtain point cloud features;

[0160] Based on the scale linear transformation module of the point cloud compression model, the point cloud features are compressed at a scale corresponding to the scaling factor to obtain a potential representation of the point cloud to be compressed;

[0161] Based on a preset first encoding module, the potential representation is compressed to obtain a compressed point cloud that meets the required bit rate.

[0162] Optionally, after the step of performing point cloud compression on the point cloud features at a scale corresponding to the scaling factor by the scale linear transformation module based on the point cloud compression model to obtain a potential representation of the point cloud to be compressed, the method includes:

[0163] Based on a preset super-prior network, parallel feature convolution is performed on the potential representation to obtain point cloud features after the first convolution;

[0164] Based on the scale linear transformation module of the point cloud compression model, the point cloud features after the first convolution are compressed at a scale corresponding to the scaling factor, and a first probability matrix of the potential representation is output, wherein the first probability matrix reflects the side information of the point cloud to be compressed.

[0165] Optionally, the step of compressing the potential representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate includes:

[0166] Based on a preset first encoding module, the potential representation and the first probability matrix are compressed to obtain a compressed point cloud that meets the required bit rate.

[0167] Optionally, after the step of performing point cloud compression on the first convolved point cloud features at a scale corresponding to the scaling factor by the scale linear transformation module based on the point cloud compression model and outputting the first probability matrix of the potential representation, the method includes:

[0168] Compressing the first probability matrix based on a preset second encoding module to obtain a compressed first probability matrix;

[0169] Based on a preset first decoding module, the compressed first probability matrix is ​​decompressed to obtain a decompressed first probability matrix, so as to transmit the first probability matrix to a preset first encoding module.

[0170] Optionally, the step of compressing the potential representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate includes:

[0171] Based on a preset first encoding module, the potential representation and the decompressed first probability matrix are compressed to obtain a compressed point cloud that meets the required bit rate.

[0172] Optionally, after the step of performing point cloud compression on the convolved point cloud features at a scale corresponding to the scaling factor by the scale linear transformation module based on the point cloud compression model and outputting the first probability matrix of the potential representation, the method includes:

[0173] Inputting the first probability matrix into a preset decomposition density entropy model, performing probability calculation on the first probability matrix based on the decomposition density entropy model and a preset number of entropy parameters of the decomposition density entropy model, and obtaining a corresponding number of probability matrix sets of the first probability matrices;

[0174] The transmission code rate value of each probability matrix under the probability matrix set is calculated, and the probability matrix corresponding to the minimum transmission code rate value is selected as the target probability matrix.

[0175] Optionally, the step of compressing the first probability matrix based on a preset second encoding module to obtain a compressed first probability matrix includes:

[0176] Based on a preset second encoding module, the first probability matrix and the target probability matrix are compressed to obtain a compressed first probability matrix.

[0177] Optionally, after the step of performing point cloud compression on the point cloud features at a scale corresponding to the scaling factor by the scale linear transformation module based on the point cloud compression model to obtain a potential representation of the point cloud to be compressed, the method includes:

[0178] Based on a preset channel context conversion network, performing serial iterative feature convolution on the potential representation to obtain a point cloud feature after the second convolution;

[0179] Based on the scale linear transformation module of the point cloud compression model, perform point cloud compression on the point cloud features after the second convolution at a scale corresponding to the scaling factor, and output a second probability matrix of the potential representation, wherein the second probability matrix reflects channel context information of the point cloud to be compressed;

[0180] Determining a first probability matrix of the latent representation based on a preset hyper-prior network, wherein the first probability matrix reflects side information of the point cloud to be compressed, and the channel context conversion network and the hyper-prior network process the latent representation synchronously;

[0181] Based on a preset aggregation module, the first probability matrix and the second probability matrix are aggregated to obtain a third probability matrix.

[0182] Optionally, the step of compressing the potential representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate includes:

[0183] Based on a preset first encoding module, the potential representation and the third probability matrix are compressed to obtain a compressed point cloud that meets the required bit rate.

[0184] Optionally, the aggregation module includes a hybrid reference mean aggregation module and a hybrid reference variance aggregation module, and the step of aggregating the first probability matrix and the second probability matrix based on the preset aggregation module to obtain a third probability matrix includes:

[0185] Based on a preset hybrid reference mean aggregation module, extracting a first mean of the first probability matrix and a second mean of the second probability matrix respectively, and aggregating the first mean and the second mean to obtain an aggregated target mean;

[0186] Based on a preset mixed reference variance aggregation module, extracting a first variance entropy parameter of the first probability matrix and a second variance entropy parameter of the second probability matrix respectively, and aggregating the first variance entropy parameter and the second variance entropy parameter to obtain an aggregated target variance entropy parameter;

[0187] A third probability matrix is ​​formed based on the aggregated target mean and the aggregated target variance entropy parameter.

[0188] Optionally, the step of determining a scaling factor corresponding to the required bit rate includes:

[0189] Get a set of scaling factors, where the scaling factors are training configuration parameters obtained after the point cloud compression model training is completed;

[0190] Determining a target scaling factor closest to the required bit rate based on the scaling factor set;

[0191] Based on a preset interpolation, the target scaling factor is adjusted and calculated to obtain an adjusted scaling factor, and the adjusted scaling factor is used as the scaling factor corresponding to the required bit rate.

[0192] The specific implementation of the storage medium of the present application is basically the same as the various embodiments of the above-mentioned variable bit rate point cloud compression method, and will not be repeated here.

[0193] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned variable-rate point cloud compression method.

[0194] The specific implementation of the computer program product of the present application is basically the same as the various embodiments of the above-mentioned variable bit rate point cloud compression method, and will not be repeated here.

[0195] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0196] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0197] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0198] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A variable bitrate point cloud compression method, characterized in that, The variable bitrate point cloud compression method includes: Obtaining the point cloud to be compressed and the required bitrate; Determining the scaling factor corresponding to the required bitrate; Based on the point cloud to be compressed, performing point cloud compression at the scale corresponding to the scaling factor through a preset point cloud compression model and the scale linear transformation module of the point cloud compression model, to obtain a compressed point cloud that meets the required bitrate. Among them, the training configuration parameters of the point cloud compression model include the scaling factor, the point cloud compression model includes a scale linear transformation module, and the scale linear transformation module is used to implement feature scaling at different scales.

2. The variable bitrate point cloud compression method according to claim 1, characterized in that, The step of performing point cloud compression at the scale corresponding to the scaling factor based on the point cloud to be compressed through a preset point cloud compression model and the scale linear transformation module of the point cloud compression model, to obtain a compressed point cloud that meets the required bitrate, includes: Based on the point cloud to be compressed, performing feature extraction through a preset point cloud compression model to obtain point cloud features; Based on the scale linear transformation module of the point cloud compression model, performing point cloud compression at the scale corresponding to the scaling factor on the point cloud features to obtain a latent representation of the point cloud to be compressed; Based on a preset first encoding module, compressing the latent representation to obtain a compressed point cloud that meets the required bitrate.

3. The variable bitrate point cloud compression method according to claim 2, wherein, After the step of performing point cloud compression at the scale corresponding to the scaling factor based on the scale linear transformation module of the point cloud compression model on the point cloud features to obtain a latent representation of the point cloud to be compressed, the method includes: Based on a preset hyperprior network, performing parallel feature convolution on the latent representation to obtain first-convolved point cloud features; Based on the scale linear transformation module of the point cloud compression model, performing point cloud compression at the scale corresponding to the scaling factor on the first-convolved point cloud features, and outputting a first probability matrix of the latent representation, where the first probability matrix reflects the side information of the point cloud to be compressed.

4. The variable bitrate point cloud compression method according to claim 3, wherein, The step of compressing the latent representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bitrate includes: Based on a preset first encoding module, compressing the latent representation and the first probability matrix to obtain a compressed point cloud that meets the required bitrate.

5. The variable bitrate point cloud compression method according to claim 3, characterized in that After the step of performing point cloud compression at the scale corresponding to the scaling factor based on the scale linear transformation module of the point cloud compression model on the first-convolved point cloud features and outputting a first probability matrix of the latent representation, the method includes: Based on a preset second encoding module, compressing the first probability matrix to obtain a compressed first probability matrix; Based on a preset first decoding module, decompressing the compressed first probability matrix to obtain a decompressed first probability matrix, so as to transmit the first probability matrix to a preset first encoding module.

6. The variable bitrate point cloud compression method according to claim 5, characterized in that The step of compressing the latent representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bitrate includes: Based on a preset first encoding module, compressing the latent representation and the decompressed first probability matrix to obtain a compressed point cloud that meets the required bitrate.

7. The variable bitrate point cloud compression method according to claim 3, characterized in that, After the step of performing point cloud compression at the scale corresponding to the scaling factor on the point cloud features after convolution by the scale linear transformation module based on the point cloud compression model, and outputting the first probability matrix of the latent representation, the method includes: Inputting the first probability matrix into a preset decomposition density entropy model, and performing probability calculation on the first probability matrix based on the decomposition density entropy model and a preset number of entropy parameters of the decomposition density entropy model to obtain a probability matrix set of the corresponding number of the first probability matrices; Calculating the transmission code rate values of each probability matrix in the probability matrix set, and selecting the probability matrix corresponding to the minimum transmission code rate value as the target probability matrix.

8. The variable bitrate point cloud compression method according to claim 7, characterized in that, The step of compressing the first probability matrix based on a preset second coding module to obtain the compressed first probability matrix includes: Based on a preset second coding module, compressing the first probability matrix and the target probability matrix to obtain the compressed first probability matrix.

9. The variable bitrate point cloud compression method according to claim 2, characterized in that, After the step of performing point cloud compression at the scale corresponding to the scaling factor on the point cloud features by the scale linear transformation module based on the point cloud compression model to obtain the latent representation of the point cloud to be compressed, the method includes: Based on a preset channel context conversion network, performing serial iterative feature convolution on the latent representation to obtain the second-convolved point cloud features; Performing point cloud compression at the scale corresponding to the scaling factor on the second-convolved point cloud features by the scale linear transformation module based on the point cloud compression model, and outputting the second probability matrix of the latent representation, where the second probability matrix reflects the channel context information of the point cloud to be compressed; Based on a preset hyperprior network, determining the first probability matrix of the latent representation, where the first probability matrix reflects the side information of the point cloud to be compressed, and the processing processes of the channel context conversion network and the hyperprior network on the latent representation are carried out synchronously; Based on a preset aggregation module, aggregating the first probability matrix and the second probability matrix to obtain a third probability matrix.

10. The variable bitrate point cloud compression method according to claim 9, characterized in that, The step of compressing the latent representation based on a preset first coding module to obtain the compressed point cloud that meets the required code rate includes: Based on a preset first coding module, compressing the latent representation and the third probability matrix to obtain the compressed point cloud that meets the required code rate.

11. The variable bitrate point cloud compression method according to claim 9, wherein The aggregation module includes a hybrid reference mean aggregation module and a hybrid reference variance aggregation module. The step of aggregating the first probability matrix and the second probability matrix based on a preset aggregation module to obtain a third probability matrix includes: Based on a preset hybrid reference mean aggregation module, respectively extracting the first mean of the first probability matrix and the second mean of the second probability matrix, and aggregating the first mean and the second mean to obtain the aggregated target mean; Based on a preset hybrid reference variance aggregation module, respectively extract the first variance entropy parameter of the first probability matrix and the second variance entropy parameter of the second probability matrix, and aggregate the first variance entropy parameter and the second variance entropy parameter to obtain an aggregated target variance entropy parameter; Based on the aggregated target mean and the aggregated target variance entropy parameter, form a third probability matrix.

12. The variable bitrate point cloud compression method according to claim 1, characterized in that, The step of determining the scaling factor corresponding to the required bitrate includes: Obtain a set of scaling factors, where the scaling factor is a training configuration parameter obtained after the completion of the training of the point cloud compression model; Based on the set of scaling factors, determine the target scaling factor closest to the required bitrate; Based on a preset interpolation, perform adjustment calculation on the target scaling factor to obtain an adjusted scaling factor, and use the adjusted scaling factor as the scaling factor corresponding to the required bitrate.

13. A variable bitrate point cloud compression device, characterized in that, The variable-bitrate point cloud compression device includes: An acquisition module for acquiring a point cloud to be compressed and a required bitrate; A determination module for determining the scaling factor corresponding to the required bitrate; A compression module for performing point cloud compression at the scale corresponding to the scaling factor on the point cloud to be compressed through a preset point cloud compression model and the scale linear transformation module of the point cloud compression model to obtain a compressed point cloud that meets the required bitrate, where the training configuration parameter of the point cloud compression model includes a scaling factor, the point cloud compression model includes a scale linear transformation module, and the scale linear transformation module is used to implement feature scaling at different scales.

14. The variable bitrate point cloud compression device according to claim 13, wherein The compression module includes: An extraction module for extracting point cloud features based on the point cloud to be compressed through a preset point cloud compression model; A feature compression module for performing point cloud compression at the scale corresponding to the scaling factor on the point cloud features based on the scale linear transformation module of the point cloud compression model to obtain a latent representation of the point cloud to be compressed; A first encoding module for compressing the latent representation based on a preset first encoding module to obtain a compressed point cloud that meets the required bit rate.

15. The variable bitrate point cloud compression device according to claim 14, characterized in that, The variable-bitrate point cloud compression device further includes: A parallel convolution module for performing parallel feature convolution on the latent representation based on a preset hyperprior network to obtain first-convolved point cloud features; A first probability matrix calculation module for performing point cloud compression at the scale corresponding to the scaling factor on the first-convolved point cloud features based on the scale linear transformation module of the point cloud compression model and outputting a first probability matrix of the latent representation, where the first probability matrix reflects the side information of the point cloud to be compressed.

16. The variable bitrate point cloud compression device according to claim 15, characterized in that, The first encoding module includes: A second encoding module for compressing the latent representation and the decompressed first probability matrix based on a preset first encoding module to obtain a compressed point cloud that meets the required bitrate.

17. The variable bitrate point cloud compression device according to claim 15, wherein, The variable-bitrate point cloud compression device further includes: A matrix compression module for compressing the first probability matrix based on a preset second encoding module to obtain a compressed first probability matrix; A decompression module, configured to decompress the compressed first probability matrix based on a preset first decoding module to obtain a decompressed first probability matrix, so as to implement transmitting the first probability matrix to a preset first encoding module.

18. The variable bitrate point cloud compression device according to claim 15, characterized in that, The variable bitrate point cloud compression device further includes: A probability calculation module, configured to input the first probability matrix into a preset decomposition density entropy model, and perform probability calculation on the first probability matrix based on the decomposition density entropy model and a preset number of entropy parameters of the decomposition density entropy model, to obtain a probability matrix set of the corresponding number of the first probability matrices; A selection module, configured to calculate transmission bitrate values of each probability matrix in the probability matrix set, and select the probability matrix corresponding to the minimum transmission bitrate value as a target probability matrix.

19. A variable bitrate point cloud compression device, characterized in that, The variable bitrate point cloud compression device includes: a memory, a processor, and a program stored on the memory for implementing the variable bitrate point cloud compression method, The memory is used to store a program for implementing the variable bitrate point cloud compression method; The processor is configured to execute the program for implementing the variable bitrate point cloud compression method, so as to implement the steps of the variable bitrate point cloud compression method as claimed in claim 1.

20. A storage medium, characterized in that, A program for implementing the variable bitrate point cloud compression method is stored on the storage medium, and the program for implementing the variable bitrate point cloud compression method is executed by a processor to implement the steps of the variable bitrate point cloud compression method as claimed in claim 1.

Images