Dynamic point cloud compression method, device and equipment and computer readable storage medium

Abstract

The invention discloses a dynamic point cloud compression method, device and equipment and a computer readable storage medium, and relates to the technical field of data processing, and the method comprises the steps: aiming at any first point cloud frame in all point cloud frames except a first point cloud frame in to-be-compressed dynamic point cloud data; based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generating an estimated code rate of each coding path; determining a target code rate of the first point cloud frame, and selecting a target encoding path in the encoding paths based on the target code rate to encode the first point cloud frame to obtain a first point cloud feature; and generating compressed dynamic point cloud data based on the respective first point cloud feature of each point cloud frame and a second point cloud feature obtained by encoding the first point cloud frame in the dynamic point cloud data. According to the invention, the code rate control precision during dynamic point cloud compression can be improved, so that the coding quality is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a dynamic point cloud compression method, device, equipment and computer-readable storage medium. Background Art

[0002] With the rapid development of 3D scanning technology and sensor technology, point cloud data, as an important form of 3D information representation, has been widely used in many fields such as virtual reality, autonomous driving, telemedicine and digital entertainment. In order to efficiently store and transmit these point cloud data, dynamic point cloud compression technology is particularly important.

[0003] At present, the traditional dynamic point cloud compression method is to determine a fixed bit rate based on limited bandwidth, and uniformly encode all point cloud data according to the fixed bit rate. However, this method of uniform encoding with a fixed bit rate lacks consideration of the complexity of the point cloud. For example, high-complexity point cloud frames may have poor encoding effects due to insufficient bit allocation during the encoding process, and low-complexity point cloud frames may waste bandwidth resources due to excessive bit allocation. In other words, this compression method with low bit rate control accuracy may directly lead to a reduction in the overall encoding quality.

[0004] Therefore, how to improve the bit rate control accuracy during dynamic point cloud compression to improve the encoding quality is a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of this application is to provide a dynamic point cloud compression method, device, equipment and computer-readable storage medium, aiming to improve the bit rate control accuracy during dynamic point cloud compression to improve the encoding quality.

[0006] To achieve the above-mentioned object, the present application provides a dynamic point cloud compression method, the dynamic point cloud compression method comprising: for any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generating the estimated bit rate of each encoding path, wherein the second point cloud frame is the point cloud frame before the first point cloud frame in the dynamic point cloud data;

[0007] Determine a target bit rate of the first point cloud frame, and select a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature;

[0008] Compressed dynamic point cloud data are generated based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0009] In one embodiment, the step of generating an estimated bit rate of each encoding path based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame includes:

[0010] The first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain an estimated bit rate for each encoding path, wherein the convolutional neural network model is used to adaptively adjust the accuracy of the bit rate estimation according to the input features.

[0011] In one embodiment, the step of obtaining the estimated bit rate of each encoding path includes:

[0012] Determining the complexity and geometric distribution of the first point cloud frame by using the convolutional neural network model, and determining the encoding of the second point cloud frame based on the motion vector;

[0013] The convolutional neural network model generates an estimated bit rate for each coding path based on the complexity, the geometric distribution and the coding situation.

[0014] In one embodiment, the step of determining a target bit rate of the first point cloud frame includes:

[0015] Obtaining a preset expected average bit rate and a preset sliding window length, and determining the number of encoded point cloud frames in the dynamic point cloud data, and determining a total bit rate consumed by the encoded point cloud frames;

[0016] Add the number of frames to the sliding window length to obtain a sum;

[0017] The total bit rate is subtracted from the sum to obtain a difference, and the difference is divided by the sliding window length to obtain a target bit rate of the first point cloud frame.

[0018] In one embodiment, the step of selecting a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame includes:

[0019] Determine the product of the expected average bit rate and the number of frames, and determine whether the product is greater than the total bit rate;

[0020] If so, determining that a coding path corresponding to a first bit rate among the estimated bit rates is a target coding path, wherein the first bit rate is an estimated bit rate among the estimated bit rates that is greater than the target bit rate and has the smallest difference with the target bit rate;

[0021] If not, determine that the encoding path corresponding to the second bit rate among the estimated bit rates is the target encoding path, wherein the second bit rate is the estimated bit rate among the estimated bit rates that is smaller than the target bit rate and has the smallest difference with the target bit rate.

[0022] In one embodiment, before the step of generating the estimated bit rates of the respective encoding paths based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, the method further includes:

[0023] Splicing the first point cloud frame and the second point cloud frame to obtain a spliced ​​point cloud frame;

[0024] Performing coarse-grained convolution processing on the spliced ​​point cloud frames to generate motion embedding;

[0025] performing fine-grained convolution processing on the motion embedding to generate a motion estimate;

[0026] Based at least on the motion embedding and the motion estimation, a motion vector of the first point cloud frame relative to the second point cloud frame is generated.

[0027] In one embodiment, the step of generating encoded data based on the first point cloud features of each of the first point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data includes:

[0028] Based on the motion vector, motion compensation is performed on the second point cloud frame to obtain a predicted point cloud frame corresponding to the first point cloud frame;

[0029] Performing super-a priori encoding processing on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data to obtain super-a priori information;

[0030] Based on the predicted point cloud frames corresponding to the point cloud frames and the super priori information, conditional probability modeling is performed on the first point cloud features and the second point cloud features to obtain respective probability values ​​of the first point cloud features and the second point cloud features;

[0031] Entropy encoding is performed on each of the first point cloud features and the second point cloud features based on each probability value to obtain compressed dynamic point cloud data.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a dynamic point cloud compression device, the dynamic point cloud compression device comprising:

[0033] A bit rate prediction module, for generating an estimated bit rate of each encoding path for any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and a motion vector of the first point cloud frame relative to a second point cloud frame, wherein the second point cloud frame is a point cloud frame previous to the first point cloud frame in the dynamic point cloud data;

[0034] an encoding module, configured to determine a target bit rate of the first point cloud frame, and select a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature;

[0035] The compression module is used to generate compressed dynamic point cloud data based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a program for implementing the dynamic point cloud compression method is stored on the computer-readable storage medium. The program for implementing the dynamic point cloud compression method is executed by a processor to implement the steps of the dynamic point cloud compression method as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, including a computer program, which implements the steps of the dynamic point cloud compression method as described above when executed by a processor.

[0038] The present application provides a dynamic point cloud compression method. The present application generates an estimated bit rate for each encoding path for any point cloud frame (i.e., the first point cloud frame) of each point cloud frame type except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, wherein the second point cloud frame refers to the previous point cloud frame of the first point cloud frame in the dynamic point cloud data, determines the target bit rate of the first point cloud frame, selects the target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame, obtains the first point cloud feature in the first point cloud frame, and then generates compressed dynamic point cloud data based on the first point cloud feature of each point cloud frame and the second point cloud feature obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0039] In summary, compared with the traditional dynamic point cloud compression method that uniformly encodes each point cloud frame based on a fixed bit rate determined by a limited bandwidth, the present application encodes the first point cloud frame of the dynamic point cloud data separately to obtain the second point cloud feature, and predicts the bit rate of each encoding path for each point cloud frame except the first point cloud frame in the dynamic point cloud data, and then selects a suitable target encoding path from the encoding paths with different estimated bit rates based on the target bit rate of each point cloud frame to encode the point cloud frame to obtain the first point cloud feature, and then generates compressed dynamic point cloud data based on the point cloud features of all point cloud frames in the dynamic point cloud data. It should be understood that the bit rates of the optional encoding paths corresponding to a point cloud frame are different, and the corresponding calculation complexities are also different. In this way, the present application can avoid the problems of poor encoding effect and waste of bandwidth resources caused by uniform encoding of all point cloud frames using a fixed bit rate by allocating a suitable encoding path to each point cloud frame. In other words, it can improve the encoding quality of dynamic point cloud data by improving the control accuracy of the bit rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 This is a flow chart of the first embodiment of the dynamic point cloud compression method of the present application;

[0043] Figure 2 A schematic diagram of a dynamic encoding process involved in an embodiment of a dynamic point cloud compression method of the present application;

[0044] Figure 3 A schematic diagram of an inter-frame prediction process involved in an embodiment of a dynamic point cloud compression method of the present application;

[0045] Figure 4 This is a schematic diagram of the overall process of dynamic point cloud compression involved in one embodiment of the dynamic point cloud compression method of the present application;

[0046] Figure 5 This is a schematic diagram of the module structure of the dynamic point cloud compression device of this application;

[0047] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the dynamic point cloud compression method in the embodiment of the present application.

[0048] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0051] The main solution of the present application is: for any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generate an estimated bit rate for each encoding path, wherein the second point cloud frame is the previous point cloud frame of the first point cloud frame in the dynamic point cloud data; determine the target bit rate of the first point cloud frame, and select the target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature; based on the first point cloud features of each point cloud frame and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data, generate compressed dynamic point cloud data.

[0052] At present, the traditional dynamic point cloud compression method is to determine a fixed bit rate based on limited bandwidth, and uniformly encode all point cloud data according to the fixed bit rate. However, this method of uniform encoding with a fixed bit rate lacks consideration of the complexity of the point cloud. For example, high-complexity point cloud frames may have poor encoding effects due to insufficient bit allocation during the encoding process, and low-complexity point cloud frames may waste bandwidth resources due to excessive bit allocation. In other words, this compression method with low bit rate control accuracy may directly lead to a reduction in the overall encoding quality.

[0053] Specifically, the prior art also includes other dynamic point cloud compression technologies, namely non-deep learning dynamic point cloud compression technology and deep learning dynamic point cloud compression technology. Among them, the non-deep learning dynamic point cloud compression technology is V-PCC (Video-based Point Cloud Compression). V-PCC is a technology that converts the three-dimensional point cloud compression problem into a video compression problem. It mainly maps the three-dimensional point cloud data to a two-dimensional plane and uses the existing video coding technology to achieve efficient compression. First, V-PCC decomposes the geometric information and attribute information in the point cloud, and maps the coordinates and color of the three-dimensional point cloud to a two-dimensional image through geometric projection and attribute projection. The two-dimensional image corresponding to the geometric information is called a geometric video frame, and the attribute information generates an attribute video frame. These video frames are then input into a standard video encoder (such as HEVC or AVC) for compression, where the motion estimation and compensation functions of the video encoder can further reduce inter-frame redundancy. In terms of inter-frame redundancy processing, V-PCC uses inter-frame prediction and motion compensation technology to optimize the compression efficiency of point cloud frame sequences. Typically, the first frame is encoded independently as a key frame (Intra frame), and subsequent frames are motion predicted by referring to the information of the previous frame to reduce redundant data. Inter-frame prediction can greatly improve compression efficiency, especially when the point cloud changes between frames are small. This process not only effectively reduces storage requirements, but also provides an efficient solution for dynamic point cloud compression. In the decoding stage, the compressed geometric video frame and attribute video frame need to be decoded first to restore the corresponding two-dimensional image. Then, these two-dimensional images are remapped back to three-dimensional space through the inverse projection process to restore the original point cloud data. In order to ensure the continuity of the point cloud frame sequence, the decoding process will also be adjusted in combination with motion compensation information to correct the point cloud error between frames and finally generate a continuous point cloud sequence. The advantage of V-PCC technology is that it makes full use of the maturity of existing 2D video coding technology, greatly improves compression efficiency, and has strong hardware and software compatibility, which is suitable for expansion on the existing video coding infrastructure. However, since this method relies on projecting three-dimensional data onto a two-dimensional plane, point clouds with complex geometric structures may lose some geometric information during the projection process. In addition, V-PCC has high requirements for computing resources, especially when processing high-resolution or large-scale point clouds, which may affect real-time performance. In general, V-PCC is a technology that can efficiently compress 3D point cloud data, but there is still room for optimization in terms of geometric information retention and real-time processing.

[0054] In addition, deep learning dynamic point cloud compression technology includes D-DPCC (Deep Dynamic Point Cloud Compression, dynamic point cloud compression based on deep learning), which aims to efficiently compress dynamic point cloud data through an end-to-end 3D motion prediction model. The core idea of ​​this method is to use the temporal and spatial redundancy between frames to automatically learn and predict the motion between point clouds through a deep neural network, thereby improving compression efficiency. Specifically, D-DPCC adopts a multi-scale motion prediction mechanism to better capture the geometric changes and motion information between point cloud frames. In the D-DPCC method, the point cloud frame is first input into a neural network model, which performs feature extraction and multi-scale motion prediction on the point cloud data through 3D convolution. In order to more accurately estimate the motion information between frames, D-DPCC uses a motion estimation method based on a latent space to encode the geometric information of the point cloud into a compact latent representation, and perform motion prediction and compensation in this latent space. In this way, D-DPCC can effectively reduce the problem of unclear correspondence between frames caused by the sparsity of point clouds. At the same time, D-DPCC uses a deep learning model to optimize the rate-distortion trade-off, making the compression effect more adaptive and efficient.

[0055] PatchDPCC is a deep compression method for dynamic point clouds. Its core idea is to divide the point cloud frame into multiple small patches (patches) and explore and compress inter-frame redundancy on these local patches. These patches can capture the local features of the point cloud more finely. Through this local partitioning method, patchDPCC can perform more detailed inter-frame redundancy analysis within each patch, improving the accuracy of inter-frame motion estimation and compensation. PatchDPCC performs motion estimation and motion compensation operations on the basis of each patch. Unlike traditional global motion estimation, block-level motion estimation can better capture the local motion characteristics of the point cloud, thereby improving the accuracy of inter-frame prediction. By performing motion estimation in local areas, patchDPCC can effectively cope with local motion changes in large-scale point clouds, such as partial movement or rotation of objects. However, this method requires block partitioning, which results in too long encoding time and cannot meet real-time requirements. Deep learning dynamic point cloud compression technology also includes PatchDPCC (Patch-based Deep Point Cloud Compression), which is a deep compression method for dynamic point clouds. Its core idea is to divide the point cloud frame into multiple small patches (patches) and explore and compress inter-frame redundancy on these local patches. These patches can capture the local features of the point cloud more finely. Through this local partitioning method, patchDPCC can perform more detailed inter-frame redundancy analysis within each patch, improving the accuracy of inter-frame motion estimation and compensation. PatchDPCC performs motion estimation and motion compensation operations on the basis of each patch. Unlike traditional global motion estimation, block-level motion estimation can better capture the local motion features of the point cloud, thereby improving the accuracy of inter-frame prediction. By performing motion estimation in local areas, patchDPCC can effectively cope with local motion changes in large-scale point clouds, such as partial movement or rotation of objects. However, this method requires block partitioning, which results in too long encoding time and cannot meet real-time requirements.

[0056] Compared with the traditional dynamic point cloud compression method that uniformly encodes each point cloud frame based on a fixed bit rate determined by a limited bandwidth, the present application encodes the first point cloud frame of the dynamic point cloud data separately to obtain the second point cloud feature, and predicts the bit rate of each encoding path for each point cloud frame in the dynamic point cloud data except the first point cloud frame, and then selects a suitable target encoding path from the encoding paths with different estimated bit rates based on the target bit rate of each point cloud frame to encode the point cloud frame to obtain the first point cloud feature, and then generates compressed dynamic point cloud data based on the point cloud features of all point cloud frames in the dynamic point cloud data. It should be understood that the bit rates of the optional encoding paths corresponding to a point cloud frame are different, and the corresponding calculation complexities are also different. In this way, the present application can avoid the problems of poor encoding effect and waste of bandwidth resources caused by uniform encoding of all point cloud frames with a fixed bit rate by allocating a suitable encoding path to each point cloud frame. In other words, it can improve the encoding quality of dynamic point cloud data by improving the control accuracy of the bit rate.

[0057] The overall idea of ​​this application is to achieve rate-distortion-complexity optimization (RDCO) through an adaptive dynamic point cloud compression framework to meet the efficient processing requirements of dynamic point cloud compression under different bandwidth and computing resource conditions. The framework supports multiple encoding paths, each path corresponds to a different computational complexity, and can flexibly adjust the compression code rate and the allocation of computing resources through the bit rate control module, so as to achieve efficient compression and decoding of dynamic point clouds in a single model. Specifically, this application introduces a dynamic variational autoencoder to compress the spatial redundancy of point cloud data, and designs multiple encoding paths on this basis. Each path shares part of the computational complexity and achieves the best rate-distortion optimization at different complexity levels. Through the joint training of these encoding paths, the model can achieve optimal compression performance at different complexities. At the same time, in order to deal with the redundancy between frames in dynamic point cloud sequences, this application proposes an inter-frame prediction module from coarse-grained to fine-grained, which performs inter-frame compensation through motion estimation and geometric information deconstruction, thereby expanding the perception field and reducing the calculation time, thereby improving the utilization efficiency of inter-frame redundancy.

[0058] It should be noted that the execution subject of the method in each embodiment of the dynamic point cloud compression method of the present application can be a dynamic point cloud compression system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a dynamic point cloud compression device capable of realizing the above functions, etc., and this embodiment does not specifically limit this. The following takes the dynamic point cloud compression system as the execution subject as an example to illustrate this embodiment and the following embodiments.

[0059] Based on this, this application proposes a dynamic point cloud compression method of the first embodiment, please refer to Figure 1 The dynamic point cloud compression method includes steps S10 to S30:

[0060] Step S10, for any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generating an estimated bit rate of each encoding path, wherein the second point cloud frame is a point cloud frame previous to the first point cloud frame in the dynamic point cloud data;

[0061] It should be noted that any point cloud frame in each point cloud frame except the first point cloud frame (i.e., the first point cloud frame mentioned above) in the dynamic point cloud data to be compressed is referred to as the first point cloud frame for distinction. The first point cloud frame can be understood as the current frame. It should be understood that in a group of point cloud frames, the first frame (I frame) is usually assigned a higher bit rate in order to improve the encoding quality of subsequent frames (P frames), thereby reducing the cumulative error. In other words, the first point cloud frame in the dynamic point cloud data is preferentially assigned a higher bit rate for encoding and does not participate in the selection of the encoding path of the present application. In addition, the embodiment of the present application pre-sets multiple encoding paths, and each point cloud frame corresponds to multiple encoding paths for subsequent selection of appropriate encoding paths for encoding. The point cloud frame located in the previous frame of the first point cloud frame in the dynamic point cloud data is referred to as the second point cloud frame for distinction. The motion vector of the first point cloud frame relative to the second point cloud frame refers to the position change of the corresponding point cloud feature from the second point cloud frame to the first point cloud frame.

[0062] For any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generate the estimated bit rate of each encoding path.

[0063] In this embodiment, the step S10 may include:

[0064] Step S101: input the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame into a preset convolutional neural network model to obtain an estimated bit rate for each encoding path, wherein the convolutional neural network model is used to adaptively adjust the accuracy of the bit rate estimation according to input features.

[0065] It should be noted that the dynamic point cloud compression system of the present application includes a rate control module, which is designed to achieve precise frame-level rate control to ensure that the compression process can be adaptively adjusted when bandwidth is limited and the rate changes. The rate control module includes a rate estimation unit, a bit allocation unit, and a bit implementation unit. Among them, the rate estimation unit is responsible for predicting the rate of each coding path, that is, for each coding path, calculating the estimated rate of each coding path, and when the number of coding paths is K+1, each coding path can be represented as Route0, Route1...Route k The bitrate estimation is completed through a convolutional neural network model, which can adaptively adjust the accuracy of bitrate estimation according to the input features.

[0066] The first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain the estimated bit rates of each encoding path corresponding to the first point cloud frame.

[0067] Exemplarily, when the number of coding paths is 5, each time a point cloud frame and the motion vector of the point cloud frame relative to the previous frame are input into the convolutional neural network model, the estimated bit rates of the five coding paths are obtained. Moreover, it should be understood that the five estimated bit rates output are different, that is, the estimated bit rates of each coding path are different, and accordingly, the computational complexity corresponding to each coding path is also different. Since the bit rate and computational complexity are strongly correlated, it can be understood that the computational complexity of the coding path with a higher bit rate is also greater.

[0068] In this embodiment, the step S101 may include:

[0069] Step S1011, determining the complexity and geometric distribution of the first point cloud frame through the convolutional neural network model, and determining the encoding status of the second point cloud frame based on the motion vector;

[0070] Step S1012, generating an estimated bit rate for each coding path based on the complexity, the geometric distribution and the coding situation through the convolutional neural network model.

[0071] After the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into the convolutional neural network model, the complexity and geometric distribution of the first point cloud frame are analyzed by the convolutional neural network model, and the encoding of the second point cloud frame is analyzed based on the motion vector of the first point cloud frame relative to the second point cloud frame. Then, based on the complexity and geometric distribution of the first point cloud frame and the encoding of the second point cloud frame, an estimated bit rate is generated for different encoding paths.

[0072] Step S20, determining a target bit rate of the first point cloud frame, and selecting a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature;

[0073] It should be noted that the target bit rate of the point cloud frame refers to the expected bit rate set for the point cloud frame.

[0074] After determining the target bit rate of the first point cloud frame, a suitable encoding path (hereinafter referred to as the target encoding path for distinction) is selected from the encoding paths corresponding to the first point cloud frame based on the target bit rate to encode the first point cloud frame to obtain a potential representation in the first point cloud frame (hereinafter referred to as the first point cloud feature for distinction).

[0075] For example, Figure 2The figure shows a schematic diagram of the dynamic encoding process. The dynamic point cloud compression system of the present application includes a dynamic encoder (Dynamic Encoder), a dynamic decoder (Dynamic Decoder) and an adjustable width convolution (Slimmable Convolution), wherein the dynamic encoder is used to encode each point cloud frame, the dynamic decoder is used to decode each encoded code stream, and the adjustable width convolution is used to adjust the channel dimension of the input feature to dynamically adapt to the computing requirements. The dynamic encoder and the dynamic decoder are based on a scalable convolution operator (i.e., adjustable width convolution), which dynamically adapts to the computing requirements by adjusting the channel dimension of the input feature. Specifically, each path (route) of the encoder and the decoder corresponds to a different computational complexity, and these paths are jointly trained to form a multi-level complexity compression network structure (supernet). By decomposing the entire network into several overlapping subnets (subnets), each subnet only handles part of the total computing load. In this way, the system can dynamically select a lower complexity encoding path as needed without sacrificing performance, or select a high complexity path when computing resources are sufficient to obtain higher compression quality. During the encoding process, the current point cloud frame consists of its spatial coordinates and corresponding feature information. The dynamic encoder converts the point cloud data into a latent representation (i.e., point cloud features) through a specific encoding path. Each encoding path performs different convolution operations on the input data to reduce redundancy and compress the data. In this process, the selection of the encoding path is determined by the bitrate control module, which selects the appropriate path based on the bitrate requirements of the current frame and the system resources. For each encoding path, the original point cloud data is downsampled to sparse the coordinates and aggregate local geometric details. These processed data are further reduced through context modeling and entropy coding to reduce redundant information. The decoding process is similar to the encoding process. The decoder converts the compressed latent representation back into the reconstructed point cloud data according to the selected encoding path. During the decoding process, the decoder generates high-quality or low-complexity reconstructed point cloud data corresponding to the encoding path by adjusting the channel dimension and the complexity of the feature map.

[0076] In addition, it is worth emphasizing that all coding paths in the embodiment of the present application are not trained independently, but are optimized through a joint training strategy. Initially, the entire supernet is pre-trained under high bit rate conditions to support higher compression quality. On this basis, each subnetwork is gradually trained under reduced bit rate conditions to ensure that the best rate-distortion performance can be achieved under different bit rate and complexity scenarios. Through this joint optimization, the coding path finds a balance between computational complexity and compression quality, so that the model can achieve flexible dynamic reasoning within a single framework. Since each coding path shares some model parameters, this allows the model to avoid redundant calculations caused by completely independent paths when implementing reasoning of multiple different complexities. This parameter sharing strategy further reduces the overall computational load of the model and improves the resource utilization of the model. Moreover, in the dynamic point cloud compression framework based on multiple coding paths proposed in the present application, each coding path corresponds to a different computational complexity, and the best rate-distortion optimization is achieved at different complexity levels through joint training, thereby achieving adaptive adjustment of compression parameters according to the bandwidth and computing resource limitations of the actual application scenario, so as to achieve efficient point cloud compression within a single model.

[0077] In this embodiment, the step S20 may include:

[0078] Step S201, obtaining a preset expected average bit rate and a preset sliding window length, and determining the number of encoded point cloud frames in the dynamic point cloud data, and determining the total bit rate consumed by the encoded point cloud frames;

[0079] It should be noted that the preset expected average bit rate refers to the maximum average bit rate that can be allocated to each point cloud frame under the current limited bandwidth conditions, which is set by the user in advance according to the limited bandwidth. The preset sliding window length is the sliding window length of the sliding window algorithm used when calculating the target bit rate of each point cloud frame. The total bit rate consumed by the encoded point cloud frame refers to the sum of the bit rates consumed by each encoded point cloud frame.

[0080] Step S202, adding the number of frames to the sliding window length to obtain a sum;

[0081] Step S203: subtract the total bit rate from the sum to obtain a difference, and divide the difference by the sliding window length to obtain a target bit rate of the first point cloud frame.

[0082] After obtaining the preset expected average bit rate and the preset sliding window length, determining the number of encoded point cloud frames in the dynamic point cloud data, and determining the total bit rate consumed by the encoded point cloud frames, determine the sum of the number of frames and the sliding window length, subtract the total bit rate from the sum, then calculate the difference, and finally divide the difference by the sliding window length. The result value is the target bit rate of the first point cloud frame.

[0083] It should be noted that the bit rate control module includes a bit allocation unit, which is used to allocate the target bit rate of each frame through a sliding window algorithm after the bit rate estimation. For example, the target bit rate of the current frame is represented as T tar , then T tar The calculation formula can be expressed as:

[0084]

[0085] Among them, R tar is the expected average bit rate preset by the user, N coded is the number of encoded point cloud frames in the dynamic point cloud data, SW is the sliding window length, R coded The total bitrate consumed for the encoded point cloud frames in the dynamic point cloud data.

[0086] In this way, in order to solve the problem of inaccurate bit rate control in the prior art, the embodiment of the present application designs a bit rate control module based on a sliding window and a bit allocation algorithm, which can adaptively adjust the bit rate during the dynamic point cloud compression process, ensuring the optimal compression performance under limited bandwidth and maintaining the stability of video quality.

[0087] In this embodiment, the step S20 may include:

[0088] Step S204, determining the product of the expected average bit rate and the number of frames, and judging whether the product is greater than the total bit rate;

[0089] Step S205: If yes, determine that the encoding path corresponding to the first bit rate among the estimated bit rates is the target encoding path, wherein the first bit rate is the estimated bit rate among the estimated bit rates that is greater than the target bit rate and has the smallest difference with the target bit rate;

[0090] Step S206: If not, determine that the encoding path corresponding to the second bit rate among the estimated bit rates is the target encoding path, wherein the second bit rate is the estimated bit rate among the estimated bit rates that is smaller than the target bit rate and has the smallest difference with the target bit rate.

[0091] After determining the target bit rate of the first point cloud frame, the product of the expected average bit rate and the number of encoded point cloud frames is calculated, and it is determined whether the product is greater than the total bit rate consumed by the encoded point cloud frames. If so, the estimated bit rate that is greater than the target bit rate and has the smallest difference with the target bit rate (hereinafter referred to as the first bit rate for distinction) among the estimated bit rates is determined, and the encoding path corresponding to the first bit rate is used as the target encoding path to encode the first point cloud frame through the target encoding path; if not, the estimated bit rate that is less than the target bit rate and has the smallest difference with the target bit rate (hereinafter referred to as the second bit rate for distinction) among the estimated bit rates is determined, and the encoding path corresponding to the second bit rate is used as the target encoding path to encode the first point cloud frame through the target encoding path.

[0092] Exemplarily, the path identifier of the target coding path is represented as i. For example, when the target coding path is Route1, i is 1. Then, the calculation formula of i can be expressed as:

[0093]

[0094] in, is the estimated bit rate of coding path i. In this way, the embodiment of the present application selects a suitable target coding path adaptively according to the current coding state through the bit realization unit, thereby ensuring that the point cloud compression effect and the accuracy of bit rate control are improved within the given bit rate limit.

[0095] In this way, the embodiment of the present application solves the problem of inaccurate bit rate control in the prior art. Through an adaptive bit rate prediction and regulation mechanism, it can flexibly allocate the path of the current encoding frame under different bandwidth conditions, ensuring that efficient encoding effects are always maintained in real-time applications and meeting network bandwidth requirements.

[0096] Step S30, generating compressed dynamic point cloud data based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0097] First, the first point cloud frame in the dynamic point cloud data is encoded to obtain the potential representation of the first point cloud frame (hereinafter referred to as the second point cloud feature for distinction), and then each point cloud frame except the first point cloud frame in the dynamic point cloud data is encoded to obtain the first point cloud feature of each point cloud frame, and then the compressed dynamic point cloud data is generated based on the first point cloud feature and the second point cloud feature. It can be understood that the compressed dynamic point cloud data is a code stream.

[0098] In the embodiment of the present application, the first point cloud frame of the dynamic point cloud data is separately encoded to obtain the second point cloud feature, and the code rate of each encoding path is predicted for each point cloud frame in the dynamic point cloud data except the first point cloud frame, and then a suitable target encoding path is selected from the encoding paths with different estimated code rates based on the target code rate of each point cloud frame to encode the point cloud frame to obtain the first point cloud feature, and then compressed dynamic point cloud data is generated based on the point cloud features of all the point cloud frames in the dynamic point cloud data. It should be understood that the code rates of the optional encoding paths corresponding to a point cloud frame are different, and the corresponding calculation complexities are also different. In this way, the embodiment of the present application can avoid the problems of poor encoding effect and waste of bandwidth resources caused by uniform encoding of all point cloud frames with a fixed code rate by allocating a suitable encoding path to each point cloud frame. In other words, it can improve the encoding quality of dynamic point cloud data by improving the control accuracy of the code rate.

[0099] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated later. On this basis, before the step S10, the dynamic point cloud compression method of the present application further includes:

[0100] Step A10, stitching the first point cloud frame and the second point cloud frame to obtain a stitched point cloud frame;

[0101] Step A20, performing coarse-grained convolution processing on the spliced ​​point cloud frame to generate motion embedding;

[0102] Step A30, performing fine-grained convolution processing on the motion embedding to generate a motion estimate;

[0103] Step A40, generating a motion vector of the first point cloud frame relative to the second point cloud frame based at least on the motion embedding and the motion estimation.

[0104] It should be noted that the dynamic point cloud compression system of the present application also includes an inter-frame prediction module, which adopts a coarse-to-fine inter-frame prediction method (Coarse-to-fine Inter-frame Prediction) to improve the efficiency of inter-frame redundancy utilization while reducing computational complexity. This module accurately captures the motion information between dynamic point cloud frames through coarse-grained and fine-grained motion estimation and compensation steps, thereby improving the efficiency and accuracy of point cloud compression.

[0105] Before the bit rate estimation, motion estimation is performed on each adjacent point cloud frame to obtain a motion vector of each point cloud frame relative to the previous point cloud frame. Specifically, the first point cloud frame and the second point cloud frame are first spliced ​​to obtain a spliced ​​point cloud frame, and then the spliced ​​point cloud frame is subjected to coarse-grained convolution processing to preliminarily generate motion embedding, and then the motion embedding is subjected to fine-grained convolution processing to obtain a more accurate motion estimation, and then the motion vector of the first point cloud frame relative to the second point cloud frame is generated based on at least the motion embedding and the motion estimation.

[0106] For example, Figure 3 The figure shows a schematic diagram of the inter-frame prediction process. The inter-frame prediction module includes motion estimation (Motion Estimation), motion compression (Motion Compression) and motion compensation (Motion Compensation). In the motion estimation stage, the inter-frame prediction module operates in the feature space and fuses the temporal information of two consecutive frames. Specifically, the point cloud features of the current frame and the previous frame are first spliced, and then processed by coarse-grained convolution to preliminarily generate motion embedding. In order to ensure the integrity of the motion information, the system further optimizes the motion embedding through fine-grained convolution to generate more accurate motion estimates. In addition, in order to reduce the information loss caused by the deep network, the residual branch is introduced to directly fuse the residual details with the motion embedding, and combine the motion estimation to obtain the final motion vector. This motion vector represents the three-dimensional motion information of each point in the current frame.

[0107] In this way, the embodiment of the present application introduces a coarse-to-fine inter-frame prediction and compensation mechanism, breaking through the bottleneck of the traditional KNN (K-nearest neighbor) method in the limited perception field and low matching accuracy, and can more effectively utilize the redundant information between point cloud frames, greatly improving the balance between compression rate and compression quality. Specifically, the perception field is expanded through motion embedding and geometric information deconstruction to achieve efficient inter-frame compensation. Compared with the existing KNN method, this module significantly reduces the computational complexity of inter-frame prediction and more effectively eliminates redundant information between frames. Moreover, compared with the defects of high computational complexity and poor real-time performance of existing deep learning methods, the embodiment of the present application reduces the consumption of computing resources and improves the compression speed through dynamic encoding paths and lightweight inter-frame prediction algorithms, which is particularly suitable for real-time processing scenarios of large-scale dynamic point cloud data.

[0108] In this embodiment, the step S30 may include:

[0109] Step S301, performing motion compensation on the second point cloud frame based on the motion vector to obtain a predicted point cloud frame corresponding to the first point cloud frame;

[0110] Based on the motion vector of the first point cloud frame relative to the second point cloud frame, motion compensation is performed on the second point cloud frame to obtain a predicted point cloud frame corresponding to the first point cloud frame. The predicted point cloud frame corresponding to each point cloud frame is used as motion compensation information to participate in subsequent conditional probability modeling.

[0111] Exemplarily, in the motion compensation stage, the inter-frame prediction module first quantizes the decoded motion vector to generate a block-level coarse-grained motion vector. This vector is used to preliminarily align the point cloud coordinates of the current frame with the coordinates of the previous frame to reduce the geometric differences between frames. Then, the geometric information deconstruction module processes the point cloud geometric information of the current frame and the previous frame to generate anchor point geometric information. These anchor points provide precise geometric references to help further adjust the point cloud alignment between frames. On this basis, the module generates more refined motion vectors for fine-grained motion compensation, and the result of motion compensation is a predicted frame. For example, the predicted frame corresponding to the current frame is determined based on the motion vector of the current frame relative to the previous frame and the point cloud frame of the previous frame. By refining the motion vector layer by layer, the system can accurately align the point cloud features between frames, thereby greatly improving the accuracy of inter-frame prediction.

[0112] Step S302, performing super-a priori encoding processing on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data to obtain super-a priori information;

[0113] The second point cloud feature corresponding to the first point cloud frame in the dynamic point cloud data and the first point cloud feature of each point cloud frame except the first point cloud frame are subjected to super-prior coding processing to obtain super-prior information. It can be understood that the super-prior information refers to the potential representation of each point cloud feature obtained by encoding each point cloud feature.

[0114] Step S303, based on the predicted point cloud frames corresponding to each of the point cloud frames and the super priori information, conditional probability modeling is performed on each of the first point cloud features and the second point cloud features to obtain respective probability values ​​of each of the first point cloud features and the second point cloud features;

[0115] Based on the predicted point cloud frames and the super priori information corresponding to each point cloud frame, conditional probability modeling is performed on each first point cloud feature and the second point cloud feature to obtain respective probability values ​​of each first point cloud feature and the second point cloud feature.

[0116] Step S304: performing entropy coding on each of the first point cloud features and the second point cloud features based on each probability value to obtain compressed dynamic point cloud data.

[0117] Based on the probability value of each point cloud feature, each point cloud feature is entropy encoded to obtain compressed dynamic point cloud data, that is, code stream.

[0118] It should be noted that the dynamic point cloud compression system of the present application also includes a conditional coding module, which makes full use of the geometric redundancy of point cloud data by combining temporal and spatial context information to improve compression efficiency and quality. Specifically, conditional coding first extracts context from the motion compensation information obtained from inter-frame prediction and spatial hyper-prior information, which is used to perform conditional probability modeling on the potential representation in the point cloud frame. Through this conditional probability model, the geometric structure and attribute information of the point cloud can be encoded more accurately, further reducing redundancy.

[0119] Exemplarily, in the conditional coding module, the temporal context and hyper-prior context are first extracted, and then combined through an entropy parameters network to generate an overall context. The overall context is used to guide the probability distribution modeling of the potential representation, thereby realizing conditional coding. Specifically, the probability distribution of the potential representation is modeled as a decomposed Gaussian distribution combined with soft quantization noise. This modeling method can effectively reduce the entropy value of the potential representation and reduce the bit rate consumption during the encoding process. In this process, the rate-distortion loss consists of multiple parts, including motion embedding, hyper-prior information, and encoding bit rate loss of the potential representation. Through end-to-end training, the model can significantly improve the accuracy of reconstruction while maintaining a low bit rate.

[0120] like Figure 4The figure shows the overall process diagram of dynamic point cloud compression. First, bit estimation is performed, that is, the bit rate of each encoding path corresponding to each point cloud frame is estimated; at the same time, the expected average bit rate set by the user, that is, the target bit rate (Target Bitrate Rtar), is determined, and bit allocation is performed based on the expected average bit rate to obtain the target bit rate of each point cloud frame; then, bit implementation is performed based on each estimated bit rate and the target bit rate of the point cloud frame; then, the point cloud frame is encoded by the dynamic encoder in the dynamic inference route to obtain the potential representation; then, the potential representation of the current frame and the previous frame is motion estimated and motion compressed to obtain the temporal context of the current frame, that is, inter-frame prediction; then, conditional coding is performed. Specifically, the potential representation of the current frame is super-encoded by the hyper encoder to obtain the potential representation of the point cloud features of the current frame, and the potential representation of the point cloud features is input into the hyper decoder. Decoder) to obtain the super prior information; input the time context, super prior information input and entropy parameters into the context model to obtain the compressed bit stream.

[0121] It should be noted that insufficient utilization of inter-frame redundancy is one of the main bottlenecks of current technology. Most existing methods rely on KNN-based motion estimation and compensation mechanisms. Although these methods can predict the motion between point cloud frames in the latent space, the sparsity of the point cloud data itself makes it difficult to match corresponding points, and the perception field is limited, which ultimately affects the accuracy of the prediction. This inefficient use of inter-frame redundancy directly leads to the difficulty in optimizing the balance between compression rate and quality. However, the embodiment of the present application improves the compression efficiency while further reducing the point cloud redundancy through the above-mentioned conditional coding mechanism, ensuring the compression quality and real-time performance at different complexities.

[0122] The present application also provides a dynamic point cloud compression device. Figure 5 , the dynamic point cloud compression device comprises:

[0123] The bit rate prediction module 10 is used to generate an estimated bit rate of each encoding path for any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, wherein the second point cloud frame is a point cloud frame previous to the first point cloud frame in the dynamic point cloud data;

[0124] The encoding module 20 is used to determine a target bit rate of the first point cloud frame, and select a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature;

[0125] The compression module 30 is used to generate compressed dynamic point cloud data based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0126] Optionally, the bit rate prediction module 10 is further used for:

[0127] The first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain an estimated bit rate for each encoding path, wherein the convolutional neural network model is used to adaptively adjust the accuracy of the bit rate estimation according to the input features.

[0128] Optionally, the bit rate prediction module 10 is further used for:

[0129] Determining the complexity and geometric distribution of the first point cloud frame by using the convolutional neural network model, and determining the encoding of the second point cloud frame based on the motion vector;

[0130] The convolutional neural network model generates an estimated bit rate for each coding path based on the complexity, the geometric distribution and the coding situation.

[0131] Optionally, the encoding module 20 is further used for:

[0132] Obtaining a preset expected average bit rate and a preset sliding window length, and determining the number of encoded point cloud frames in the dynamic point cloud data, and determining a total bit rate consumed by the encoded point cloud frames;

[0133] Add the number of frames to the sliding window length to obtain a sum;

[0134] The total bit rate is subtracted from the sum to obtain a difference, and the difference is divided by the sliding window length to obtain a target bit rate of the first point cloud frame.

[0135] Optionally, the encoding module 20 is further used for:

[0136] Determine the product of the expected average bit rate and the number of frames, and determine whether the product is greater than the total bit rate;

[0137] If so, determining that a coding path corresponding to a first bit rate among the estimated bit rates is a target coding path, wherein the first bit rate is an estimated bit rate among the estimated bit rates that is greater than the target bit rate and has the smallest difference with the target bit rate;

[0138] If not, determine that the encoding path corresponding to the second bit rate among the estimated bit rates is the target encoding path, wherein the second bit rate is the estimated bit rate among the estimated bit rates that is smaller than the target bit rate and has the smallest difference with the target bit rate.

[0139] Optionally, the dynamic point cloud compression device further includes a motion estimation module, and the motion estimation module is used to:

[0140] Splicing the first point cloud frame and the second point cloud frame to obtain a spliced ​​point cloud frame;

[0141] Performing coarse-grained convolution processing on the spliced ​​point cloud frames to generate motion embedding;

[0142] performing fine-grained convolution processing on the motion embedding to generate a motion estimate;

[0143] Based at least on the motion embedding and the motion estimation, a motion vector of the first point cloud frame relative to the second point cloud frame is generated.

[0144] Optionally, the compression module 30 is further used for:

[0145] Based on the motion vector, motion compensation is performed on the second point cloud frame to obtain a predicted point cloud frame corresponding to the first point cloud frame;

[0146] Performing super-a priori encoding processing on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data to obtain super-a priori information;

[0147] Based on the predicted point cloud frames corresponding to the point cloud frames and the super priori information, conditional probability modeling is performed on the first point cloud features and the second point cloud features to obtain respective probability values ​​of the first point cloud features and the second point cloud features;

[0148] Entropy encoding is performed on each of the first point cloud features and the second point cloud features based on each probability value to obtain compressed dynamic point cloud data.

[0149] The dynamic point cloud compression device provided in the embodiment of the present application adopts the dynamic point cloud compression method in the above embodiment, which can solve the technical problem of how to improve the bit rate control accuracy during dynamic point cloud compression to improve the encoding quality. Compared with the prior art, the beneficial effects of the dynamic point cloud compression device provided in the embodiment of the present application are the same as the beneficial effects of the dynamic point cloud compression method provided in the above embodiment, and other technical features in the dynamic point cloud compression device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0150] The present application provides a dynamic point cloud compression device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the dynamic point cloud compression method in the above-mentioned embodiment one.

[0151] Reference below Figure 6 , which shows a schematic diagram of the structure of a dynamic point cloud compression device suitable for implementing the embodiment of the present application. The dynamic point cloud compression device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The dynamic point cloud compression device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0152] like Figure 6As shown, the dynamic point cloud compression device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the dynamic point cloud compression device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the dynamic point cloud compression device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a dynamic point cloud compression device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0153] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0154] The dynamic point cloud compression device provided by the present application adopts the dynamic point cloud compression method in the above embodiment, which can solve the technical problem of how to improve the bit rate control accuracy during dynamic point cloud compression to improve the encoding quality. Compared with the prior art, the beneficial effects of the dynamic point cloud compression device provided by the present application are the same as the beneficial effects of the dynamic point cloud compression method provided by the above embodiment, and the other technical features in the dynamic point cloud compression device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0155] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0156] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0157] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the dynamic point cloud compression method in the above-mentioned embodiment.

[0158] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0159] The computer-readable storage medium may be included in the dynamic point cloud compression device; or may exist independently without being assembled into the dynamic point cloud compression device.

[0160] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the dynamic point cloud compression device, the dynamic point cloud compression device: for any first point cloud frame in each point cloud frame except the first frame point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generate the estimated bit rate of each encoding path, wherein the second point cloud frame is the previous point cloud frame of the first point cloud frame in the dynamic point cloud data; determine the target bit rate of the first point cloud frame, and select the target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature; based on the first point cloud feature of each point cloud frame and the second point cloud feature obtained by encoding the first frame point cloud frame in the dynamic point cloud data, generate compressed dynamic point cloud data.

[0161] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0164] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned dynamic point cloud compression method, and can solve the technical problem of how to improve the bit rate control accuracy during dynamic point cloud compression to improve the encoding quality. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the dynamic point cloud compression method provided in the above-mentioned embodiment, and will not be repeated here.

[0165] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the dynamic point cloud compression method as described above when the computer program is executed by a processor.

[0166] The computer program product provided in this application can improve the bit rate control accuracy during dynamic point cloud compression to improve the encoding quality. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the dynamic point cloud compression method provided in the above embodiment, which will not be repeated here.

[0167] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A dynamic point cloud compression method, characterized in that: The dynamic point cloud compression method comprises: For any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generate the estimated bit rate of each encoding path, wherein the second point cloud frame is a point cloud frame previous to the first point cloud frame in the dynamic point cloud data; Determine a target bit rate of the first point cloud frame, and select a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature; Compressed dynamic point cloud data are generated based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

2. The method according to claim 1, characterized in that The step of generating an estimated bit rate of each encoding path based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame includes: The first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain an estimated bit rate for each encoding path, wherein the convolutional neural network model is used to adaptively adjust the accuracy of the bit rate estimation according to the input features.

3. The method according to claim 2, characterized in that The step of obtaining the estimated bit rate of each encoding path includes: Determining the complexity and geometric distribution of the first point cloud frame by using the convolutional neural network model, and determining the encoding of the second point cloud frame based on the motion vector; The convolutional neural network model generates an estimated bit rate for each coding path based on the complexity, the geometric distribution and the coding situation.

4. The method according to claim 1, characterized in that The step of determining a target bit rate of the first point cloud frame comprises: Obtaining a preset expected average bit rate and a preset sliding window length, and determining the number of encoded point cloud frames in the dynamic point cloud data, and determining a total bit rate consumed by the encoded point cloud frames; Add the number of frames to the sliding window length to obtain a sum; The total bit rate is subtracted from the sum to obtain a difference, and the difference is divided by the sliding window length to obtain a target bit rate of the first point cloud frame.

5. The method according to claim 4, characterized in that The step of selecting a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame includes: Determine the product of the expected average bit rate and the number of frames, and determine whether the product is greater than the total bit rate; If so, determining that a coding path corresponding to a first bit rate among the estimated bit rates is a target coding path, wherein the first bit rate is an estimated bit rate among the estimated bit rates that is greater than the target bit rate and has the smallest difference with the target bit rate; If not, determine that the encoding path corresponding to the second bit rate among the estimated bit rates is the target encoding path, wherein the second bit rate is the estimated bit rate among the estimated bit rates that is smaller than the target bit rate and has the smallest difference with the target bit rate.

6. The method according to claim 1, characterized in that Before the step of generating the estimated bit rates of the respective encoding paths based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, the method further includes: Splicing the first point cloud frame and the second point cloud frame to obtain a spliced ​​point cloud frame; Performing coarse-grained convolution processing on the spliced ​​point cloud frames to generate motion embedding; performing fine-grained convolution processing on the motion embedding to generate a motion estimate; Based at least on the motion embedding and the motion estimation, a motion vector of the first point cloud frame relative to the second point cloud frame is generated.

7. The method according to any one of claims 1 to 6, characterized in that The step of generating encoded data based on the first point cloud features of each of the first point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data comprises: Based on the motion vector, motion compensation is performed on the second point cloud frame to obtain a predicted point cloud frame corresponding to the first point cloud frame; Performing super-a priori encoding processing on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data to obtain super-a priori information; Based on the predicted point cloud frames corresponding to the point cloud frames and the super priori information, conditional probability modeling is performed on the first point cloud features and the second point cloud features to obtain respective probability values ​​of the first point cloud features and the second point cloud features; Entropy encoding is performed on each of the first point cloud features and the second point cloud features based on each probability value to obtain compressed dynamic point cloud data.

8. A dynamic point cloud compression device, characterized in that: The dynamic point cloud compression device comprises: A bit rate prediction module, for generating an estimated bit rate of each encoding path for any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and a motion vector of the first point cloud frame relative to a second point cloud frame, wherein the second point cloud frame is a point cloud frame previous to the first point cloud frame in the dynamic point cloud data; an encoding module, configured to determine a target bit rate of the first point cloud frame, and select a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature; The compression module is used to generate compressed dynamic point cloud data based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

9. A dynamic point cloud compression device, characterized in that: The dynamic point cloud compression device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the dynamic point cloud compression method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the dynamic point cloud compression method according to any one of claims 1 to 7 are implemented.

Images