Dynamic point cloud compression method, device and equipment and computer readable storage medium

Abstract

The application discloses a dynamic point cloud compression method and device, equipment and a computer readable storage medium, and relates to the technical field of data processing. The method comprises the following steps: for any first point cloud frame in each point cloud frame except the first frame point cloud frame in the dynamic point cloud data to be compressed, generating an estimated code rate of each coding path based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame; determining a target code rate of the first point cloud frame, selecting a target coding path in each coding path based on the target code rate to encode the first point cloud frame, and obtaining first point cloud features; and generating compressed dynamic point cloud data based on the first point cloud features of each point cloud frame and second point cloud features obtained by encoding the first frame point cloud frame in the dynamic point cloud data. The application can improve the code rate control accuracy during dynamic point cloud compression, thereby improving the coding quality.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a dynamic point cloud compression method, apparatus, device, and computer-readable storage medium. Background Art

[0002] With the rapid development of 3D scanning and sensor technologies, point cloud data, as an important form of 3D information representation, has been widely used in various fields such as virtual reality, autonomous driving, telemedicine, and digital entertainment. In order to efficiently store and transmit this point cloud data, dynamic point cloud compression technology is particularly important.

[0003] Currently, the traditional dynamic point cloud compression method is to determine a fixed bit rate based on limited bandwidth and uniformly encode all point cloud data according to this fixed bit rate. However, this method of uniform encoding with a fixed bit rate lacks consideration of the complexity of the point cloud. For example, high-complexity point cloud frames may have poor encoding effects due to insufficient bit allocation during the encoding process, and low-complexity point cloud frames may have excessive bit allocation, resulting in a waste of bandwidth resources. In other words, this compression method with low bit rate control accuracy may directly lead to a decrease in overall encoding quality.

[0004] Therefore, how to improve the bit rate control accuracy during dynamic point cloud compression to improve encoding quality is a problem that needs to be solved urgently. Summary of the Invention

[0005] The main purpose of this application is to provide a dynamic point cloud compression method, device, equipment and computer-readable storage medium, aiming to improve the bit rate control accuracy during dynamic point cloud compression to improve the encoding quality.

[0006] To achieve the above-mentioned object, the present application provides a dynamic point cloud compression method, the dynamic point cloud compression method comprising: for any first point cloud frame in each point cloud frame other than a first point cloud frame in dynamic point cloud data to be compressed, generating an estimated bit rate for each encoding path based on the first point cloud frame and a motion vector of the first point cloud frame relative to a second point cloud frame, wherein the second point cloud frame is a point cloud frame preceding the first point cloud frame in the dynamic point cloud data;

[0007] determining a target bit rate for the first point cloud frame, and selecting a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature;

[0008] Compressed dynamic point cloud data is generated based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0009] In one embodiment, the step of generating an estimated bit rate for each encoding path based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame includes:

[0010] The first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain the estimated bit rate of each encoding path, wherein the convolutional neural network model is used to adaptively adjust the accuracy of the bit rate estimation according to the input features.

[0011] In one embodiment, the step of obtaining the estimated bit rate of each encoding path includes:

[0012] Determining the complexity and geometric distribution of the first point cloud frame through the convolutional neural network model, and determining the encoding status of the second point cloud frame based on the motion vector;

[0013] The convolutional neural network model generates an estimated bit rate for each coding path based on the complexity, the geometric distribution and the coding situation.

[0014] In one embodiment, the step of determining the target bit rate of the first point cloud frame includes:

[0015] Obtaining a preset expected average bit rate and a preset sliding window length, and determining the number of encoded point cloud frames in the dynamic point cloud data, and determining a total bit rate consumed by the encoded point cloud frames;

[0016] Adding the number of frames to the sliding window length to obtain a sum;

[0017] The total bit rate is subtracted from the sum to obtain a difference, and the difference is divided by the sliding window length to obtain a target bit rate for the first point cloud frame.

[0018] In one embodiment, the step of selecting a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame includes:

[0019] Determining a product of the expected average bit rate and the number of frames, and determining whether the product is greater than the total bit rate;

[0020] If so, determining a coding path corresponding to a first bit rate among the estimated bit rates as the target coding path, wherein the first bit rate is an estimated bit rate that is greater than the target bit rate and has the smallest difference with the target bit rate among the estimated bit rates;

[0021] If not, determine that the encoding path corresponding to the second bit rate among each of the estimated bit rates is the target encoding path, wherein the second bit rate is the estimated bit rate among each of the estimated bit rates that is smaller than the target bit rate and has the smallest difference with the target bit rate.

[0022] In one embodiment, before the step of generating an estimated bit rate for each encoding path based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, the method further includes:

[0023] Splicing the first point cloud frame and the second point cloud frame to obtain a spliced ​​point cloud frame;

[0024] Performing coarse-grained convolution processing on the spliced ​​point cloud frames to generate motion embedding;

[0025] performing fine-grained convolution on the motion embedding to generate a motion estimate;

[0026] Based at least on the motion embedding and the motion estimation, a motion vector of the first point cloud frame relative to the second point cloud frame is generated.

[0027] In one embodiment, the step of generating encoded data based on the first point cloud features of each of the first point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data includes:

[0028] Based on the motion vector, motion compensation is performed on the second point cloud frame to obtain a predicted point cloud frame corresponding to the first point cloud frame;

[0029] Performing super-prior encoding processing on the first point cloud feature of each of the point cloud frames and the second point cloud feature obtained by encoding the first point cloud frame in the dynamic point cloud data to obtain super-prior information;

[0030] Based on the predicted point cloud frames corresponding to the respective point cloud frames and the super priori information, conditional probability modeling is performed on the respective first point cloud features and the second point cloud features to obtain respective probability values ​​of the respective first point cloud features and the second point cloud features;

[0031] Entropy encoding is performed on each of the first point cloud features and the second point cloud features based on each probability value to obtain compressed dynamic point cloud data.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a dynamic point cloud compression device, which includes:

[0033] a bit rate prediction module, configured to generate, for each encoding path, an estimated bit rate for any first point cloud frame among the point cloud frames to be compressed except the first point cloud frame, based on the first point cloud frame and a motion vector of the first point cloud frame relative to a second point cloud frame, wherein the second point cloud frame is a point cloud frame preceding the first point cloud frame in the dynamic point cloud data;

[0034] an encoding module, configured to determine a target bit rate for the first point cloud frame, and select a target encoding path from among the encoding paths based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature;

[0035] The compression module is used to generate compressed dynamic point cloud data based on the first point cloud features of each point cloud frame and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores a program for implementing the dynamic point cloud compression method. The program for implementing the dynamic point cloud compression method is executed by the processor to implement the steps of the dynamic point cloud compression method as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned dynamic point cloud compression method when executed by a processor.

[0038] The present application provides a dynamic point cloud compression method. The present application generates an estimated bit rate for each encoding path for any point cloud frame (i.e., the first point cloud frame) of each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, wherein the second point cloud frame refers to the point cloud frame previous to the first point cloud frame in the dynamic point cloud data, determines the target bit rate of the first point cloud frame, selects the target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame, obtains the first point cloud feature in the first point cloud frame, and then generates compressed dynamic point cloud data based on the first point cloud feature of each point cloud frame and the second point cloud feature obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0039] In summary, compared to the traditional dynamic point cloud compression method that uniformly encodes each point cloud frame based on a fixed bit rate determined by a limited bandwidth, the present application separately encodes the first point cloud frame of the dynamic point cloud data to obtain the second point cloud feature, and separately predicts the bit rate of each encoding path for each point cloud frame in the dynamic point cloud data except the first point cloud frame. Then, based on the target bit rate of each point cloud frame, a suitable target encoding path is selected from the encoding paths with different estimated bit rates to encode the point cloud frame to obtain the first point cloud feature. Then, based on the point cloud features of each point cloud frame in the dynamic point cloud data, compressed dynamic point cloud data is generated. It should be understood that the bit rates corresponding to the optional encoding paths of a point cloud frame are different, and the corresponding computational complexity is also different. In this way, the present application can avoid the problems of poor encoding effect and waste of bandwidth resources caused by uniform encoding of all point cloud frames with a fixed bit rate by assigning a suitable encoding path to each point cloud frame. In other words, it can improve the encoding quality of dynamic point cloud data by improving the accuracy of bit rate control. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 This is a flow chart of the first embodiment of the dynamic point cloud compression method of the present application;

[0043] Figure 2 This is a schematic diagram of a dynamic encoding process involved in an embodiment of the dynamic point cloud compression method of the present application;

[0044] Figure 3 This is a schematic diagram of the inter-frame prediction process involved in one embodiment of the dynamic point cloud compression method of the present application;

[0045] Figure 4 This is a schematic diagram of the overall process of dynamic point cloud compression involved in one embodiment of the dynamic point cloud compression method of the present application;

[0046] Figure 5 This is a schematic diagram of the module structure of the dynamic point cloud compression device of this application;

[0047] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the dynamic point cloud compression method in the embodiment of the present application.

[0048] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0051] The main solution of the present application is: for any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generate an estimated bit rate for each encoding path, wherein the second point cloud frame is the previous point cloud frame of the first point cloud frame in the dynamic point cloud data; determine the target bit rate of the first point cloud frame, and select the target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature; based on the first point cloud feature of each point cloud frame and the second point cloud feature obtained by encoding the first point cloud frame in the dynamic point cloud data, generate compressed dynamic point cloud data.

[0052] Currently, the traditional dynamic point cloud compression method is to determine a fixed bit rate based on limited bandwidth and uniformly encode all point cloud data according to this fixed bit rate. However, this method of uniform encoding with a fixed bit rate lacks consideration of the complexity of the point cloud. For example, high-complexity point cloud frames may have poor encoding effects due to insufficient bit allocation during the encoding process, and low-complexity point cloud frames may have excessive bit allocation, resulting in a waste of bandwidth resources. In other words, this compression method with low bit rate control accuracy may directly lead to a decrease in overall encoding quality.

[0053] Specifically, existing technologies also include other dynamic point cloud compression technologies, including non-deep learning-based dynamic point cloud compression technologies and deep learning-based dynamic point cloud compression technologies. Among these technologies, V-PCC (Video-based Point Cloud Compression) transforms the 3D point cloud compression problem into a video compression problem. It primarily achieves efficient compression by mapping 3D point cloud data onto a 2D plane and leveraging existing video coding techniques. First, V-PCC decomposes the geometric and attribute information in the point cloud, mapping the 3D point cloud's coordinates, color, and other attributes into a 2D image through geometric and attribute projections. The 2D image corresponding to the geometric information is called a geometric video frame, while the attribute information generates an attribute video frame. These video frames are then input into a standard video encoder (such as HEVC or AVC) for compression, where the video encoder's motion estimation and compensation functions further reduce inter-frame redundancy. To address inter-frame redundancy, V-PCC employs inter-frame prediction and motion compensation techniques to optimize the compression efficiency of point cloud frame sequences. Typically, the first frame is encoded independently as a keyframe (Intra frame), while subsequent frames use motion prediction based on the previous frame's information to reduce redundant data. Inter-frame prediction can significantly improve compression efficiency, especially when the point cloud changes between frames are small. This process not only effectively reduces storage requirements but also provides an efficient solution for dynamic point cloud compression. During the decoding phase, the compressed geometric and attribute video frames must first be decoded to recover the corresponding 2D images. These 2D images are then re-projected back into 3D space through a back-projection process to recover the original point cloud data. To ensure the coherence of the point cloud frame sequence, the decoding process also incorporates motion compensation information to correct for point cloud errors between frames, ultimately generating a continuous point cloud sequence. The advantage of V-PCC technology lies in its ability to leverage the maturity of existing 2D video coding technology, significantly improving compression efficiency. Its strong hardware and software compatibility makes it suitable for scaling within existing video coding infrastructure. However, because this method relies on projecting 3D data onto a 2D plane, point clouds with complex geometric structures may lose some geometric information during the projection process. Furthermore, V-PCC places high demands on computing resources, particularly when processing high-resolution or large-scale point clouds, which can affect real-time performance. Overall, while V-PCC is a technology that can efficiently compress 3D point cloud data, there is still room for improvement in terms of geometric information preservation and real-time processing.

[0054] And, the deep learning dynamic point cloud compression technology includes D-DPCC (Deep Dynamic Point Cloud Compression, dynamic point cloud compression based on deep learning), which aims to efficiently compress dynamic point cloud data through an end-to-end 3D motion prediction model. The core idea of this method is to utilize the temporal and spatial redundancy between frames to automatically learn and predict the motion between point clouds through a deep neural network, thereby improving compression efficiency. Specifically, D-DPCC adopts a multi-scale motion prediction mechanism to better capture the geometric changes and motion information between point cloud frames. In the D-DPCC method, the point cloud frames are first input into a neural network model, which extracts features and performs multi-scale motion prediction on the point cloud data through 3D convolution. In order to more accurately estimate the inter-frame motion information, D-DPCC uses a latent space-based motion estimation method to encode the geometric information of the point cloud into a compact latent representation and perform motion prediction and compensation in this latent space. Through such a method, D-DPCC can effectively reduce the problem of unclear inter-frame correspondence caused by the sparsity of point clouds. At the same time, D-DPCC uses a deep learning model to optimize the rate-distortion trade-off, making the compression effect more adaptive and efficient.

[0055] PatchDPCC is a deep compression method for dynamic point clouds. Its core concept is to divide the point cloud frame into multiple small patches (patches) and explore and compress inter-frame redundancy within these local patches. These patches can more finely capture the local features of the point cloud. Through this local partitioning method, patchDPCC can perform more detailed inter-frame redundancy analysis within each patch, improving the accuracy of inter-frame motion estimation and compensation. PatchDPCC performs motion estimation and motion compensation operations on a per-patch basis. Unlike traditional global motion estimation, block-level motion estimation can better capture the local motion characteristics of the point cloud, thereby improving the accuracy of inter-frame prediction. By performing motion estimation in local areas, patchDPCC can effectively cope with local motion changes in large-scale point clouds, such as partial movement or rotation of objects. However, this method requires block partitioning, which results in excessive encoding time and cannot meet real-time requirements. Deep learning dynamic point cloud compression technology also includes PatchDPCC (Patch-based Deep Point Cloud Compression), a deep compression method for dynamic point clouds. Its core concept is to divide the point cloud frame into multiple small patches (patches) and explore and compress inter-frame redundancy within these local patches. These patches can more finely capture the local features of the point cloud. This local partitioning method allows patchDPCC to perform more detailed inter-frame redundancy analysis within each patch, improving the accuracy of inter-frame motion estimation and compensation. PatchDPCC performs motion estimation and compensation on a per-patch basis. Unlike traditional global motion estimation, block-level motion estimation better captures local motion characteristics of the point cloud, thereby improving the accuracy of inter-frame prediction. By performing motion estimation in local areas, patchDPCC can effectively cope with local motion changes in large point clouds, such as partial movement or rotation of objects. However, this method requires patch partitioning, which results in excessive encoding time and cannot meet real-time requirements.

[0056] Compared with a traditional dynamic point cloud compression method of uniformly encoding each point cloud frame based on a fixed code rate determined according to a limited bandwidth, the first frame of point cloud frames of the dynamic point cloud data is separately encoded to obtain second point cloud features, and then the code rate of each encoding path is predicted for each point cloud frame except the first frame of point cloud frames in the dynamic point cloud data. Then, a suitable target encoding path is selected from the encoding paths with different estimated code rates based on the target code rate of each point cloud frame to encode the point cloud frame, to obtain first point cloud features. Then, the compressed dynamic point cloud data is generated based on the point cloud features of all point cloud frames in the dynamic point cloud data. It should be understood that the code rates of the optional encoding paths corresponding to one point cloud frame are different, and the corresponding calculation complexities are also different. Therefore, the application can assign an appropriate encoding path to each point cloud frame, thereby avoiding the problems of poor encoding effect and waste of bandwidth resources caused by uniformly encoding all point cloud frames with a fixed code rate. That is, the control accuracy of the code rate can be improved to improve the encoding quality of the dynamic point cloud data.

[0057] The general idea of the application is to realize rate-distortion-complexity optimization (RDCO) through an adaptive dynamic point cloud compression framework to meet the efficient processing needs of dynamic point cloud compression under different bandwidth and computing resource conditions. The framework supports multiple encoding paths, each path corresponding to different computing complexity, and can flexibly adjust the allocation of compression code rate and computing resources through a code rate control module, thereby realizing efficient compression and decoding of dynamic point cloud in a single model. Specifically, the application introduces a dynamic variational autoencoder to compress the spatial redundancy of point cloud data, and designs multiple encoding paths based on this. Each path shares part of the computing complexity and realizes the best rate-distortion optimization at different complexity levels. Through the joint training of these encoding paths, the model can realize the optimal compression performance at different complexities. At the same time, in order to cope with the inter-frame redundancy in the dynamic point cloud sequence, the application proposes an inter-frame prediction module from coarse to fine granularity. The module performs inter-frame compensation through motion estimation and geometric information decomposition, expands the perception field, reduces the calculation time, and thus improves the utilization efficiency of inter-frame redundancy.

[0058] It should be noted that the execution subject of the method in each embodiment of the dynamic point cloud compression method of the application can be a dynamic point cloud compression system, a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a dynamic point cloud compression device capable of realizing the above functions, etc. The embodiments are not limited specifically. The following takes the dynamic point cloud compression system as an example to illustrate the embodiments and the following embodiments.

[0059] Based on this, this application proposes a dynamic point cloud compression method of the first embodiment, please refer to Figure 1 The dynamic point cloud compression method includes steps S10 to S30:

[0060] Step S10: for any first point cloud frame among the point cloud frames to be compressed except the first point cloud frame in the dynamic point cloud data, generating an estimated bit rate for each encoding path based on the first point cloud frame and a motion vector of the first point cloud frame relative to a second point cloud frame, wherein the second point cloud frame is a point cloud frame preceding the first point cloud frame in the dynamic point cloud data;

[0061] It should be noted that any point cloud frame in each point cloud frame other than the first point cloud frame (i.e. the above-mentioned first point cloud frame) in the dynamic point cloud data to be compressed is referred to as the first point cloud frame for distinction. The first point cloud frame can be understood as the current frame. It should be understood that in a group of point cloud frames, the first frame (I frame) is usually assigned a higher bit rate in order to improve the encoding quality of subsequent frames (P frames), thereby reducing the cumulative error. In other words, the first point cloud frame in the dynamic point cloud data is preferentially assigned a higher bit rate for encoding and does not participate in the selection of the encoding path of this application. In addition, the embodiment of the present application pre-sets multiple encoding paths, and each point cloud frame corresponds to multiple encoding paths for subsequent selection of a suitable encoding path for encoding. The point cloud frame located in the previous frame of the first point cloud frame in the dynamic point cloud data is referred to as the second point cloud frame for distinction. The motion vector of the first point cloud frame relative to the second point cloud frame refers to the position change of the corresponding point cloud feature from the second point cloud frame to the first point cloud frame.

[0062] For any first point cloud frame in each point cloud frame except the first point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, an estimated bit rate of each encoding path is generated.

[0063] In this embodiment, step S10 may include:

[0064] In step S101, the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain an estimated bit rate for each encoding path, wherein the convolutional neural network model is used to adaptively adjust the accuracy of the bit rate estimation according to the input features.

[0065] It should be noted that the dynamic point cloud compression system of the present application includes a code rate control module, which is designed to achieve accurate frame-level code rate control to ensure that the compression process can be adaptively adjusted in the case of limited bandwidth and code rate change. The code rate control module includes a code rate estimation unit, a bit allocation unit and a bit realization unit. Among them, the code rate estimation unit is responsible for predicting the code rate of each encoding path, that is, for each encoding path, the estimated code rate of each encoding path is calculated, and when the number of encoding paths is K+1, each encoding path can be represented as Route0, Route1…Route k Code rate estimation is completed through a convolutional neural network model, which can adaptively adjust the accuracy of code rate estimation according to input features.

[0066] The first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain the estimated code rate of each encoding path corresponding to the first point cloud frame.

[0067] For example, when the number of encoding paths is 5, each input of a point cloud frame and the motion vector of the point cloud frame relative to the previous frame into the convolutional neural network model obtains the estimated code rate of each encoding path. It should be understood that the output of the five estimated code rates is different, that is, the estimated code rate of each encoding path is different, and accordingly, the calculation complexity corresponding to each encoding path is also different. Since the code rate and the calculation complexity are strongly related, it can be understood that the greater the code rate of the encoding path, the greater the calculation complexity of the encoding path.

[0068] In the present embodiment, the step S101 can include:

[0069] Step S1011, determining the complexity and geometric distribution of the first point cloud frame through the convolutional neural network model, and determining the encoding condition of the second point cloud frame based on the motion vector;

[0070] Step S1012, generating an estimated code rate of each encoding path based on the complexity, the geometric distribution and the encoding condition through the convolutional neural network model.

[0071] After inputting the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame into the convolutional neural network model, the complexity and the geometric distribution of the first point cloud frame are analyzed through the convolutional neural network model, and the encoding condition of the second point cloud frame is analyzed based on the motion vector of the first point cloud frame relative to the second point cloud frame, and then based on the complexity and the geometric distribution of the first point cloud frame and the encoding condition of the second point cloud frame, an estimated code rate is generated for different encoding paths.

[0072] Step S20: determining a target bit rate of the first point cloud frame, and selecting a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature;

[0073] It should be noted that the target bit rate of a point cloud frame refers to the expected bit rate set for the point cloud frame.

[0074] After determining the target bit rate of the first point cloud frame, a suitable coding path (hereinafter referred to as the target coding path for distinction) is selected from the coding paths corresponding to the first point cloud frame based on the target bit rate to encode the first point cloud frame and obtain a potential representation in the first point cloud frame (hereinafter referred to as the first point cloud feature for distinction).

[0075] For example, Figure 2The figure shows a schematic diagram of the dynamic encoding process. The dynamic point cloud compression system of the present application includes a dynamic encoder (Dynamic Encoder), a dynamic decoder (Dynamic Decoder) and a slimmable convolution (SlimmableConvolution), wherein the dynamic encoder is used to encode each point cloud frame, the dynamic decoder is used to decode each encoded code stream, and the slimmable convolution is used to adjust the channel dimension of the input feature to dynamically adapt to the computing requirements. The dynamic encoder and dynamic decoder are based on a scalable convolution operator (i.e., adjustable width convolution), which dynamically adapts to the computing requirements by adjusting the channel dimension of the input feature. Specifically, each path of the encoder and decoder corresponds to a different computational complexity, and these paths are jointly trained to form a multi-level complexity compression network structure (supernet). By decomposing the entire network into several overlapping subnets, each subnet only handles part of the total computational load. In this way, the system can dynamically select a lower complexity encoding path as needed without sacrificing performance, or select a high complexity path to obtain higher compression quality when computing resources are sufficient. During the encoding process, the current point cloud frame consists of its spatial coordinates and corresponding feature information. The dynamic encoder converts the point cloud data into a latent representation (i.e., point cloud features) through a specific encoding path. Each encoding path performs different convolution operations on the input data, reducing redundancy and compressing the data. The encoding path selection is determined by the bitrate control module, which selects the appropriate path based on the current frame's bitrate requirements and system resources. For each encoding path, the raw point cloud data is downsampled to sparse the coordinates and aggregate local geometric details. This processed data is further reduced in redundancy through context modeling and entropy coding. The decoding process is similar to the encoding process. Based on the selected encoding path, the decoder converts the compressed latent representation back into reconstructed point cloud data. During the decoding process, the decoder adjusts the channel dimension and the complexity of the feature map to generate high-quality or low-complexity reconstructed point cloud data corresponding to the encoding path.

[0076] In addition, it is worth emphasizing that all encoding paths in the embodiments of the present application are not trained independently, but are optimized through a joint training strategy. Initially, the entire supernet is pre-trained under high bit rate conditions to support higher compression quality. On this basis, each sub-network is gradually trained under reduced bit rate conditions to ensure that the best rate-distortion performance can be achieved in different bit rate and complexity scenarios. Through this joint optimization, the encoding path finds a balance between computational complexity and compression quality, enabling the model to achieve flexible dynamic reasoning within a single framework. Since each encoding path shares some model parameters, this allows the model to avoid redundant calculations caused by completely independent paths when implementing reasoning of multiple different complexities. This parameter sharing strategy further reduces the overall computational load of the model and improves the resource utilization of the model. Moreover, in the dynamic point cloud compression framework based on multiple encoding paths proposed in this application, each encoding path corresponds to a different computational complexity, and through joint training, the best rate-distortion optimization is achieved at different complexity levels, thereby achieving adaptive adjustment of compression parameters according to the bandwidth and computing resource limitations of the actual application scenario to achieve efficient point cloud compression within a single model.

[0077] In this embodiment, step S20 may include:

[0078] Step S201: obtaining a preset expected average bit rate and a preset sliding window length, and determining the number of encoded point cloud frames in the dynamic point cloud data, and determining the total bit rate consumed by the encoded point cloud frames;

[0079] It should be noted that the preset expected average bitrate refers to the maximum average bitrate that can be allocated to each point cloud frame under the current limited bandwidth conditions, which is set by the user in advance based on the limited bandwidth. The preset sliding window length is the sliding window length of the sliding window algorithm used to calculate the target bitrate for each point cloud frame. The total bitrate consumed by the encoded point cloud frame refers to the sum of the bitrates consumed by the encoded point cloud frames.

[0080] Step S202, adding the number of frames to the sliding window length to obtain a sum;

[0081] Step S203 : Subtract the total bit rate from the sum to obtain a difference, and divide the difference by the sliding window length to obtain a target bit rate of the first point cloud frame.

[0082] After obtaining the preset expected average bit rate and the preset sliding window length, determining the number of encoded point cloud frames in the dynamic point cloud data, and determining the total bit rate consumed by the encoded point cloud frames, determine the sum of the number of frames and the sliding window length, subtract the total bit rate from the sum, then take the difference, and finally divide the difference by the sliding window length. The result value is the target bit rate of the first point cloud frame.

[0083] It should be noted that the bit rate control module includes a bit allocation unit, which is used to allocate the target bit rate of each frame through a sliding window algorithm after the bit rate estimation. For example, the target bit rate of the current frame is represented as T tar , then T tar The calculation formula can be expressed as:

[0084]

[0085] Among them, R tar The expected average bit rate preset by the user, N coded is the number of encoded point cloud frames in the dynamic point cloud data, SW is the sliding window length, R coded The total bitrate consumed for the encoded point cloud frames in the dynamic point cloud data.

[0086] In this way, in order to solve the problem of inaccurate bit rate control in the prior art, the embodiment of the present application designs a bit rate control module based on a sliding window and a bit allocation algorithm, which can adaptively adjust the bit rate during the dynamic point cloud compression process, ensuring the optimal compression performance under limited bandwidth and maintaining the stability of video quality.

[0087] In this embodiment, step S20 may include:

[0088] Step S204, determining the product of the expected average bit rate and the number of frames, and judging whether the product is greater than the total bit rate;

[0089] Step S205: If yes, determining the encoding path corresponding to the first bit rate among the estimated bit rates as the target encoding path, wherein the first bit rate is the estimated bit rate among the estimated bit rates that is greater than the target bit rate and has the smallest difference with the target bit rate;

[0090] Step S206: If not, determine that the encoding path corresponding to the second bit rate among the estimated bit rates is the target encoding path, wherein the second bit rate is the estimated bit rate among the estimated bit rates that is smaller than the target bit rate and has the smallest difference with the target bit rate.

[0091] After determining the target bit rate of the first point cloud frame, the product of the expected average bit rate and the number of encoded point cloud frames is calculated, and it is determined whether the product is greater than the total bit rate consumed by the encoded point cloud frames. If so, the estimated bit rate that is greater than the target bit rate and has the smallest difference with the target bit rate is determined (hereinafter referred to as the first bit rate for distinction), and the encoding path corresponding to the first bit rate is used as the target encoding path to encode the first point cloud frame through the target encoding path; if not, the estimated bit rate that is less than the target bit rate and has the smallest difference with the target bit rate is determined (hereinafter referred to as the second bit rate for distinction), and the encoding path corresponding to the second bit rate is used as the target encoding path to encode the first point cloud frame through the target encoding path.

[0092] Exemplarily, the path identifier of the target coding path is represented as i. For example, when the target coding path is Route1, i is 1. Then, the calculation formula of i can be expressed as:

[0093]

[0094] in, is the estimated bit rate of coding path i. In this way, the embodiment of the present application uses the bit implementation unit to adaptively select an appropriate target coding path according to the current coding state, thereby ensuring that the point cloud compression effect and the accuracy of bit rate control are improved within the established bit rate limit.

[0095] In this way, the embodiments of the present application solve the problem of inaccurate bit rate control in the prior art. Through an adaptive bit rate prediction and regulation mechanism, it can flexibly allocate the path of the current encoding frame under different bandwidth conditions, ensuring that efficient encoding effects are always maintained in real-time applications and meeting network bandwidth requirements.

[0096] Step S30 , generating compressed dynamic point cloud data based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0097] First, the first point cloud frame in the dynamic point cloud data is encoded to obtain the potential representation of the first point cloud frame (hereinafter referred to as the second point cloud feature for distinction). Then, each point cloud frame in the dynamic point cloud data except the first point cloud frame is encoded to obtain the first point cloud feature of each point cloud frame. Then, based on each first point cloud feature and the second point cloud feature, compressed dynamic point cloud data is generated. It can be understood that the compressed dynamic point cloud data is a code stream.

[0098] In the embodiment of the present application, the first point cloud frame of the dynamic point cloud data is separately encoded to obtain the second point cloud feature, and the code rate of each coding path is predicted for each point cloud frame in the dynamic point cloud data except the first point cloud frame. Then, based on the target code rate of each point cloud frame, a suitable target coding path is selected from the coding paths with different estimated code rates to encode the point cloud frame to obtain the first point cloud feature. Then, based on the point cloud features of all the point cloud frames in the dynamic point cloud data, compressed dynamic point cloud data is generated. It should be understood that the code rates of the optional coding paths corresponding to a point cloud frame are different, and the corresponding computational complexity is also different. In this way, the embodiment of the present application can avoid the problems of poor encoding effect and waste of bandwidth resources caused by uniform encoding of all point cloud frames with a fixed code rate by allocating a suitable coding path to each point cloud frame. In other words, it can improve the encoding quality of dynamic point cloud data by improving the accuracy of code rate control.

[0099] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction and will not be repeated hereafter. On this basis, before the step S10, the dynamic point cloud compression method of the present application further includes:

[0100] Step A10: stitching the first point cloud frame and the second point cloud frame to obtain a stitched point cloud frame;

[0101] Step A20, performing coarse-grained convolution processing on the spliced ​​point cloud frame to generate motion embedding;

[0102] Step A30, performing fine-grained convolution processing on the motion embedding to generate a motion estimate;

[0103] Step A40: Generate a motion vector of the first point cloud frame relative to the second point cloud frame based at least on the motion embedding and the motion estimation.

[0104] It should be noted that the dynamic point cloud compression system of this application also includes an inter-frame prediction module, which adopts a coarse-to-fine inter-frame prediction method (Coarse-to-fine Inter-frame Prediction) to improve the efficiency of inter-frame redundancy utilization while reducing computational complexity. This module accurately captures the motion information between dynamic point cloud frames through coarse-grained and fine-grained motion estimation and compensation steps, thereby improving the efficiency and accuracy of point cloud compression.

[0105] Before bitrate estimation, motion estimation is performed on adjacent point cloud frames to obtain a motion vector for each point cloud frame relative to the previous point cloud frame. Specifically, the first point cloud frame and the second point cloud frame are first spliced ​​to obtain a spliced ​​point cloud frame. The spliced ​​point cloud frame is then subjected to coarse-grained convolution to initially generate a motion embedding. The motion embedding is then subjected to fine-grained convolution to obtain a more accurate motion estimate. Based on at least the motion embedding and the motion estimate, a motion vector for the first point cloud frame relative to the second point cloud frame is then generated.

[0106] For example, Figure 3 The figure shows a schematic diagram of the inter-frame prediction process. The inter-frame prediction module includes motion estimation (Motion Estimation), motion compression (Motion Compression) and motion compensation (Motion Compensation). In the motion estimation stage, the inter-frame prediction module operates in the feature space and fuses the temporal information of two consecutive frames. Specifically, the point cloud features of the current frame and the previous frame are first spliced, and then processed through coarse-grained convolution to preliminarily generate motion embedding. To ensure the integrity of the motion information, the system further optimizes the motion embedding through fine-grained convolution to generate more accurate motion estimates. In addition, in order to reduce the information loss caused by the deep network, the residual branch is introduced to directly fuse the residual details with the motion embedding, and combine with the motion estimation to obtain the final motion vector. This motion vector represents the three-dimensional motion information of each point in the current frame.

[0107] In this way, the embodiment of the present application introduces a coarse-to-fine inter-frame prediction and compensation mechanism, breaking through the bottleneck of the traditional KNN (K-nearest neighbor) method in the limited perception field and low matching accuracy, and can more effectively utilize the redundant information between point cloud frames, greatly improving the balance between compression rate and compression quality. Specifically, the perception field is expanded through motion embedding and geometric information deconstruction to achieve efficient inter-frame compensation. Compared with the existing KNN method, this module significantly reduces the computational complexity of inter-frame prediction and more effectively eliminates redundant information between frames. Moreover, compared with the defects of high computational complexity and poor real-time performance of existing deep learning methods, the embodiment of the present application reduces the consumption of computing resources and improves the compression speed through dynamic encoding paths and lightweight inter-frame prediction algorithms. It is particularly suitable for real-time processing scenarios of large-scale dynamic point cloud data.

[0108] In this embodiment, step S30 may include:

[0109] Step S301: performing motion compensation on the second point cloud frame based on the motion vector to obtain a predicted point cloud frame corresponding to the first point cloud frame;

[0110] Based on the motion vector of the first point cloud frame relative to the second point cloud frame, the second point cloud frame is motion compensated to obtain a predicted point cloud frame corresponding to the first point cloud frame. The predicted point cloud frame corresponding to each point cloud frame is taken as the motion compensation information to participate in the subsequent conditional probability modeling.

[0111] Exemplarily, in the motion compensation stage, the inter-frame prediction module first quantizes the decoded motion vector to generate a block-level coarse-grained motion vector. This vector is used to preliminarily align the point cloud coordinates of the current frame with the coordinates of the previous frame to reduce the geometric difference between the frames. Then, the geometric information deconstruction module processes the point cloud geometric information of the current frame and the previous frame to generate anchor point geometric information. These anchors provide accurate geometric references to help further adjust the alignment of the point clouds between the frames. On this basis, the module generates a more refined motion vector for fine-grained motion compensation, and the result of the motion compensation is a predicted frame, for example, a predicted frame corresponding to the current frame is determined based on the motion vector of the current frame relative to the previous frame and the previous frame point cloud frame. Through the layer-by-layer refinement of the motion vector, the system can accurately align the point cloud features between the frames, thereby greatly improving the accuracy of inter-frame prediction.

[0112] In step S302, the hyper-prior encoding processing is performed on the first point cloud feature of each of the point cloud frames and the second point cloud feature obtained by encoding the first frame point cloud frame in the dynamic point cloud data, to obtain hyper-prior information.

[0113] The hyper-prior encoding processing is performed on the second point cloud feature corresponding to the first frame point cloud frame and the first point cloud feature of each point cloud frame other than the first frame point cloud frame in the dynamic point cloud data, to obtain hyper-prior information. It can be understood that the hyper-prior information refers to the latent representation of each point cloud feature obtained by encoding each point cloud feature.

[0114] In step S303, based on the predicted point cloud frame corresponding to each of the point cloud frames and the hyper-prior information, the conditional probability modeling is performed on each of the first point cloud features and the second point cloud feature, to obtain the probability value of each of the first point cloud features and the second point cloud feature.

[0115] Based on the predicted point cloud frame corresponding to each of the point cloud frames and the hyper-prior information, the conditional probability modeling is performed on each of the first point cloud features and the second point cloud feature, to obtain the probability value of each of the first point cloud features and the second point cloud feature.

[0116] In step S304, based on the probability value, the entropy encoding is performed on each of the first point cloud features and the second point cloud feature, to obtain the compressed dynamic point cloud data.

[0117] Based on the probability value of each point cloud feature, the entropy encoding is performed on each point cloud feature, to obtain the compressed dynamic point cloud data, i.e., the code stream.

[0118] It should be noted that the dynamic point cloud compression system of the present application further comprises a conditional coding module, which combines temporal and spatial context information to make full use of the geometric redundancy of point cloud data to improve compression efficiency and quality. Specifically, conditional coding first extracts context from the motion compensation information obtained from inter-frame prediction and spatial hyper-prior information, which is used to model the conditional probability of the latent representation in the point cloud frame. Through this conditional probability model, the geometric structure and attribute information of the point cloud can be encoded more accurately, further reducing redundancy.

[0119] Exemplarily, in the conditional coding module, temporal context and hyper-prior context are first extracted, and then combined by an entropy parameter network to generate an overall context. The overall context is used to guide the probability distribution modeling of the latent representation, thereby realizing conditional coding. Specifically, the probability distribution of the latent representation is modeled as a decomposed Gaussian distribution combined with soft quantization noise. This modeling method can effectively reduce the entropy value of the latent representation during encoding, reducing the code rate consumption. In this process, the rate-distortion loss is composed of multiple parts, including motion embedding, hyper-prior information and the coding rate loss of the latent representation. Through end-to-end training, the model can significantly improve the accuracy of reconstruction while maintaining a low code rate.

[0120] As Figure 4The figure shows the overall process of dynamic point cloud compression. First, bit estimation is performed, that is, the bit rate of each encoding path corresponding to each point cloud frame is estimated; at the same time, the expected average bit rate set by the user, that is, the target bit rate (Target Bitrate Rtar), is determined, and bit allocation is performed based on the expected average bit rate to obtain the target bit rate of each point cloud frame; then, bit implementation is performed based on each estimated bit rate and the target bit rate of the point cloud frame; then, the point cloud frame is encoded by the dynamic encoder in the dynamic inference route to obtain a potential representation; then, the potential representation of the current frame and the previous frame is motion estimated and motion compressed to obtain the temporal context of the current frame, that is, inter-frame prediction; then, conditional coding is performed. Specifically, the potential representation of the current frame is super-encoded by the hyper encoder to obtain the potential representation of the point cloud features of the current frame, and the potential representation of the point cloud features is input into the hyper decoder. The temporal context, the super-prior information input and the entropy parameters are input into the context model to obtain the compressed bit stream.

[0121] It should be noted that the insufficient utilization of inter-frame redundancy is one of the main bottlenecks of current technology. Most existing methods rely on KNN-based motion estimation and compensation mechanisms. Although these methods can predict the motion between point cloud frames in the latent space, the sparsity of the point cloud data itself makes it difficult to match corresponding points, and the perception field is limited, which ultimately affects the accuracy of the prediction. This inefficient use of inter-frame redundancy directly leads to the difficulty in optimizing the balance between compression rate and quality. However, the embodiment of the present application, through the above-mentioned conditional coding mechanism, further reduces point cloud redundancy while improving compression efficiency, ensuring compression quality and real-time performance under different complexities.

[0122] The present application also provides a dynamic point cloud compression device, please refer to Figure 5 , the dynamic point cloud compression device includes:

[0123] The bit rate prediction module 10 is configured to generate an estimated bit rate for each encoding path for any first point cloud frame among the point cloud frames to be compressed except the first point cloud frame in the dynamic point cloud data, based on the first point cloud frame and a motion vector of the first point cloud frame relative to a second point cloud frame, wherein the second point cloud frame is a point cloud frame preceding the first point cloud frame in the dynamic point cloud data;

[0124] An encoding module 20 is configured to determine a target bit rate for the first point cloud frame, and select a target encoding path from among the encoding paths based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature;

[0125] The compression module 30 is configured to generate compressed dynamic point cloud data based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data.

[0126] Optionally, the bit rate prediction module 10 is further configured to:

[0127] The first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain the estimated bit rate of each encoding path, wherein the convolutional neural network model is used to adaptively adjust the accuracy of the bit rate estimation according to the input features.

[0128] Optionally, the bit rate prediction module 10 is further configured to:

[0129] Determining the complexity and geometric distribution of the first point cloud frame through the convolutional neural network model, and determining the encoding status of the second point cloud frame based on the motion vector;

[0130] The convolutional neural network model generates an estimated bit rate for each coding path based on the complexity, the geometric distribution and the coding situation.

[0131] Optionally, the encoding module 20 is further configured to:

[0132] Obtaining a preset expected average bit rate and a preset sliding window length, and determining the number of encoded point cloud frames in the dynamic point cloud data, and determining a total bit rate consumed by the encoded point cloud frames;

[0133] Adding the number of frames to the sliding window length to obtain a sum;

[0134] The total bit rate is subtracted from the sum to obtain a difference, and the difference is divided by the sliding window length to obtain a target bit rate for the first point cloud frame.

[0135] Optionally, the encoding module 20 is further configured to:

[0136] determining a product between the expected average code rate and the number of frames, and determining whether the product is greater than the total code rate;

[0137] If yes, determining that an encoding path corresponding to a first code rate among the estimated code rates is a target encoding path, wherein the first code rate is an estimated code rate among the estimated code rates that is greater than the target code rate and has a minimum difference with the target code rate.

[0138] If no, determining that an encoding path corresponding to a second code rate among the estimated code rates is a target encoding path, wherein the second code rate is an estimated code rate among the estimated code rates that is less than the target code rate and has a minimum difference with the target code rate.

[0139] Optionally, the dynamic point cloud compression device further comprises a motion estimation module, configured to:

[0140] stitching the first point cloud frame and the second point cloud frame to obtain a stitched point cloud frame;

[0141] performing coarse-grained convolution processing on the stitched point cloud frame to generate a motion embedding;

[0142] performing fine-grained convolution processing on the motion embedding to generate a motion estimation;

[0143] generating a motion vector of the first point cloud frame relative to the second point cloud frame based at least on the motion embedding and the motion estimation.

[0144] Optionally, the compression module 30 is further configured to:

[0145] performing motion compensation on the second point cloud frame based on the motion vector to obtain a predicted point cloud frame corresponding to the first point cloud frame;

[0146] performing hyper-prior encoding processing on the first point cloud feature of each of the point cloud frames and a second point cloud feature obtained by encoding the first frame point cloud frame in the dynamic point cloud data to obtain hyper-prior information;

[0147] performing conditional probability modeling on the first point cloud feature and the second point cloud feature based on the predicted point cloud frame corresponding to each of the point cloud frames and the hyper-prior information to obtain a probability value of each of the first point cloud feature and the second point cloud feature;

[0148] performing entropy encoding on the first point cloud feature and the second point cloud feature based on the probability value to obtain compressed dynamic point cloud data.

[0149] The dynamic point cloud compression device provided by the embodiments of the present application adopts the dynamic point cloud compression method in the above embodiments, and can solve the technical problem of how to improve the code rate control accuracy during dynamic point cloud compression, so as to improve the coding quality. Compared with the prior art, the dynamic point cloud compression device provided by the embodiments of the present application has the same beneficial effects as the dynamic point cloud compression method provided by the above embodiments, and other technical features in the dynamic point cloud compression device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0150] The present application provides a dynamic point cloud compression device, which comprises at least one processor and a memory connected in communication with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the dynamic point cloud compression method in Embodiment I.

[0151] Reference will be made to the following description of the embodiments of the present application, taken in conjunction with the accompanying drawings, in which Figure 6 which shows a structural diagram of a dynamic point cloud compression device suitable for implementing the embodiments of the present application. The dynamic point cloud compression device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 6 The dynamic point cloud compression device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0152] As Figure 6As shown, the dynamic point cloud compression device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the dynamic point cloud compression device. The processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the dynamic point cloud compression device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a dynamic point cloud compression device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0153] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0154] The dynamic point cloud compression device provided in this application, employing the dynamic point cloud compression method of the aforementioned embodiment, can address the technical problem of improving the bit rate control accuracy during dynamic point cloud compression, thereby enhancing encoding quality. Compared to the prior art, the beneficial effects of the dynamic point cloud compression device provided in this application are the same as those of the dynamic point cloud compression method provided in the aforementioned embodiment. Other technical features of the dynamic point cloud compression device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0155] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0156] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0157] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the dynamic point cloud compression method in the above embodiment.

[0158] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0159] The computer-readable storage medium may be included in the dynamic point cloud compression device; or it may exist independently without being assembled into the dynamic point cloud compression device.

[0160] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the dynamic point cloud compression device, the dynamic point cloud compression device: for any first point cloud frame in each point cloud frame other than the first frame point cloud frame in the dynamic point cloud data to be compressed, based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, generates an estimated bit rate for each encoding path, wherein the second point cloud frame is the previous frame point cloud frame of the first point cloud frame in the dynamic point cloud data; determines the target bit rate of the first point cloud frame, and selects the target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature; generates compressed dynamic point cloud data based on the first point cloud feature of each point cloud frame and the second point cloud feature obtained by encoding the first frame point cloud frame in the dynamic point cloud data.

[0161] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0162] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0163] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0164] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned dynamic point cloud compression method. This computer-readable storage medium addresses the technical problem of improving the accuracy of bit rate control during dynamic point cloud compression to enhance encoding quality. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the dynamic point cloud compression method provided in the aforementioned embodiments, and are not further elaborated here.

[0165] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the dynamic point cloud compression method as described above.

[0166] The computer program product provided in this application can improve the bit rate control accuracy during dynamic point cloud compression, thereby improving encoding quality. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the dynamic point cloud compression method provided in the above embodiments, and will not be repeated here.

[0167] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A dynamic point cloud compression method, characterized in that: The dynamic point cloud compression method comprises: For any first point cloud frame among the point cloud frames to be compressed except the first point cloud frame in the dynamic point cloud data, generating an estimated bit rate for each encoding path based on the first point cloud frame and a motion vector of the first point cloud frame relative to a second point cloud frame, wherein the second point cloud frame is a point cloud frame preceding the first point cloud frame in the dynamic point cloud data; determining a target bit rate for the first point cloud frame, and selecting a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain a first point cloud feature; Generate compressed dynamic point cloud data based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data; The step of determining a target bit rate of the first point cloud frame includes: Obtaining a preset expected average bit rate and a preset sliding window length, and determining the number of encoded point cloud frames in the dynamic point cloud data, and determining a total bit rate consumed by the encoded point cloud frames; Adding the number of frames to the sliding window length to obtain a sum; Subtracting the total bit rate from the sum to obtain a difference, and dividing the difference by the sliding window length to obtain a target bit rate for the first point cloud frame; The step of selecting a target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame includes: Determining a product of the expected average bit rate and the number of frames, and determining whether the product is greater than the total bit rate; If so, determining a coding path corresponding to a first bit rate among the estimated bit rates as the target coding path, wherein the first bit rate is an estimated bit rate that is greater than the target bit rate and has the smallest difference with the target bit rate among the estimated bit rates; If not, determining a coding path corresponding to a second bit rate among the estimated bit rates as the target coding path, wherein the second bit rate is an estimated bit rate among the estimated bit rates that is smaller than the target bit rate and has the smallest difference with the target bit rate; Before the step of generating an estimated bit rate for each encoding path based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame, the method further includes: Splicing the first point cloud frame and the second point cloud frame to obtain a spliced ​​point cloud frame; Performing coarse-grained convolution processing on the spliced ​​point cloud frames to generate motion embedding; performing fine-grained convolution on the motion embedding to generate a motion estimate; Based at least on the motion embedding and the motion estimation, a motion vector of the first point cloud frame relative to the second point cloud frame is generated.

2. The method according to claim 1, wherein The step of generating an estimated bit rate for each encoding path based on the first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame includes: The first point cloud frame and the motion vector of the first point cloud frame relative to the second point cloud frame are input into a preset convolutional neural network model to obtain the estimated bit rate of each encoding path, wherein the convolutional neural network model is used to adaptively adjust the accuracy of the bit rate estimation according to the input features.

3. The method according to claim 2, wherein The step of obtaining the estimated bit rate of each encoding path includes: Determining the complexity and geometric distribution of the first point cloud frame through the convolutional neural network model, and determining the encoding status of the second point cloud frame based on the motion vector; The convolutional neural network model generates an estimated bit rate for each coding path based on the complexity, the geometric distribution and the coding situation.

4. The method according to any one of claims 1 to 3, characterized in that The step of generating encoded data based on the first point cloud features of each of the first point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data includes: Based on the motion vector, motion compensation is performed on the second point cloud frame to obtain a predicted point cloud frame corresponding to the first point cloud frame; Performing super-prior encoding processing on the first point cloud feature of each of the point cloud frames and the second point cloud feature obtained by encoding the first point cloud frame in the dynamic point cloud data to obtain super-prior information; Based on the predicted point cloud frames corresponding to the respective point cloud frames and the super priori information, conditional probability modeling is performed on the respective first point cloud features and the second point cloud features to obtain respective probability values ​​of the respective first point cloud features and the second point cloud features; Entropy encoding is performed on each of the first point cloud features and the second point cloud features based on each probability value to obtain compressed dynamic point cloud data.

5. A dynamic point cloud compression device, characterized in that: The dynamic point cloud compression device comprises: a bit rate prediction module, configured to generate, for each encoding path, an estimated bit rate for any first point cloud frame among the point cloud frames to be compressed except the first point cloud frame, based on the first point cloud frame and a motion vector of the first point cloud frame relative to a second point cloud frame, wherein the second point cloud frame is a point cloud frame preceding the first point cloud frame in the dynamic point cloud data; The encoding module is used to determine the target bit rate of the first point cloud frame, select the target encoding path in each encoding path based on the target bit rate to encode the first point cloud frame to obtain the first point cloud feature; obtain a preset expected average bit rate and a preset sliding window length, and determine the number of frames of the encoded point cloud frames in the dynamic point cloud data, and determine the total bit rate consumed by the encoded point cloud frames; add the number of frames to the sliding window length to obtain a sum; subtract the total bit rate from the sum to obtain a difference, and divide the difference by the sliding window length to obtain the target bit rate of the first point cloud frame. determining a product of the expected average bit rate and the number of frames, and determining whether the product is greater than the total bit rate; if so, determining a coding path corresponding to a first bit rate among the estimated bit rates as a target coding path, wherein the first bit rate is an estimated bit rate that is greater than the target bit rate and has the smallest difference with the target bit rate among the estimated bit rates; if not, determining a coding path corresponding to a second bit rate among the estimated bit rates as the target coding path, wherein the second bit rate is an estimated bit rate that is less than the target bit rate and has the smallest difference with the target bit rate among the estimated bit rates; a compression module, configured to generate compressed dynamic point cloud data based on the first point cloud features of each of the point cloud frames and the second point cloud features obtained by encoding the first point cloud frame in the dynamic point cloud data; A motion estimation module is configured to stitch the first point cloud frame and the second point cloud frame to obtain a stitched point cloud frame; perform coarse-grained convolution on the stitched point cloud frame to generate a motion embedding; perform fine-grained convolution on the motion embedding to generate a motion estimate; and generate a motion vector of the first point cloud frame relative to the second point cloud frame based at least on the motion embedding and the motion estimate.

6. A dynamic point cloud compression device, characterized in that: The dynamic point cloud compression device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the dynamic point cloud compression method according to any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the dynamic point cloud compression method according to any one of claims 1 to 4 are implemented.

Images