Point cloud compression method, encoder, decoder, and storage medium

Abstract

The embodiment of the application discloses a point cloud compression method, an encoder, a decoder and a storage medium, wherein the method comprises the following steps: acquiring a current block of a video to be encoded; determining geometric information and corresponding attribute information of point cloud data of the current block; using a sparse convolution network to down-sample the geometric information and the corresponding attribute information, so as to obtain hidden layer features; and compressing the hidden layer features, so as to obtain a compressed code stream.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 202010677169.6, filed on July 14, 2020, entitled "Point Cloud Compression Method, Encoder, Decoder and Storage Medium". Technical Field

[0002] This application relates to video encoding technology, including but not limited to point cloud compression methods, encoders, decoders, and storage media. Background Technology

[0003] In learning-based point cloud geometry compression techniques, the application of compression on point sets is limited to small point clouds with a fixed number of points, and cannot be used for complex point clouds in real-world scenarios. Moreover, point cloud compression techniques based on 3D dense convolution convert sparse point clouds into volumetric models for compression, failing to fully exploit the sparse structure of point clouds, resulting in computational redundancy and low coding performance. Summary of the Invention

[0004] This application provides a point cloud compression method, an encoder, a decoder, and a storage medium. The technical solution of this application is implemented as follows:

[0005] In a first aspect, the point cloud compression method provided in the embodiments of this application includes:

[0006] Get the current block of the video to be compressed;

[0007] Determine the geometric information and corresponding attribute information of the point cloud data of the current block;

[0008] A sparse convolutional network is used to downsample the geometric information and the corresponding attribute information to obtain hidden layer features;

[0009] The hidden layer features are compressed to obtain a compressed bitstream.

[0010] Secondly, the point cloud compression method provided in the embodiments of this application includes:

[0011] Get the current block of the video to be decompressed;

[0012] Determine the geometric information and corresponding attribute information of the point cloud data of the current block;

[0013] A transposed convolutional network is used to upsample the geometric information and the corresponding attribute information to obtain hidden layer features;

[0014] The hidden layer features are decompressed to obtain a decompressed bitstream.

[0015] Thirdly, the encoder provided in the embodiments of this application includes:

[0016] First memory and first processor;

[0017] The first memory stores a computer program that can run on a first processor, which, when executing the program, implements the point cloud compression method described in the first aspect.

[0018] Fourthly, the decoder provided in the embodiments of this application includes:

[0019] Second memory and second processor;

[0020] The second memory stores a computer program that can run on a second processor, which, when executing the program, implements the point cloud compression method described in the second aspect.

[0021] In this embodiment, for the current block of the acquired video to be encoded, firstly, the geometric information and corresponding attribute information of the point cloud data of the current block are determined; then, a sparse convolutional network is used to downsample the geometric information and corresponding attribute information to obtain hidden layer features; finally, the hidden layer features are compressed to obtain a compressed bitstream. In this way, by using a sparse convolutional network to perform sparse downsampling of the geometric and attribute information of the point cloud in the current block, sparse transformation of complex point clouds can be achieved. By compressing the obtained hidden layer features to obtain a compressed bitstream, not only can the computing speed be improved, but the encoding performance is also high, thus making it suitable for complex point clouds in real-world scenes. Attached Figure Description

[0022] Figure 1 An exemplary coding flowchart provided for embodiments of this application;

[0023] Figure 2 An exemplary decoding flowchart provided for embodiments of this application;

[0024] Figure 3A This is a schematic diagram illustrating the implementation process of the point cloud compression method in an embodiment of this application;

[0025] Figure 3B This is a schematic diagram of the neural network structure in an embodiment of this application;

[0026] Figure 3C This is a schematic diagram illustrating another implementation process of the point cloud compression method provided in the embodiments of this application;

[0027] Figure 4 This is a schematic diagram illustrating another implementation process of the point cloud compression method in this application.

[0028] Figure 5A This is a schematic diagram illustrating the implementation process of the point cloud compression and decompression method in an embodiment of this application;

[0029] Figure 5B This is a schematic diagram of the structure of the IRN in an embodiment of this application;

[0030] Figure 5C This is a schematic diagram of the context model in an embodiment of this application;

[0031] Figure 6 This is a schematic diagram of the reconstruction process in an embodiment of this application;

[0032] Figure 7 This is a schematic diagram comparing the bitrate curves of various data and other methods in the embodiments of this application;

[0033] Figure 8 This is a schematic diagram comparing the subjective quality of similar bitrates on red and black data in this application embodiment with the subjective quality obtained by other methods;

[0034] Figure 9 A schematic diagram of the encoder structure provided in the embodiments of this application;

[0035] Figure 10 This is a schematic diagram of another component structure of the encoder provided in an embodiment of this application;

[0036] Figure 11 A schematic diagram of the decoder structure provided in the embodiments of this application;

[0037] Figure 12 This is a schematic diagram of another component structure of the decoder provided in an embodiment of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0040] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0041] It should be noted that the terms "first, second, and third" used in the embodiments of this application are merely used to distinguish similar or different objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0042] To facilitate understanding of the technical solutions provided in the embodiments of this application, a flowchart of G-PCC encoding and a flowchart of G-PCC decoding are first provided. It should be noted that the flowcharts of G-PCC encoding and decoding described in the embodiments of this application are only for more clearly illustrating the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided in the embodiments of this application. Those skilled in the art will understand that with the evolution of G-PCC encoding and decoding technologies and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0043] In this embodiment of the application, in the point cloud G-PCC encoder framework, the point cloud of the input 3D image model is divided into slices, and each slice is encoded independently.

[0044] like Figure 1The flowchart of G-PCC encoding shown is applied in a point cloud encoder. For the point cloud data to be encoded, it is first divided into multiple slices. Within each slice, the geometric information and the attribute information corresponding to each point cloud are encoded separately. During geometric encoding, coordinate transformation is performed on the geometric information so that the entire point cloud is contained within a bounding box. Then, quantization is performed; this step mainly serves a scaling function. Due to quantization rounding, some point clouds have identical geometric information, so parameters are used to determine whether to remove duplicate points. This process of quantization and removing duplicate points is also called voxelization. Next, the bounding box is divided into octrees. In the octree-based geometric information encoding process, the bounding box is divided into eight equal sub-cubes. Non-empty sub-cubes (containing points from the point cloud) are further divided into eight equal parts until the resulting leaf nodes are 1x1x1 unit cubes. Arithmetic encoding is then performed on the points in the leaf nodes to generate a binary geometric bitstream, i.e., the geometric code stream. In the geometric information encoding process based on triangle soup (trisoup), octree partitioning is also performed first. However, unlike octree-based geometric information encoding, this trisoup does not need to partition the point cloud into unit cubes with side lengths of 1x1x1. Instead, it stops partitioning when the side length of a sub-block is W. Based on the surface formed by the distribution of the point cloud in each block, at most twelve intersection points (vertices) are obtained between this surface and the twelve edges of the block. Arithmetic encoding is performed on the vertices (surface fitting based on the intersection points) to generate a binary geometric bitstream, i.e., a geometric code stream. Vertexes are also used in the implementation of the geometric reconstruction process, and the reconstructed geometric information is used when encoding the attributes of the point cloud.

[0045] In the attribute encoding process, after geometric encoding is completed and the geometric information is reconstructed, color conversion is performed to transform the color information (i.e., attribute information) from the RGB color space to the YUV color space. Then, the reconstructed geometric information is used to recolor the point cloud, so that the unencoded attribute information corresponds with the reconstructed geometric information. There are two main transformation methods in the color information encoding process: one is distance-based lifting transformation that relies on Level of Detail (LOD) partitioning, and the other is a direct Region Adaptive Hierarchical Transform (RAHT) transformation. Both methods transform the color information from the spatial domain to the frequency domain, obtaining high-frequency and low-frequency coefficients through the transformation. Finally, the coefficients are quantized (i.e., quantization coefficients). Finally, the geometrically encoded data after octree partitioning and surface fitting, and the attribute-encoded data processed by quantization coefficients are sliced ​​and synthesized. The vertex coordinates of each block are then encoded sequentially (i.e., arithmetic encoding) to generate a binary attribute bitstream, i.e., the attribute code stream.

[0046] like Figure 2 The flowchart of G-PCC decoding shown is applied to the point cloud decoder. The decoder acquires the binary bitstream and performs independent decoding on the geometric bitstream and attribute bitstream within the binary bitstream. When decoding the geometric bitstream, the geometric information of the point cloud is obtained through arithmetic decoding, octree synthesis, surface fitting, geometry reconstruction, and inverse coordinate transformation. When decoding the attribute bitstream, the attribute information of the point cloud is obtained through arithmetic decoding, inverse quantization, inverse LOD-based boosting or RAHT-based inverse transformation, and inverse color conversion. Based on the geometric and attribute information, a 3D image model of the point cloud data to be encoded is reconstructed.

[0047] The point cloud compression method in this application embodiment is mainly applied in, for example, Figure 1 The G-PCC encoding process shown is as follows: Figure 2 The flowchart for G-PCC decoding is shown. In other words, the point cloud compression method in this embodiment can be applied to both the flowchart for G-PCC encoding and the flowchart for G-PCC decoding, or even simultaneously.

[0048] Figure 3A This is a schematic diagram illustrating the implementation process of the point cloud compression method in an embodiment of this application. This method can be implemented by an encoder, such as... Figure 3A As shown, the method includes at least the following steps:

[0049] Step S301: Obtain the current block of the video to be compressed.

[0050] It should be noted that a video image can be divided into multiple image blocks, and each image block to be encoded can be called a coding block (CB). Here, each coding block may include a first image component, a second image component, and a third image component; and the current block is the coding block in the video image to be predicted for the first image component, the second image component, or the third image component.

[0051] In this context, if the current block performs prediction of the first image component, and the first image component is the luminance component, that is, the image component to be predicted is the luminance component, then the current block can also be called the luminance block; or, if the current block performs prediction of the second image component, and the second image component is the chrominance component, that is, the image component to be predicted is the chrominance component, then the current block can also be called the chrominance block.

[0052] It should also be noted that the prediction mode parameters indicate the coding mode of the current block and the parameters associated with that mode. Rate Distortion Optimization (RDO) is typically used to determine the prediction mode parameters of the current block.

[0053] Specifically, in some embodiments, the encoder determines the prediction mode parameters of the current block as follows: the encoder determines the image components to be predicted in the current block; based on the parameters of the current block, it performs prediction encoding on the image components to be predicted using multiple prediction modes, calculates the rate-distortion cost result corresponding to each prediction mode under multiple prediction modes; selects the minimum rate-distortion cost result from the multiple calculated rate-distortion cost results, and determines the prediction mode corresponding to the minimum rate-distortion cost result as the prediction mode parameters of the current block.

[0054] In other words, on the encoder side, multiple prediction modes can be used to encode the image components to be predicted for the current block. Here, multiple prediction modes typically include inter-frame prediction modes, traditional intra-frame prediction modes, and non-traditional intra-frame prediction modes. Traditional intra-frame prediction modes can include Direct Current (DC) mode, Planar mode, and Angle mode, etc. Non-traditional intra-frame prediction modes can include MIP mode, Cross-component Linear Model Prediction (CCLM) mode, Intra Block Copy (IBC) mode, and PLT (Palette) mode, etc. Inter-frame prediction modes can include: Geometrical partitioning for inter blocks (GEO) mode, Geometric partitioning prediction mode, Triangle partition mode (TPM), etc.

[0055] In this way, after encoding the current block using multiple prediction modes, the rate-distortion cost result corresponding to each prediction mode can be obtained. Then, the minimum rate-distortion cost result is selected from the multiple rate-distortion cost results, and the prediction mode corresponding to the minimum rate-distortion cost result is determined as the prediction mode parameter of the current block. In this way, the current block can be encoded using the determined prediction mode. Moreover, under this prediction mode, the prediction residual can be small, which can improve the coding efficiency.

[0056] Step S302: Determine the geometric information and corresponding attribute information of the point cloud data of the current block.

[0057] In some embodiments, point cloud data includes the number of points in a point cloud region; the point cloud data in the current block meets preset conditions, including: the point cloud data in the current block is a dense point cloud. Taking a two-dimensional case as an example, such as... Figure 3B As shown, this illustrates a comparison between sparse and dense convolution: In dense convolution, the convolution kernel traverses every pixel position in plane 321; in sparse convolution, since the data is sparsely distributed within plane 322, it is not necessary to traverse all positions on the plane, but only to perform convolution processing on the positions where the data exists (i.e., the positions of the colored boxes). This can greatly reduce the processing load for point clouds, which are spatially sparsely distributed data. In some possible implementations, the geometric information of these points and the corresponding attribute information are determined. The geometric information includes the coordinates of the point, and the attribute information includes at least color, brightness, and pixel values.

[0058] Step S303: A sparse convolutional network is used to downsample the geometric information and the corresponding attribute information to obtain hidden layer features.

[0059] In some embodiments, the hidden layer features are the geometric information and corresponding attribute information of the current block after downsampling. Step S303 can be understood as using a sparse convolutional network to downsample the geometric information and the corresponding attribute information multiple times to obtain the downsampled geometric information and corresponding attribute information. For example, using a convolution implementation with a stride of 2 and a kernel size of 2, the features of each voxel in a 2*2*2 spatial unit are aggregated onto one voxel. After each downsampling, the length, width, and height scales of the point cloud are reduced by half. A total of 3 downsampling operations are performed to obtain the hidden layer features.

[0060] Step S304: Compress the hidden layer features to obtain a compressed bitstream.

[0061] In some embodiments, the geometric and attribute information of the final hidden layer features are encoded into binary bitstreams to obtain a compressed bitstream.

[0062] In some possible implementations, firstly, the frequency of occurrence of geometric information in the hidden layer features is determined. For example, the frequency of occurrence of geometric coordinates in the point cloud is determined using an entropy model; where the entropy model is based on a trainable probability density distribution represented by factorization, or a conditional entropy model based on contextual information. Then, the hidden layer features are weighted according to the frequency to obtain adjusted hidden layer features; for example, the higher the probability of occurrence, the larger the weight. Finally, the adjusted hidden layer features are encoded into a binary bitstream to obtain the compressed bitstream. For example, the coordinates and attributes of the hidden layer features are encoded separately using arithmetic encoding to obtain the compressed bitstream.

[0063] In this embodiment of the application, a sparse convolutional network is used to determine the point cloud regions with fewer points in the point cloud, thereby extracting the feature attributes of the point cloud regions with more points. This not only improves the computing speed but also has high encoding performance, and can therefore be used for complex point clouds in real-world scenarios.

[0064] In some embodiments, to better apply the technology to complex point cloud scenarios, after obtaining the current block of the video to be compressed, the following steps can be taken: First, determine the number of points in the point cloud data of the current block; second, determine the point cloud region in the current block where the number of points is greater than or equal to a preset value; third, determine the geometric information and corresponding attribute information of the point cloud data in the point cloud region. Finally, downsample the geometric information and corresponding attribute information of this region using a sparse convolutional network to obtain hidden layer features for compression. In this way, by downsampling regions including dense point clouds using a sparse convolutional network, point cloud compression of complex scenes can be achieved.

[0065] In some embodiments, to improve the accuracy of the determined geometric and attribute information, step S302 can be achieved through the following steps:

[0066] Step S321: Determine the coordinates of any point in the point cloud data in the world coordinate system to obtain the geometric information.

[0067] Here, for any point in the point cloud data, the coordinates of that point in the world coordinate system are determined, and these coordinates are used as geometric information; alternatively, the geometric information can be set to all 1s as a placeholder. This saves on the computational load of geometric information.

[0068] Step S322: Perform feature extraction on any point to obtain the attribute information corresponding to the geometric information.

[0069] Here, feature extraction is performed on each point to obtain attribute information including the point's color, brightness, and pixel count.

[0070] In this embodiment of the application, by determining the coordinate values ​​of points in the point cloud data in the world coordinate system, using the coordinate values ​​as set information, and performing feature extraction to obtain attribute information, the accuracy of the determined geometric information and attribute information is improved.

[0071] In some embodiments, step S303, employing a sparse convolutional network to downsample the geometric information and corresponding attribute information to obtain hidden layer features, can be achieved through the following steps, such as... Figure 3C As shown, Figure 3C This is a schematic diagram illustrating another implementation process of the point cloud compression method provided in the embodiments of this application, combined with... Figure 3A The following explanation is provided:

[0072] Step S401: Quantize the geometric information and the corresponding attribute information to obtain unit voxels and a set of unit voxels.

[0073] Here, geometric information and corresponding attribute information are represented in the form of a three-dimensional sparse tensor. This three-dimensional sparse tensor is then quantized into unit voxels, resulting in a set of unit voxels. A unit voxel can be understood as the smallest unit representing point cloud data.

[0074] Step S402: Determine the number of downsampling steps based on the downsampling stride and kernel size of the sparse convolutional network.

[0075] Here, sparse convolutional networks can be implemented using sparse convolutional neural networks, such as... Figure 3B As shown in Figure 322, the larger the downsampling stride and kernel size, the fewer the downsampling iterations. In a specific example, the number of downsampling iterations is the product of the downsampling stride and the kernel size. For instance, first, based on the downsampling stride and kernel size of the sparse convolutional network, the compressible voxel space is determined, and then the number of iterations is determined based on the size of this space. In a sparse convolutional neural network, the downsampling stride can be set to 2, and the network's kernel size is 2. Therefore, the compressible voxel space is 2*2*2, and the number of iterations is determined to be 3.

[0076] Step S403: Aggregate the unit voxels in the unit voxel set according to the number of downsampling times to obtain the hidden layer features.

[0077] For example, by downsampling 3 times, it is possible to aggregate unit voxels within each 2*2*2 spatial unit.

[0078] In some possible implementations, firstly, based on the number of downsampling iterations, the area occupied by the point cloud is divided into multiple unit aggregation regions; for example, with three downsampling iterations, the area occupied by the point cloud is divided into multiple 2*2*2 unit aggregation regions. Then, the voxels within each unit aggregation region are aggregated to obtain a target voxel set; for example, the voxels in each 2*2*2 unit aggregation region are aggregated into one target voxel, thus obtaining the target voxel set. Finally, the geometric information and corresponding attribute information of each target voxel in the target voxel set are determined to obtain the hidden layer features. Here, by aggregating the voxels within the unit aggregation regions, the geometric information and corresponding attribute information of each target pixel are determined to obtain the hidden layer features.

[0079] In this embodiment, multiple unit voxels within a unit aggregation region are aggregated into a single target voxel through multiple downsampling operations. The geometric information and corresponding attribute information of the target voxel are then used as hidden layer features. This achieves compression of multiple voxels, thereby improving coding performance.

[0080] This application provides a point cloud compression method applied to a video decoding device, i.e., a decoder. The functionality achieved by this method can be implemented by a processor in the video decoding device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the video encoding device includes at least a processor and a storage medium.

[0081] In some embodiments, Figure 4 This is a schematic diagram illustrating another implementation process of the point cloud compression method in this application. This method can be implemented by a decoder, such as... Figure 4 As shown, the method includes at least the following steps:

[0082] Step S501: Obtain the current block of the video to be decompressed.

[0083] Step S502: Determine the geometric information and corresponding attribute information of the point cloud data of the current block.

[0084] Step S503: A transposed convolutional network is used to upsample the geometric information and the corresponding attribute information to obtain hidden layer features.

[0085] Here, the kernel size of the transposed convolutional network is the same as that of the sparse convolutional network. In some possible implementations, a transposed convolutional network with a stride of 2 and a kernel size of 2 can be used to upsample geometric information and corresponding attribute information.

[0086] Step S504: Decompress the hidden layer features to obtain a decompressed bitstream.

[0087] In some embodiments, the geometric and attribute information of the final hidden layer features are encoded into binary bitstreams to obtain a compressed bitstream.

[0088] In some possible implementations, firstly, the frequency of occurrence of geometric information in the hidden layer features is determined. For example, the frequency of occurrence of geometric coordinates in the point cloud is determined using an entropy model; where the entropy model is based on a trainable probability density distribution represented by factorization, or a conditional entropy model based on contextual information. Then, the hidden layer features are weighted according to the frequency to obtain adjusted hidden layer features; for example, the higher the probability of occurrence, the larger the weight. Finally, the adjusted hidden layer features are decoded into a binary bitstream to obtain the decompressed bitstream. For example, the coordinates and attributes of the hidden layer features are decoded separately using arithmetic decoding to obtain the decompressed bitstream.

[0089] In this embodiment of the application, by using a sparse transpose network to decompress the compressed point cloud data, not only can the computing speed be improved, but the encoding performance is also high, which can then be used for complex point clouds in real-world scenarios.

[0090] In some embodiments, to better apply the technology to complex point cloud scenarios, after obtaining the current block of the video to be compressed, the following steps can be taken: First, determine the number of points in the point cloud data of the current block; second, determine the point cloud region in the current block where the number of points is greater than or equal to a preset value; third, determine the geometric information and corresponding attribute information of the point cloud data in the point cloud region. Finally, downsample the geometric information and corresponding attribute information of this region using a sparse convolutional network to obtain hidden layer features for compression. In this way, by downsampling regions including dense point clouds using a sparse convolutional network, point cloud compression of complex scenes can be achieved.

[0091] In some embodiments, to improve the accuracy of the determined geometric and attribute information, step S502 can be achieved through the following steps:

[0092] Step S521: Determine the coordinates of any point in the point cloud data in the world coordinate system to obtain the geometric information.

[0093] Step S522: Perform feature extraction on any point to obtain the attribute information corresponding to the geometric information.

[0094] In this embodiment of the application, by determining the coordinate values ​​of points in the point cloud data in the world coordinate system, using the coordinate values ​​as set information, and performing feature extraction to obtain attribute information, the accuracy of the determined geometric information and attribute information is improved.

[0095] In some embodiments, step S503, using a transposed convolutional network to upsample the geometric information and the corresponding attribute information to obtain hidden layer features, can be achieved through the following steps:

[0096] The first step is to determine the target voxel to which the geometric information and corresponding attribute information belong.

[0097] Here, since the current block is obtained through compression, the geometric information and attribute information are also compressed. First, the target voxel to which the geometric information and corresponding attribute information belong is determined. This target voxel is obtained by compressing multiple unit voxels. Therefore, the target voxel to which the geometric information and corresponding attribute information belong is determined first.

[0098] The second step is to determine the number of upsampling operations based on the upsampling stride and kernel size of the transposed convolutional network.

[0099] Here, the transposed convolutional network can be implemented using a sparse transposed convolutional neural network. The larger the downsampling stride and the kernel size, the fewer the upsampling times.

[0100] In some possible implementations, firstly, the unit aggregation region occupied by the target voxel is determined; for example, it is determined which regions' unit voxels the target voxel is aggregated from.

[0101] Then, within the unit aggregation region, the target voxel is decompressed into multiple unit voxels according to the number of upsampling operations; for example, if the unit aggregation region is 2*2*2, then the target voxel is decompressed into multiple unit voxels three times according to the number of upsampling operations.

[0102] Finally, the geometric information and corresponding attribute information of each unit voxel are determined to obtain the hidden layer features. For example, the geometric information and corresponding attribute information are represented in the form of a three-dimensional sparse tensor. This three-dimensional sparse tensor is then quantized into unit voxels, thereby obtaining a set of unit voxels.

[0103] In some possible implementations, firstly, the proportion of non-empty voxels in the current layer of the current block relative to the total target voxels is determined. Here, the number of occupied voxels (i.e., non-empty voxels) and unoccupied voxels (i.e., empty voxels) in the current layer is determined to obtain the proportion of non-empty voxels in the current layer relative to the total target voxels. Furthermore, for each layer of the current block, the number of occupied voxels and unoccupied empty voxels is determined to obtain the proportion of non-empty voxels in the total target voxels. In some embodiments, a binary classification neural network is first used to determine the probability that the next voxel is a non-empty voxel based on the current voxel; here, a binary classification neural network is used to predict the probability that the next voxel is a non-empty voxel based on whether the current voxel is non-empty. Then, voxels with a probability greater than or equal to a preset proportion threshold are identified as predicted non-empty voxels to determine the proportion. For example, voxels with a probability greater than 0.8 are predicted as non-empty voxels, thereby determining the proportion of non-empty voxels in the total target voxels.

[0104] Secondly, based on the aforementioned ratio, determine the number of non-empty unit voxels in the next layer of the current block.

[0105] Here, the ratio is defined as the proportion occupied by non-empty unit voxels in the next layer of the current layer, thereby determining the number of non-empty unit voxels in the next layer.

[0106] Furthermore, geometric information is reconstructed for the next layer of the current layer, based at least on the number of non-empty unit voxels.

[0107] Here, following the previous step, the number of non-empty unit voxels is determined, and the number of non-empty unit voxels that meet this number in the next layer is predicted. Based on the predicted non-empty unit voxels and the unpredicted non-empty unit voxels, the geometric information of the next layer of the current layer is reconstructed.

[0108] Finally, the geometric information and corresponding attribute information of the point cloud data of the next layer are determined to obtain the hidden layer features.

[0109] Here, after reconstructing the next layer, the geometric information and corresponding attribute information of the cloud data in that layer are determined; after reconstructing each layer of the current block, the geometric information and corresponding attribute information of that layer can be determined; the geometric information and corresponding attribute information of the multiple layers are used as the hidden layer features of the current block.

[0110] In this embodiment, the number of non-empty voxels in the next layer is predicted by the proportion of non-empty voxels in the current layer, so that the number of non-empty voxels in the next layer is closer to the true value. The preset proportion threshold is adjusted according to the true number of non-empty voxels in the point cloud, thereby enabling the setting of an adaptive threshold using the number of voxels during classification and reconstruction, which improves the encoding performance.

[0111] In some embodiments, standards organizations such as the Moving Picture Experts Group (MPEG), the Joint Photographic Experts Group (JPEG), and the Audio Video Coding Standard (AVS) are developing technical standards related to point cloud compression. Among them, the MPEG Point Cloud Compression (PCC) standard is relatively advanced and representative. It includes Geometry-based Point Cloud Compression (G-PCC) and Video-based Point Cloud Compression (V-PCC). Geometric compression in G-PCC is mainly achieved through an octree model and / or a triangular surface model. V-PCC is mainly achieved through 3D-to-2D projection and video compression.

[0112] Based on the content being compressed, point cloud compression can be divided into geometric compression and attribute compression. The technical solution described in this application belongs to geometric compression.

[0113] Similar to the embodiments in this application is an emerging point cloud geometry compression technique utilizing neural networks and deep learning. This research direction is currently in its early exploratory stages. Existing technical resources can be categorized into volumetric model compression techniques based on 3D convolutional neural networks and point cloud compression techniques that directly utilize networks such as PointNet on point sets.

[0114] However, G-PCC has a low compression ratio because it cannot fully extract and transform the geometric structure of point clouds. V-PCC performs better than G-PCC on dense point clouds, but it is still limited by the projection method and cannot fully compress the 3D geometric features, and the encoder has high complexity.

[0115] Related learning-based point cloud geometry compression techniques lack test results that meet standard conditions, and there is a lack of sufficient peer review and publicly available technical and data resources for comparative verification. Furthermore, these various schemes have the following significant drawbacks: techniques that directly compress the point set are limited to small point clouds with a fixed number of points, and cannot be directly applied to complex point clouds in real-world scenes. Point cloud compression techniques based on 3D dense convolution, by converting sparse point clouds into volumetric models for compression, fail to fully exploit the sparse structure of the point cloud, resulting in computational redundancy and low coding performance.

[0116] Based on this, the following will describe an exemplary application of the embodiments of this application in a practical application scenario.

[0117] This application provides a multi-scale point cloud geometry compression method using an end-to-end learning autoencoder framework and sparse convolutional neural networks to construct analytical and synthetic transformations. Point cloud data is represented as coordinates and corresponding attributes in the form of a three-dimensional sparse tensor: {C, F}. The corresponding attribute FX of the input point cloud geometric data X is all 1s, serving as a placeholder. In the encoder, the input X is progressively downsampled to multiple scales after the analytical transformation. During this process, geometric features are automatically extracted and embedded into the attribute F of the sparse tensor. The coordinates CY and feature attributes FY of the final hidden layer representation Y are encoded into binary code streams. In the decoder, the hidden layer representation Y is decoded and then progressively upsampled in the synthetic transformation to output the multi-scale reconstruction result.

[0118] The detailed process and codec structure of the method are as follows: Figure 5A As shown in the figure, AE represents Arithmetic Encoder, and AD (Arithmetic Decoder) represents Arithmetic Decoder;

[0119] Detailed explanation is as follows:

[0120] The encoding and decoding transformation consists of multiple layers of sparse convolutional neural networks: the Inception-Residual Network (IRN) is used to improve the network's feature analysis capabilities. The IRN structure is as follows: Figure 5BAs shown, after each upsampling and downsampling step, a feature extraction module consisting of three IRN units is implemented. Downsampling is achieved through convolutions with a stride of 2 and a kernel size of 2, aggregating the features of voxels within each 2×2×2 spatial unit to a single voxel. After each downsampling, the length, width, and height scales of the point cloud are halved, with a total of three downsampling operations. Upsampling in the decoder is achieved through transposed convolutions with a stride of 2 and a kernel size of 2, dividing one voxel into 2×2×2 voxels, thus doubling the length, width, and height scales of the point cloud. After each upsampling, binary classification is used to retain voxels predicted to be occupied and remove voxels predicted to be empty and their attributes, thereby reconstructing geometric details. Through hierarchical and progressive reconstruction, the coarse point cloud gradually recovers its detailed structure. Figure 5B The REL shown represents the Corrected Linear Unit.

[0121] The detailed explanation of multi-scale hierarchical reconstruction is as follows: Reconstruction can be achieved by distinguishing generated voxels through binary classification. Therefore, for each scale of features in the decoder, a convolutional layer with an output channel of 1 is used to predict the probability of each voxel being occupied. During training, a binary cross-entropy loss function (L0) is used. BCE )Measure classification distortion and use it for training; in hierarchical reconstruction, multi-scale L is used accordingly. BCE ,Right now To achieve multi-scale training; where N represents the number of different scales, and L represents the number of scales. BCE This can be called distortion loss, or distortion loss D as described below. During the inference process, classification is performed by setting a probability threshold. This threshold is not fixed but adaptively set based on the number of points; that is, by sorting, voxels with higher probabilities are selected. Optimal results are often obtained when the number of reconstructed voxels is the same as the number of original voxels. A specific reconstruction process can be referenced... Figure 6 Understanding, such as Figure 6As shown, (a) to (b) represent one downsampling, (b) to (c) represent one downsampling, and (c) to (d) represent one downsampling, that is, (a) to (d) represent three downsampling operations of the point cloud during the encoding process. (e) to (j) represent the layered reconstruction process of the point cloud, and (e), (g), and (i) represent the results of three upsampling operations. The color represents the probability of a voxel being occupied; the closer it is to the light gray shown in (a), the higher the probability of being occupied; the closer it is to the dark gray among the two colors shown in (e), the lower the probability of being occupied. (f), (h), and (j) are the results of probability classification, with three possibilities. Light gray and dark gray represent correct and incorrect results among the predicted voxels, respectively, while black (such as the black among the three colors shown in (h) and (j)) represents voxels that were not correctly predicted. During training, to avoid the impact of unpredicted voxels on subsequent reconstructions, both predicted and unpredicted voxels are retained for the next level of reconstruction.

[0122] The following explains how features are encoded: The coordinates C of the hidden feature Y obtained through analysis and transformation are... Y and attribute F Y They are encoded separately. Coordinate C Y Lossless encoding using a classic octree encoder occupies only a small bit rate. Attribute F Y Quantified Then compression is performed using arithmetic coding. Arithmetic coding relies on a learned entropy model to estimate each... probability As shown in equation (1), this entropy model is obtained through a complete factorization probability density distribution:

[0123]

[0124] In the formula, ψ (i) For each univariate distribution The distribution of . This distribution is related to a uniform probability density. Convolution yields probability values.

[0125] Furthermore, this application also provides a conditional entropy model based on contextual information, assuming that the numerical values ​​of the features follow a Gaussian distribution N(μ). i ,σ i 2 This distribution can be used to obtain the entropy model. To predict the parameters of the Gaussian distribution using context, a context model can be designed based on mask convolution. This model is used to extract contextual information, such as... Figure 5CAs shown, it illustrates the structure of the context model via autoregressive prior, for the current input voxel. A mask convolution with a 5×5×5 kernel is used to mask the voxels following the current voxel, allowing prediction of the current voxel using the preceding voxels. The mean and variance μ and σ of the obtained output are obtained from a normal distribution. Experiments show that the context-based conditional entropy model achieves an average BD-Rate of -7.28% compared to the probability density model based on full factorization on the test set.

[0126] The parameters of the codec need to be obtained through training. The training details are as follows: The dataset used is the ShapeNet dataset. This dataset is sampled to obtain a dense point cloud; and the coordinates of the points in this dense point cloud are quantized to the range [0, 127] for training. The loss function used for training is a weighted sum of distortion loss D and bitrate loss R: J = R + λD;

[0127] Where R is the probability that can be estimated using the entropy model described above. The information entropy is calculated using the following formula: K represents the total number of digits to be encoded (i.e., the values ​​obtained from the convolution transformation); the expression for the distortion loss is: The parameter λ controls the weighting of the bitrate loss R and the distortion loss D. This parameter can be set to any value, such as 0.5, 1, 2, 4, or 6, to obtain models with different bitrates. Training can use the Adaptive Moment Estimation (Adam) optimization algorithm. The loss function decays from 0.0008 to 0.00002, with 32,000 training batches, each containing 8 point clouds.

[0128] This application embodiment was tested on various datasets required by MPEG PCC, including test point clouds of long dresses, red and black, basketball players and Andrews, loots, soldiers and dancers, as well as datasets required by the Joint Photographic Experts Group. Peak signal-to-noise ratio (D1 PSNR) based on point-to-point distance was used as an objective quality evaluation metric. Compared to V-PCC, G-PCC (octree), and G-PCC (trisoup), the BD-rate (Bjontegaard Delta Rate) was -36.93%, -90.46%, and -91.06%, respectively.

[0129] The bitrate curves for the four data points—long dress, red and black, basketball player, and Andrew—are compared to other methods, for example... Figure 7 As shown, from Figure 7 As can be seen from the data, the PSNR obtained by the method provided in this application embodiment is higher than that of other methods for each type of data, regardless of the bit rate. That is, the compression performance obtained by this application embodiment is better.

[0130] Subjective quality comparison of similar bitrates on the Red and Black dataset, for example Figure 8 As shown in the figure, the data above demonstrates that the compression performance of the proposed method is significantly improved compared to V-PCC and G-PCC.

[0131] Furthermore, due to its full adaptation to the sparse and unstructured characteristics of point clouds, the embodiments of this application offer greater flexibility compared to other learning-based point cloud geometric compression methods. They do not require limitations on the number of points or the size of the volumetric model, and can easily handle point clouds of arbitrary sizes. Moreover, the time and storage overhead required for encoding and decoding are significantly reduced compared to volumetric model-based methods. Average tests on Longdress, Loot, Redandblack, and Soldier show that encoding requires approximately 333MB of memory and takes approximately 1.58 seconds, while decoding requires approximately 1273MB of memory and takes approximately 5.4 seconds. The test equipment used was an Intel Core i7-8700KW CPU and an Nvidia GeForce GTX 1070 GPU.

[0132] In this embodiment, a point cloud geometric encoding and decoding scheme based on sparse tensors and sparse convolution is designed. During the encoding and decoding transformation, a multi-scale structure and loss function are applied to provide multi-scale reconstruction. Adaptive threshold setting is performed using the number of points during classification-based reconstruction.

[0133] In some embodiments, the structural parameters of the neural network can be modified, such as increasing or decreasing the number of upsampling and downsampling, and / or changing the number of network layers.

[0134] Based on the foregoing embodiments, the encoder and decoder for point cloud compression provided in this application may include the included modules and the units included in each module, which can be implemented by a processor in an electronic device; of course, they can also be implemented by specific logic circuits; in the implementation process, the processor may be a central processing unit, a microprocessor, a digital signal processor or a field programmable gate array, etc.

[0135] like Figure 9 As shown, this application embodiment provides an encoder 900, including:

[0136] The first acquisition module 901 is used to acquire the current block of the video to be encoded;

[0137] The first determining module 902 is used to determine the geometric information and corresponding attribute information of the point cloud data of the current block;

[0138] The downsampling module 903 is used to downsample the geometric information and the corresponding attribute information using a sparse convolutional network to obtain hidden layer features.

[0139] The first compression module 904 is used to compress the hidden layer features to obtain a compressed bitstream.

[0140] In some embodiments of this application, the first determining module 902 is further configured to: determine the coordinates of any point in the point cloud data in the world coordinate system to obtain the geometric information; and perform feature extraction on the any point to obtain the attribute information corresponding to the geometric information.

[0141] In some embodiments of this application, the downsampling module 903 is further configured to: quantize the geometric information and the corresponding attribute information to obtain unit voxels and a set of unit voxels; determine the number of downsampling steps and the kernel size of the sparse convolutional network; and aggregate the unit voxels in the set of unit voxels according to the number of downsampling steps to obtain the hidden layer features.

[0142] In some embodiments of this application, the downsampling module 903 is further configured to: divide the area occupied by the point cloud into multiple unit aggregation regions according to the downsampling number; aggregate the unit voxels in each unit aggregation region to obtain a target voxel set; and determine the geometric information and corresponding attribute information of each target voxel in the target voxel set to obtain the hidden layer features.

[0143] In some embodiments of this application, the first compression module 904 is further configured to: determine the frequency of occurrence of geometric information in the hidden layer features; adjust the hidden layer features by weighting according to the frequency to obtain adjusted hidden layer features; and encode the adjusted hidden layer features into a binary bitstream to obtain the compressed bitstream.

[0144] In practical applications, such as Figure 10 As shown, this application embodiment also provides an encoder 1000, including: a first memory 1001 and a first processor 1002; the first memory 1001 stores a computer program that can run on the first processor 1002, and the first processor 1002 executes the program to implement a point cloud compression method on the encoder side.

[0145] like Figure 11As shown, this application embodiment provides a decoder 1100, including:

[0146] The second acquisition module 1101 is used to acquire the current block of the video to be decompressed;

[0147] The second determining module 1102 is used to determine the geometric information and corresponding attribute information of the point cloud data of the current block;

[0148] The upsampling module 1103 is used to upsample the geometric information and the corresponding attribute information using a transposed convolutional network to obtain hidden layer features.

[0149] The decompression module 1104 is used to decompress the hidden layer features to obtain a decompressed bitstream.

[0150] In some embodiments of this application, the second acquisition module 1101 is further configured to: determine the number of points in the point cloud data of the current block; determine a point cloud region in the current block where the number of points is greater than or equal to a preset value; and determine the geometric information and corresponding attribute information of the point cloud data in the point cloud region.

[0151] In some embodiments of this application, the second determining module 1102 is further configured to: determine the coordinate value of any point in the point cloud data in the world coordinate system to obtain the geometric information; and perform feature extraction on the any point to obtain the attribute information corresponding to the geometric information.

[0152] In some embodiments of this application, the upsampling module 1103 is further configured to: determine the target voxel to which the geometric information and the corresponding attribute information belong; determine the number of upsampling operations based on the upsampling stride and kernel size of the transposed convolutional network; and decompress the target voxel into multiple unit voxels according to the number of upsampling operations to obtain the hidden layer features.

[0153] In some embodiments of this application, the upsampling module 1103 is further configured to: determine the unit aggregation region occupied by the target voxel; within the unit aggregation region, decompress the target voxel into multiple unit voxels according to the number of upsampling; and determine the geometric information and corresponding attribute information of each unit voxel to obtain the hidden layer features.

[0154] In some embodiments of this application, the upsampling module 1103 is further configured to: determine the proportion of non-empty voxels in the current layer of the current block to the total target voxels; determine the number of non-empty voxels in the next layer of the current block based on the proportion; reconstruct the geometric information of the next layer of the current block based at least on the number of non-empty voxels; and determine the geometric information and corresponding attribute information of the point cloud data of the next layer to obtain the hidden layer features.

[0155] In some embodiments of this application, the upsampling module 1103 is further configured to: employ a binary classification neural network to determine the probability that the next unit voxel is a non-empty voxel based on the current unit voxel; and determine the voxels with a probability greater than or equal to a preset proportion threshold as the non-empty unit voxels, thereby determining the proportion.

[0156] In some embodiments of this application, the decompression module 1104 is further configured to: determine the frequency of occurrence of geometric information in the hidden layer features; perform weighted adjustment on the hidden layer features according to the frequency to obtain adjusted hidden layer features; and decompress the adjusted hidden layer features into a binary code stream to obtain the decompressed code stream.

[0157] In practical applications, such as Figure 12 As shown in the illustration, this application also provides a decoder 1200, comprising:

[0158] Second memory 1201 and second processor 1202;

[0159] The second memory 1201 stores a computer program that can run on the second processor 1202, which implements a point cloud compression method on the decoder side when executing the program.

[0160] Accordingly, embodiments of this application provide a storage medium storing a computer program thereon. When the computer program is executed by a first processor, it implements a point cloud compression method for an encoder; or, when the computer program is executed by a second processor, it implements a point cloud compression method for a decoder.

[0161] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0162] It should be noted that, in the embodiments of this application, if the above-described point cloud compression method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, laptop computer, desktop computer, robot, drone, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0163] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0164] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0165] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0167] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0168] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0169] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0170] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, laptop computer, desktop computer, robot, drone, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0171] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0172] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0173] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0174] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A point cloud compression method, characterized in that, The method includes: Get the current block of the video to be decompressed; Determine the geometric information and corresponding attribute information of the point cloud data of the current block; wherein the attribute information includes at least one of the following: color, brightness, and pixel value; Determine the target voxel to which the geometric information and corresponding attribute information belong; The number of upsampling iterations is determined based on the upsampling stride and kernel size of the transposed convolutional network. Determine the unit aggregation region occupied by the target voxel; Within the unit aggregation region, the target voxel is decompressed into multiple unit voxels according to the number of upsampling operations; Determine the proportion of non-empty unit voxels in the current layer of the current block to the total target voxels; Based on the aforementioned ratio, determine the number of non-empty unit voxels in the next layer of the current block; At least based on the number of non-empty unit voxels, the geometric information of the next layer of the current layer is reconstructed; Determine the geometric information and corresponding attribute information of the point cloud data in the next layer; The geometric information and corresponding attribute information of the point cloud data of the current layer and the next layer are determined as the hidden layer features of the current block; The hidden layer features are decompressed to obtain a decompressed bitstream.

2. The method according to claim 1, characterized in that, After obtaining the current block of the video to be decompressed, the method further includes: Determine the number of points in the point cloud data of the current block; In the current block, identify a point cloud region where the number of points is greater than or equal to a preset value; Determine the geometric information and corresponding attribute information of the point cloud data in the point cloud region.

3. The method according to claim 1, characterized in that, Determining the geometric information and corresponding attribute information of the point cloud data includes: Determine the coordinates of any point in the point cloud data in the world coordinate system to obtain the geometric information; Feature extraction is performed on any point to obtain the attribute information corresponding to the geometric information.

4. The method according to claim 1, characterized in that, The method further includes: determining the proportion of non-empty voxels in each target voxel to the total number of target voxels; Determining the proportion of non-empty voxels in each target voxel to the total number of target voxels includes: A binary classification neural network is used to determine the probability that the next unit voxel is a non-empty voxel based on the current unit voxel. Voxels with a probability greater than or equal to a preset percentage threshold are identified as non-empty unit voxels, thereby determining the percentage.

5. The method according to claim 1 or 4, characterized in that, The step of decompressing the hidden layer features to obtain a decompressed bitstream includes: Determine the probability of the occurrence of geometric information in the hidden layer features; Based on the probability, the hidden layer features are weighted and adjusted to obtain the adjusted hidden layer features; The adjusted hidden layer features are decompressed into a binary bitstream to obtain the decompressed bitstream.

6. A decoder, characterized in that, include: The second acquisition module is used to acquire the current block of the video to be decompressed; The second determining module is used to determine the geometric information and corresponding attribute information of the point cloud data of the current block; wherein the attribute information includes at least one of the following: color, brightness and pixel value; An upsampling module is used to determine the target voxel to which the geometric information and corresponding attribute information belong; determine the number of upsampling operations based on the upsampling stride and kernel size of the transposed convolutional network; determine the unit aggregation region occupied by the target voxel; within the unit aggregation region, decompress the target voxel into multiple unit voxels according to the number of upsampling operations; determine the proportion of non-empty unit voxels in the current layer of the current block to the total target voxels; determine the number of non-empty unit voxels in the next layer of the current block according to the proportion; reconstruct the geometric information of the next layer of the current layer based at least on the number of non-empty unit voxels; and determine the geometric information and corresponding attribute information of the point cloud data of the next layer. The geometric information and corresponding attribute information of the point cloud data of the current layer and the next layer are determined as the hidden layer features of the current block; The decompression module is used to decompress the hidden layer features to obtain a decompressed bitstream.

7. The decoder according to claim 6, characterized in that, The second acquisition module is also used for: Determine the number of points in the point cloud data of the current block; In the current block, identify a point cloud region where the number of points is greater than or equal to a preset value; Determine the geometric information and corresponding attribute information of the point cloud data in the point cloud region.

8. The decoder according to claim 6, characterized in that, The second determining module is also used for: Determine the coordinates of any point in the point cloud data in the world coordinate system to obtain the geometric information; Feature extraction is performed on any point to obtain the attribute information corresponding to the geometric information.

9. The decoder according to claim 6, characterized in that, The upsampling module is also used for: A binary classification neural network is used to determine the probability that the next unit voxel is a non-empty voxel based on the current unit voxel. Voxels with a probability greater than or equal to a preset percentage threshold are identified as non-empty unit voxels, thereby determining the percentage.

10. The decoder according to claim 6 or 9, characterized in that, The decompression module is also used for: Determine the probability of the occurrence of geometric information in the hidden layer features; Based on the probability, the hidden layer features are weighted and adjusted to obtain the adjusted hidden layer features; The adjusted hidden layer features are decompressed into a binary bitstream to obtain the decompressed bitstream.

11. A decoder, characterized in that, include: Second memory and second processor; The second memory stores a computer program that can run on a second processor, which, when executing the program, implements the point cloud compression method according to any one of claims 1 to 5.

12. A storage medium, characterized in that, It stores a computer program, which, when executed by a second processor, implements the point cloud compression method according to any one of claims 1 to 5.

Images