Point cloud processing method, apparatus, device, and storage medium

Abstract

Embodiments of the present application disclose a point cloud processing method, device and equipment, and a storage medium. The method comprises: obtaining code stream data of point cloud data, parsing the code stream data to obtain grouping information of the point cloud data, optimizing a grouping prediction process based on an association relationship between groups to obtain attribute information of the point cloud data, and presenting the point cloud data according to the attribute information of the point cloud data. It can be seen that the grouping prediction process is optimized based on the association relationship between groups, which can make the prediction result closer to the actual result, reduce the amount of data to be coded and decoded in the coding and decoding stage, and thus improve the coding and decoding efficiency of point cloud attribute information.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a point cloud processing method, a point cloud processing device, a computer device, and a computer-readable storage medium. Background Technology

[0002] With advancements in scientific research, it is now possible to obtain large quantities of high-precision point clouds at relatively low cost and within a short timeframe. A point cloud can contain multiple points, each possessing geometric and attribute information. To improve the transmission efficiency of point clouds, the relevant information is typically encoded before transmission. Specifically, the encoding end encodes the point cloud data and transmits the encoded bitstream data to the decoding end, which then decodes the bitstream data to reconstruct the point cloud information. Due to the large volume of point cloud data, improving the encoding and decoding efficiency has become a hot research topic. Summary of the Invention

[0003] This application provides a point cloud processing method, apparatus, device, computer-readable storage medium, and product that can improve the encoding and decoding efficiency of point cloud attribute information.

[0004] On one hand, embodiments of this application provide a point cloud processing method, including:

[0005] Obtain the bitstream data of point cloud data;

[0006] The bitstream data is parsed to obtain the grouping information of the point cloud data. The grouping information is used to indicate the relationship between the various groups of the point cloud data.

[0007] The grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data;

[0008] The point cloud data is presented according to its attribute information.

[0009] In this embodiment, the bitstream data of point cloud data is acquired, and the bitstream data is parsed to obtain the grouping information of the point cloud data. Based on the correlation between each group, the grouping prediction process is optimized to obtain the attribute information of the point cloud data. The point cloud data is then presented according to this attribute information. It is evident that optimizing the grouping prediction process through the correlation between each group can make the prediction results closer to the actual results, reduce the amount of data to be decoded in the decoding stage, and thus improve the decoding efficiency of point cloud attribute information.

[0010] On one hand, embodiments of this application provide a point cloud processing method, including:

[0011] Obtain the point cloud data to be encoded;

[0012] The points in the point cloud data are divided to obtain the grouping information of the point cloud data. The grouping information is used to indicate the relationship between the corresponding groups of the point cloud data.

[0013] The grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data;

[0014] The attribute information of the point cloud data is encoded to obtain the bitstream data of the point cloud data.

[0015] In this embodiment, point cloud data to be encoded is acquired, the points in the point cloud data are divided to obtain grouping information, the grouping prediction process is optimized based on the correlation between each group to obtain attribute information of the point cloud data, and the attribute information of the point cloud data is encoded according to the correlation between each group to obtain the bitstream data of the point cloud data. It can be seen that optimizing the grouping prediction process through the correlation between each group can make the prediction results closer to the actual results, reduce the amount of data to be encoded in the encoding stage, and thus improve the encoding efficiency of point cloud attribute information.

[0016] On one hand, embodiments of this application provide a point cloud processing apparatus, which includes:

[0017] The acquisition unit is used to acquire the bitstream data of point cloud data.

[0018] The processing unit is used to parse the bitstream data to obtain the grouping information of the point cloud data. The grouping information is used to indicate the relationship between the various groups corresponding to the point cloud data.

[0019] And it is used to optimize the grouping prediction process based on the correlation between each group to obtain the attribute information of the point cloud data;

[0020] And it is used to present point cloud data according to the attribute information of point cloud data.

[0021] On one hand, embodiments of this application provide a point cloud processing apparatus, which includes:

[0022] The acquisition unit is used to acquire point cloud data to be encoded.

[0023] The processing unit is used to divide the points in the point cloud data to obtain the grouping information of the point cloud data. The grouping information is used to indicate the relationship between the various groups of the point cloud data.

[0024] And it is used to optimize the grouping prediction process based on the correlation between each group to obtain the attribute information of the point cloud data;

[0025] And it is used to encode the attribute information of point cloud data to obtain the bitstream data of point cloud data.

[0026] Accordingly, this application provides a computer device comprising:

[0027] Memory, which stores computer programs;

[0028] The processor is used to load computer programs to implement the point cloud processing method described above.

[0029] Accordingly, this application provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the above-described point cloud processing method.

[0030] Accordingly, this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned point cloud processing method.

[0031] In this embodiment, on one hand, the bitstream data of point cloud data is acquired, and the bitstream data is parsed to obtain the grouping information of the point cloud data. Based on the correlation between each group, the grouping prediction process is optimized to obtain the attribute information of the point cloud data. The point cloud data is then presented according to the attribute information. On the other hand, the point cloud data to be encoded is acquired, the points in the point cloud data are divided to obtain the grouping information of the point cloud data, and the grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data. The attribute information of the point cloud data is then encoded according to the correlation between each group to obtain the bitstream data of the point cloud data. By optimizing the grouping prediction process through the correlation between each group, the prediction result can be closer to the actual result, reducing the amount of data to be encoded and decoded in the encoding and decoding stage, thereby improving the encoding and decoding efficiency of point cloud attribute information. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1a A schematic diagram of an encoding framework provided in an embodiment of this application;

[0034] Figure 1b This application provides a schematic diagram illustrating the relationship between the current point's block and its parent block neighbors.

[0035] Figure 1c A schematic diagram of a binary tree constructed based on Hilbert order, provided for an embodiment of this application;

[0036] Figure 2 An architecture diagram of a point cloud processing system provided in an embodiment of this application;

[0037] Figure 3 A point cloud processing method provided in this application embodiment;

[0038] Figure 4 Another point cloud processing method provided in the embodiments of this application;

[0039] Figure 5 This application provides yet another point cloud processing method.

[0040] Figure 6 Another point cloud processing method provided in the embodiments of this application;

[0041] Figure 7 This is a schematic diagram of the structure of a point cloud processing device provided in an embodiment of this application;

[0042] Figure 8 This is a schematic diagram of another point cloud processing device provided in an embodiment of this application;

[0043] Figure 9 This is a schematic diagram of the structure of a decoding device provided in an embodiment of this application;

[0044] Figure 10 This is a schematic diagram of the structure of an encoding device provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0046] To better understand the technical solutions provided in the embodiments of this application, the key terms involved in the embodiments of this application will be introduced first:

[0047] (1) Point Cloud. A point cloud is a set of discrete points in space that are randomly distributed and represent the spatial structure and surface properties of a three-dimensional object or scene. Point clouds can be classified into different categories according to different classification criteria. For example, according to the acquisition method, they can be divided into dense point clouds and sparse point clouds. Also, according to the temporal type of point clouds, they can be divided into static point clouds and dynamic point clouds.

[0048] (2) Point Cloud Data. Point cloud data is composed of the geometric and attribute information of each point in a point cloud. Geometric information, also known as 3D positional information, refers to the spatial coordinates (x, y, z) of a point in the point cloud. This can include the coordinate values ​​of the point along each coordinate axis of a 3D coordinate system, such as x (x-axis), y (y-axis), and z (z-axis). Attribute information of a point in the point cloud can include at least one of the following: color information, material information, and laser reflection intensity information (also known as reflectivity). Typically, each point in the point cloud has the same number of attribute information. For example, each point in the point cloud can have both color information and laser reflection intensity information, or it can have color information, material information, and laser reflection intensity information.

[0049] (3) Point Cloud Compression (PCC). Point cloud encoding refers to the process of encoding the geometric and attribute information of each point in a point cloud to obtain a compressed bitstream. Point cloud encoding can include two main processes: geometric information encoding and attribute information encoding. Currently, mainstream point cloud encoding technologies can be divided into geometric structure-based point cloud encoding and projection-based point cloud encoding, depending on the type of point cloud. Here, we will take G-PCC (Geometry-based Point Cloud Compression) in MPEG (Moving Picture Expert Group, international audio and video codec standard) and AVS-PCC (Audio Video Coding Standard, China's national video codec standard) as examples for introduction.

[0050] The encoding frameworks of G-PCC and AVS-PCC are largely the same, such as Figure 1aAs shown, the process can be divided into geometric information encoding and attribute information encoding. The geometric information encoding process encodes the geometric information of each point in the point cloud data to obtain a geometric bitstream; the attribute information encoding process encodes the attribute information of each point in the point cloud data to obtain an attribute bitstream; the geometric bitstream and the attribute bitstream together form the compressed bitstream of the point cloud data.

[0051] The main operations and processing steps for geometric information encoding are described below:

[0052] ① Pre-processing: This can include coordinate transformation and voxelization. By scaling and translation, point cloud data in 3D space is converted into integer form, and its smallest geometric position is moved to the origin.

[0053] ② Octree Encoding: An octree is a tree-like data structure used in 3D spatial partitioning. It uniformly divides a predefined bounding box, with each node having eight child nodes. Occupancy codes ("1" and "0") are used to indicate the occupancy of each child node, providing the bitstream of the point cloud's geometric information. Bounding box encoding is an algorithm for finding the optimal bounding space for a discrete point set. Its basic idea is to approximate complex geometric objects using a slightly larger, simpler geometric shape (called a bounding box).

[0054] ③ Geometric Entropy Encoding: This method uses statistical compression encoding on the occupancy code information of an octree to output a binary (0 or 1) compressed bitstream. Statistical coding is a lossless coding method that can effectively reduce the bit rate required to represent the same signal. A commonly used statistical coding method is Content Adaptive Binary Arithmetic Coding (CABAC).

[0055] The main operations and processing steps for encoding attribute information can be found in the following description:

[0056] ① Attribute Recoloring: In lossy encoding, after encoding geometric information, the encoding end needs to decode and reconstruct the geometric information, that is, restore the geometric information of each point in the point cloud. Attribute information corresponding to one or more neighboring points in the original point cloud is used as the attribute information of the reconstructed point.

[0057] ② Attribute Transformation Encoding: Attribute transformation algorithms (such as DCT, Haar, etc.) are used to group and transform attribute information, and the transformation coefficients are quantized; through inverse quantization, the attribute reconstruction value is obtained after inverse transformation; the difference between the original attribute and the attribute reconstruction value is calculated to obtain the attribute residual and quantized; the quantized transformation coefficients and attribute residuals are encoded.

[0058] ③ Attribute Quantization: The fineness of quantization is usually determined by the quantization parameters. In attribute prediction coding, entropy encoding is performed on the quantized prediction residual information; in attribute transformation coding and attribute prediction transformation coding, entropy encoding is performed on the quantized transformation coefficients.

[0059] ④ Attribute Entropy Coding: The quantized prediction residuals or transform coefficients are generally compressed using run-length coding and arithmetic coding. The corresponding coding mode, quantization parameters, and other information are also encoded using an entropy encoder.

[0060] (4) Point Cloud Decoding. Point cloud decoding refers to the process of decoding the compressed bitstream obtained from point cloud encoding to reconstruct the point cloud; more specifically, it refers to the process of reconstructing the geometric and attribute information of each point in the point cloud based on the geometric bitstream and attribute bitstream in the compressed bitstream. After obtaining the compressed bitstream at the decoding end, for the geometric bitstream, entropy decoding is first performed to obtain the quantized geometric information of each point in the point cloud, and then inverse quantization is performed to reconstruct the geometric information of each point in the point cloud. For the attribute bitstream, entropy decoding is first performed to obtain the quantized prediction residual information or quantized transform coefficients of each point in the point cloud; then the quantized prediction residual information is inverse quantized to obtain the reconstructed residual information, and the quantized transform coefficients are inverse quantized to obtain the reconstructed transform coefficients. The reconstructed transform coefficients are inversely transformed to obtain the reconstructed residual information. The attribute information of each point in the point cloud can be reconstructed based on the reconstructed residual information of each point in the point cloud. The reconstructed attribute information of each point in the point cloud is matched one-to-one with the reconstructed geometric information in sequence to reconstruct the point cloud.

[0061] In addition, this application also relates to the following technologies:

[0062] (1) Attribute prediction:

[0063] ① Point cloud reordering, the specific process is as follows:

[0064] Obtain the coordinates (x, y, z) of the point cloud, generate a Morton code or Hilbert code for each point based on the space-filling curve, and sort them according to the encoding order during the encoding process (e.g., ascending order) to obtain the Morton order or Hilbert order. Taking the Morton code of geometric coordinates as an example, the geometric position of a point in the point cloud is represented by three-dimensional Cartesian coordinates (X, Y, Z). Each coordinate value is represented by N bits, and the coordinates (X, Y, Z) of the k-th point are... k ,Y k Z k This can be represented as:

[0065]

[0066]

[0067]

[0068] The Morton code corresponding to the k-th point can be represented as follows:

[0069]

[0070] Represent each set of three bits using an octal number. If n = 0, 1, ..., N-1, then the Morton code corresponding to the k-th point can be represented as:

[0071]

[0072] ② Order-based neighbor selection, the specific process is as follows:

[0073] Order-based neighbor selection includes distance-based selection methods and selection methods based on spatial relationships and distance. These methods are described below:

[0074] i) Distance-based method, the specific steps are as follows:

[0075] In the first P neighbor candidates of the Morton or Hilbert order, P can be determined by the maximum number of neighbors field (maxNumOfNeighbours). Each neighbor candidate is calculated (the i-th neighbor candidate is represented as (x...). i ,y i ,z i The Manhattan distance d to the current point to be decoded (x,y,z) is |xx|. i |+|yy i |+|zz iAfter obtaining the Manhattan distance from each neighbor candidate point to the current point to be decoded (x,y,z), the k neighbor candidate points with the shortest Manhattan distance can be taken as the neighbors of the current point to be decoded; or, determine the maximum distance value among the k points with the shortest Manhattan distance, and take all neighbor candidate points with Manhattan distance less than or equal to the maximum distance value as the neighbors of the current point, and finally determine the m points with the closest Manhattan distance as the nearest neighbors of the current point to be decoded, where m and k are positive integers and m≤k.

[0076] ii) Selection method based on spatial relationships and distance, the specific steps are as follows:

[0077] First, determine the initial block size, which is to determine the initial right shift bit N of the codeword corresponding to each point. Based on this, N+3 is the corresponding parent block range.

[0078] Then, the point cloud is traversed in a certain order, such as... Figure 1b As shown, the current point to be decoded (point P) searches for its nearest neighbor among the decoded points (limited to the first k points) within the range of its parent block of block B and its neighboring blocks that are coplanar, collinear, or have the same point.

[0079] If the number of neighbor points of the current point to be decoded determined by the above method is insufficient, then the neighbor point selection shall be performed according to the distance-based method in i).

[0080] ③ The predicted value is calculated as follows:

[0081] Let the attribute reconstruction value of each neighbor be... If j = 0, 1, ..., k, then the predicted attribute value of the current point for:

[0082]

[0083] i) Distance-based weighted calculation:

[0084] In one implementation, the reciprocal of the Manhattan distance between the current point to be decoded and its neighboring points is used as the weight, and the weighted average of the attribute reconstruction values ​​of the k neighboring points is finally calculated to obtain the attribute prediction value. Let the geometric coordinates of the current point to be decoded be (x... i y i , z i The geometric coordinates of the j-th neighbor point are (x, y). ij y ij , z ij Then the weight w of the j-th neighbor point is... ij for:

[0085]

[0086] In another implementation, if different weights are used for the components in the x, y, and z directions, then the weight w of the j-th neighbor point... ij for:

[0087]

[0088] ii) Weighted calculation based on distance and other parameters:

[0089] The weight of each neighbor of the current point to be decoded is w = 1 / d. The optimized weight of the neighbor candidate point whose distance is equal to the maximum distance value is wm = (1 / d) * dwm, where the size of dwm is the minimum value between Qstep (attribute quantization step size) and the number of neighbor candidate points whose distance is equal to the maximum distance value.

[0090] (2) Attribute transformation:

[0091] ① Attribute encoding based on wavelet transform:

[0092] i) Construct a binary tree

[0093] First, the point cloud is reordered using Hilbert's order, and an N-level binary tree is constructed based on the one-dimensional permutation, where N is a positive integer. A bottom-up construction method is used, as detailed below:

[0094] Suppose the current point cloud has M points. These M points are the nodes of the lowest level (level N) of the binary tree. Adjacent points are merged sequentially to form their parent nodes (i.e., points 1 and 2 are merged, points 3 and 4 are merged, and so on). These parent nodes form the nodes of level N-1. These nodes are arranged in the order of merging (i.e., the parent nodes of points 1 and 2 are in the first position, the parent nodes of points 3 and 4 are in the second position, and so on). For two adjacent nodes in level N-1, they are merged sequentially to form their parent nodes, which are the nodes of level N-2. These parent nodes are arranged in the order of merging. If the number of nodes in level n is odd, the last node in the hierarchy directly becomes a node in level n-1 (n = 2, 3, ..., N).

[0095] Merge nodes at each level using the method described above, stopping when there is only one node left in a level. This level is the root node of the tree (level 1), resulting in an N-level binary tree, as shown below. Figure 1c As shown.

[0096] ii) Encoding process

[0097] For the N-level binary tree constructed above, each leaf node of the binary tree contains a point cloud point. The attributes of the point cloud point are defined as the first attribute coefficient of each leaf node.

[0098] Next, transformation calculations are performed on each node in different levels of the N-level binary tree structure. The transformations begin at level N-1 of the N-level binary tree and end at level 1.

[0099] For the nth level of a binary tree, where n = 1, 2, ..., N-1, perform transformation calculations for each target node; if the target node has two child nodes, the transformation matrix is ​​T, which can be expressed as:

[0100]

[0101] Transform the first attribute coefficients of the two child nodes, and define the resulting transformed coefficients as the first and second attribute coefficients of the target node. If the target node has one child node, and the target node only has a first attribute coefficient and no second attribute coefficient, its first attribute coefficient is equal to the first attribute coefficient of its child node multiplied by [the transform coefficient].

[0102] According to the above transformation method, the first transformation coefficient of the final output root node and the second transformation coefficient of other layer nodes are used as transformation coefficients, and the transformation coefficients are quantized and encoded.

[0103] ② Attribute encoding of prediction and transformation fusion:

[0104] First, the point cloud to be encoded, after color space conversion and recoloring, is reordered according to its Hilbert code in ascending order. Then, adaptive grouping and corresponding K-ary transforms are applied. The specific process is as follows:

[0105] i) Point cloud grouping:

[0106] The ordered point cloud based on the space-filling curve is grouped sequentially, with points having the same first L bits of the Hilbert code grouped together. When the number of points in a group exceeds a preset limit, the grouping is further subdivided. Furthermore, the grouping can be adjusted based on the number of points in the preceding group.

[0107] ii) Within-group prediction:

[0108] The prediction point selection range is the P points preceding the first point in the current group, where P can be determined by the maximum number of neighbors (maxNumOfNeighbours). Predictions are made point-by-point for these P points, and a uniform prediction value is selected or calculated for each group. For example, the prediction value of the first point in the current group can be used as the prediction value for the current group.

[0109] iii) Based on residual value transformation:

[0110] For each set of points, the predicted attribute residual values ​​are subjected to a K_i-ary Discrete Cosine Transform (DCT), where K_i = 2…8. The transform coefficients are then quantized and entropy-encoded. When K_i = 1, no transform calculation is required; the attribute residual values ​​are directly quantized and entropy-encoded. The resulting transform coefficients are then quantized.

[0111] (3) Entropy coding:

[0112] Binarization and processing are performed on the quantized (lossy) signed attribute prediction residuals or transform coefficients. Entropy coding can employ variable-length coding, context-based adaptive binary arithmetic coding, run-length coding, etc., specifically:

[0113] ① Variable-length encoding:

[0114] The residuals or coefficients to be encoded are represented using codewords of varying lengths. The code length needs to be designed based on the probability of symbol occurrence. Commonly used variable-length codes include exponential Golomb coding and arithmetic coding.

[0115] ② Context-based adaptive binary arithmetic coding (CABAC) mainly includes the following steps:

[0116] Binarization: CABAC uses binary arithmetic encoding, meaning that only two digits (1 or 0) are encoded. A non-binary numerical symbol, such as a conversion coefficient or motion constant, is first binarized or converted into a binary codeword before arithmetic encoding. This process is similar to converting a numerical value into a variable-length codeword. In the actual implementation, this binary codeword is further encoded by an arithmetic encoder before transmission.

[0117] Context model selection: The context model is a probabilistic model selected based on the statistics of the most recently encoded data symbols. This model stores the probability that each 'bin' is 1 or 0.

[0118] Arithmetic encoding: The arithmetic encoder encodes each 'bin' according to the selected probability model.

[0119] Probability Update: The selected context model is updated based on the actual encoded value. For example, if the value of 'bin' is 1, then the frequency count of 1 will increase.

[0120] ③ Run-length coding:

[0121] The number of consecutive points with a specific symbol in the statistical data is denoted as the run-length. If the current symbol is zero, the run-length value is incremented by 1; if it is non-zero, the run-length value is first encoded, then the prediction residual for that non-zero attribute is encoded, and finally the run-length value is set to 0 to restart the counting. The residual can be encoded using common entropy coding methods, such as variable-length coding or CABAC.

[0122] Specifically, travel coding is divided into two types: fixed-length travel coding and variable-length travel coding. Fixed-length travel coding means that the number of binary bits used to encode the travel is fixed. Variable-length travel coding means that different ranges of travel are encoded using different numbers of binary bits, and a flag bit is added to indicate the number of binary bits used.

[0123] Based on the above descriptions regarding point clouds, point cloud data, point cloud encoding, point cloud decoding, attribute prediction, attribute transformation, and entropy encoding, embodiments of this application provide a point cloud processing scheme. For the decoding device, it acquires the bitstream data of the point cloud data, parses the bitstream data to obtain the grouping information of the point cloud data, optimizes the grouping prediction process based on the correlation between each group, obtains the attribute information of the point cloud data, and presents the point cloud data according to the attribute information. For the encoding device, it acquires the point cloud data to be encoded, divides the points in the point cloud data to obtain the grouping information of the point cloud data, optimizes the grouping prediction process based on the correlation between each group, obtains the attribute information of the point cloud data, and encodes the attribute information of the point cloud data according to the correlation between each group to obtain the bitstream data of the point cloud data. Optimizing the grouping prediction process through the correlation between each group can make the prediction result closer to the actual result, reduce the amount of data to be encoded and decoded in the encoding and decoding stages, and thus improve the encoding and decoding efficiency of point cloud attribute information.

[0124] The point cloud processing solution provided in this application embodiment can also be combined with cloud technologies such as cloud computing and cloud storage. Cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. Cloud computing can provide powerful computing support for the encoding and decoding stages of point cloud attributes, thus greatly improving the encoding and decoding efficiency of point cloud attributes. Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to aggregate a large number of various types of storage devices (also called storage nodes) in a network through application software or application interfaces to work collaboratively and jointly provide data storage and business access functions. Cloud storage can provide powerful storage support for the encoding and decoding stages of point cloud attributes, thus further improving the encoding and decoding efficiency of point cloud attributes.

[0125] Based on the above description, the following is combined with Figure 2 A point cloud processing system suitable for implementing the point cloud processing scheme provided in the embodiments of this application is described. For example... Figure 2 As shown, the point cloud processing system 20 may include an encoding device 201 and a decoding device 202. The encoding device 201 can be a terminal or a server, and the decoding device 202 can also be a terminal or a server. A communication connection can be established between the encoding device 201 and the decoding device 202. The terminal can be a smartphone, tablet, laptop, desktop computer, vehicle terminal, smart home appliance, unmanned aerial vehicle, wearable smart device, etc., but is not limited to these. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0126] (1) For encoding device 201:

[0127] Encoding device 201 can acquire point cloud data (i.e., the geometric and attribute information of each point in the point cloud). Point cloud data can be acquired through scene capture or device generation. Scene capture point cloud data refers to acquiring point cloud data from a real-world visual scene through a capture device associated with encoding device 201. The capture device provides point cloud data acquisition services for encoding device 201 and can include, but is not limited to, any of the following: camera equipment, sensing equipment, and scanning equipment. Camera equipment can include ordinary cameras, stereo cameras, and light field cameras, etc. Sensing equipment can include laser equipment, radar equipment, etc., and scanning equipment can include 3D laser scanning equipment, etc. The capture device associated with encoding device 201 can refer to a hardware component installed in encoding device 201, such as a camera or sensor on a terminal. Alternatively, the capture device associated with encoding device 201 can refer to a hardware device connected to encoding device 201, such as a camera connected to a server. Device generation point cloud data refers to encoding device 201 generating point cloud data based on virtual objects (e.g., virtual 3D objects and virtual 3D scenes obtained through 3D modeling).

[0128] After generating point cloud data, the encoding device 201 can encode the point cloud data to obtain bitstream data. Specifically, during the encoding process, the encoding device 201 can divide the points in the point cloud data to obtain grouping information. The grouping information is used to indicate the correlation between the corresponding groups of the point cloud data. Based on the correlation between the groups, the group prediction process is optimized to obtain the attribute information of the point cloud data. The attribute information of the point cloud data is then encoded to obtain the bitstream data of the point cloud data. After obtaining the bitstream data, the encoding device 201 transmits the bitstream data to the decoding device 202.

[0129] (2) For decoding device 202:

[0130] After receiving the bitstream data transmitted by the encoding device 201, the decoding device 202 can decode the bitstream data and present point cloud data based on the decoding result. Specifically, during the decoding process, the decoding device 202 can parse the bitstream data to obtain the grouping information of the point cloud data. The grouping information is used to indicate the correlation between the various groups corresponding to the point cloud data. Based on the correlation between the various groups, the group prediction process is optimized to obtain the attribute information of the point cloud data, and the point cloud data is presented according to the attribute information of the point cloud data.

[0131] In this embodiment, on one hand, the bitstream data of point cloud data is acquired, the bitstream data is parsed to obtain the grouping information of the point cloud data, the grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data, and the point cloud data is presented according to the attribute information of the point cloud data. On the other hand, the point cloud data to be encoded is acquired, the points in the point cloud data are divided to obtain the grouping information of the point cloud data, the grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data, and the attribute information of the point cloud data is encoded according to the correlation between each group to obtain the bitstream data of the point cloud data. By optimizing the grouping prediction process through the correlation between each group, the prediction result can be closer to the actual result, reducing the amount of data to be encoded and decoded in the encoding and decoding stage, thereby improving the encoding and decoding efficiency of point cloud attribute information. It is understood that the point cloud processing system described in this embodiment is for the purpose of more clearly illustrating the technical solution of this embodiment and does not constitute a limitation on the technical solution provided in this embodiment. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solution provided in this embodiment is also applicable to similar technical problems.

[0132] Based on the above description of the point cloud processing system, the point cloud processing solution provided in the embodiments of this application will be described in more detail below with reference to the accompanying drawings.

[0133] Please see Figure 3 , Figure 3 This application provides a point cloud processing method, which can be executed by a computer device. Specifically, the computer device may be... Figure 2 The decoding device 202 in the point cloud processing system 20 shown. For example... Figure 3 As shown, the point cloud processing method may include the following steps S301-S304:

[0134] S301, Obtain the bitstream data of point cloud data.

[0135] The computer device can acquire the bitstream data of point cloud data in two ways: either by acquiring it in real time from the encoding device, or by downloading the complete bitstream data from the encoding device or server.

[0136] S302. Parse the bitstream data to obtain the grouping information of the point cloud data.

[0137] The grouping information of point cloud data is used to indicate the relationship between the various groups of point cloud data. The relationship between the various groups may include, but is not limited to: the encoding and decoding order of the various groups, and the spatial relationship of the points in the various groups.

[0138] S303. Optimize the group prediction process based on the correlation between each group to obtain the attribute information of the point cloud data.

[0139] The method for predicting the attributes of points in point cloud data using computer equipment can be found in the implementation methods described above for attribute prediction, and will not be repeated here. It is understood that different encoding methods correspond to different attribute information in the point cloud data; for example, when encoded using attribute prediction, the attribute information of the corresponding point cloud data includes residual values; when encoded using attribute transformation, the attribute information of the corresponding point cloud data includes transformation coefficient values; and when encoded using a fusion of attribute prediction and attribute transformation, the attribute information of the corresponding point cloud data includes both residual values ​​and transformation coefficient values.

[0140] In one implementation, the points in the point cloud data are divided into K groups, where the points in the K groups are arranged in decoding order, and K is an integer greater than 1. The computer device optimizes the group prediction process based on the correlation between the groups, including optimizing the selection of prediction candidate points during the group prediction process. Specifically, assuming the computer device is decoding the (P+1)th group in the K groups, the method for optimizing the selection of prediction candidate points for the (P+1)th group includes at least one of the following embodiments:

[0141] In one embodiment, in addition to using the first N points of group P+1 as the prediction candidate points for group P+1, the computer device can also add P prediction points corresponding to the first P groups to the prediction candidate points for group P+1, with one prediction point per group. The attribute value of the j-th prediction point is the attribute reconstruction value of the points in group j, and the coordinates of the j-th prediction point are the average geometric coordinates of the points in group j. P is a positive integer less than K, j is a positive integer less than or equal to P, and N is determined by the maximum number of neighbors field (maxNumOfNeighbours). For example, assuming each group has 100 points, N=50, and P=10, then the prediction candidate points for group 11 include not only the first 50 points of group 11 but also the 10 prediction points corresponding to the first 10 groups. Among them, the attribute value of the prediction point corresponding to group 5 is the attribute reconstruction value of the points in group 5, and the set coordinates of the prediction point corresponding to group 5 are the average geometric coordinates of the points in group 5.

[0142] In another embodiment, in addition to using the first N points of group P+1 as prediction candidate points for group P+1, the computer device can also add a target prediction point to the prediction candidate points of group P+1. The value of the target prediction point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups. It is understood that when the weight of the attribute reconstruction values ​​of the points in the current P groups is 1, the value of the target prediction point is the average of the attribute reconstruction values ​​of the points in the previous P groups. The coordinates of the target prediction point are the average geometric coordinates of the points in the previous P groups. For example, assuming each group has 100 points, N=50, P=10; then the prediction candidate points of group 11, in addition to including the first 50 points of group 11, can also include the target prediction point. The attribute value of the target prediction point is the average of the attribute reconstruction values ​​of the previous 10 groups or the weighted average of the attribute reconstruction values ​​of the previous 10 groups, and the coordinates of the target prediction point are the average geometric coordinates of the previous 10 groups.

[0143] In another embodiment, in addition to using the first N points of group P+1 as predicted candidate points for group P+1, the computer device also replaces the points to be replaced in the predicted candidate points of group P+1 with target predicted points. The value of the target predicted point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups. It can be understood that when the weight of the attribute reconstruction value of the current P group points is 1, the value of the target predicted point is the average of the attribute reconstruction values ​​of the points in the previous P groups. The coordinates of the target predicted point are the average geometric coordinates of the points in the previous P groups. The points to be replaced may include, but are not limited to: the first point in the predicted candidate points, the last point in the predicted candidate points, and the farthest point in the predicted candidate points (the point farthest from the average geometric coordinates of group P+1, or the point farthest from the first point in group P+1).

[0144] It should be noted that when the number of points preceding the P+1th group is less than N (i.e., the sum of the number of points in the first P groups is less than the neighbor number threshold (maxNumOfNeighbours)), the computer device can use the P predicted points corresponding to the first P groups or the target predicted point as the candidate predicted points for the P+1th group; or it can use the P predicted points corresponding to the first P groups or the target predicted point as the neighbor points of the P+1th group for prediction.

[0145] In another implementation, the points in the point cloud data are divided into K groups, with the points in each group arranged in decoding order, where K is an integer greater than 1. The computer device optimizes the group prediction process based on the correlation between the groups, including optimizing the acquisition of residual values ​​during the group prediction process. Specifically, the computer device acquires the secondary prediction residual value and reference residual value corresponding to the i-th group, where i is a positive integer less than or equal to K. After acquiring the secondary prediction residual value and reference residual value corresponding to the i-th group, the computer device calculates the residual value for each point in the i-th group based on these values.

[0146] The secondary prediction residual value corresponding to the i-th group is obtained by the computer device parsing the bitstream data. The reference residual value corresponding to the i-th group can be obtained by the computer device parsing the bitstream data, or it can be determined by the computer device based on the residual value of at least one point in the first i groups. The specific implementation method for the computer device to determine the reference residual value corresponding to the i-th group based on the residual value of at least one point in the first i groups can be any of the following:

[0147] In one embodiment, the computer device can determine the residual value of any point in the first i groups as the reference residual value corresponding to the i-th group; for example, the computer device can use the residual value of the third point in the (i-2)-th group as the reference residual value corresponding to the i-th group; or, for another example, the computer device can use the residual value of the first point in the i-th group as the reference residual value corresponding to the i-th group.

[0148] In another embodiment, the computer device can calculate the reference residual value corresponding to the i-th group based on the residual values ​​of at least two points in the first i groups. Specifically, the computer device can determine the reference residual value corresponding to the i-th group as the weighted average of the residual values ​​of at least two points in the first i groups (when the weight of the residual value of each point in the first i groups is 1, it is the average of the residual values ​​of at least two points in the first i groups); for example, the computer device can use the average of the residual values ​​of the first point in each of the first i-1 groups (with the weight of the residual value of each point being 1) as the reference residual value corresponding to the i-th group; for another example, the computer device can use the weighted average of the residual values ​​of the first point in the (i-1)-th group and the residual values ​​of the last point in the (i-1)-th group as the reference residual value corresponding to the i-th group; for yet another example, the computer device can determine the reference residual value corresponding to the i-th group as the weighted average of the maximum and minimum residual values ​​among all the residual values ​​of the i-th group.

[0149] Optionally, the computer device may also calculate the reference residual value corresponding to the i-th group based on the sum of the residual values ​​of at least two points in the first i-th group, or based on the difference of the residual values ​​of at least two points in the first i-th group, or based on the logarithmic sum of the residual values ​​of at least two points in the first i-th group, etc., which will not be elaborated here.

[0150] In another implementation, the points in the point cloud data are divided into K groups, with the points in the K groups arranged in decoding order, where K is an integer greater than 1. The computer device optimizes the group prediction process based on the correlation between the groups, including optimizing the acquisition of transform coefficient values ​​during the group prediction process. Specifically, the computer device acquires the quadratic transform coefficient value and the reference transform coefficient value corresponding to the i-th group, where i is a positive integer less than or equal to K. After acquiring the quadratic transform coefficient value and the reference transform coefficient value corresponding to the i-th group, the computer device calculates the transform coefficient value of each point in the i-th group based on the quadratic transform coefficient value and the reference transform coefficient value.

[0151] The quadratic transform coefficient values ​​corresponding to the i-th group are obtained by the computer device parsing the bitstream data. The reference transform coefficient values ​​corresponding to the i-th group can be obtained by the computer device parsing the bitstream data, or they can be determined by the computer device based on the transform coefficient values ​​of at least one point in the first i groups. A specific implementation method for the computer device to determine the reference transform coefficient values ​​corresponding to the i-th group based on the transform coefficient values ​​of at least one point in the first i groups can be any of the following:

[0152] In one embodiment, the computer device can determine the transformation coefficient value of any point in the first i groups as the reference transformation coefficient value corresponding to the i-th group; for example, the computer device can take the transformation coefficient value of the third point in the (i-2)-th group as the reference transformation coefficient value corresponding to the i-th group; or, for another example, the computer device can take the transformation coefficient value of the first point in the i-th group as the reference transformation coefficient value corresponding to the i-th group.

[0153] In another embodiment, the computer device can calculate the reference transformation coefficient value of the i-th group based on the transformation coefficient values ​​of at least two points in the first i groups. Specifically, the computer device can determine the reference transformation coefficient value corresponding to the i-th group as the weighted average of the transformation coefficient values ​​of at least two points in the first i groups (when the weight of the transformation coefficient value of each point in the first i groups is 1, the reference transformation coefficient value corresponding to the i-th group is the average of the transformation coefficient values ​​of at least two points in the first i groups); for example, the computer device can use the weighted average of the transformation coefficient values ​​of the first point in each of the first i-1 groups as the reference transformation coefficient value corresponding to the i-th group; for another example, the computer device can use the average of the transformation coefficient values ​​of the first point in the (i-1)-th group and the transformation coefficient values ​​of the last point in the (i-1)-th group as the reference transformation coefficient value corresponding to the i-th group; for yet another example, the computer device can determine the reference transformation coefficient value corresponding to the i-th group as the weighted average of the maximum and minimum transformation coefficient values ​​among all the transformation coefficient values ​​of the i-th group.

[0154] Optionally, the computer device may also calculate the reference transformation coefficient value corresponding to the i-th group based on the sum of the transformation coefficient values ​​of at least two points in the first i-th group, or based on the difference of the transformation coefficient values ​​of at least two points in the first i-th group, or based on the logarithmic sum of the transformation coefficient values ​​of at least two points in the first i-th group, etc., which will not be elaborated here.

[0155] In another implementation, the points in the point cloud data are divided into K groups. Each point in the i-th group is associated with R attributes, and each attribute corresponds to M transformation coefficients. The M transformation coefficients for each attribute include one first transformation coefficient and M-1 second transformation coefficients. R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K. The computer device can decode the data using a decoding method corresponding to the encoding method, including but not limited to the following embodiments:

[0156] In one embodiment, the first transform coefficients (R coefficients) in the i-th group are directly encoded by the encoding device, and the second transform coefficients (R*(M-1) coefficients) in the i-th group are subjected to intra-group run-length encoding by the encoding device. Correspondingly, the computer device directly decodes the first transform coefficients in the i-th group and performs intra-group run-length decoding on the second transform coefficients in the i-th group; specifically, if the run-length corresponding to the second transform coefficient is not zero, the computer device performs run-length decoding on the second transform coefficient; if the run-length corresponding to the second transform coefficient is zero, the computer device directly decodes the second transform coefficient.

[0157] The attribute information of point cloud data can include at least one of the following: residual values ​​of each point, and transform coefficient values ​​of each point. In this specific solution, the attribute information of point cloud data can also be obtained by optimizing the prediction process based on the correlation between groups (such as secondary prediction residual values, reference residual values, secondary transform coefficient values ​​and reference transform coefficient values, run length, and the result after shift and remainder processing). Taking the attribute information of point cloud data including the first and second transform coefficients of each point as an example, the encoding and decoding methods of the first and second transform coefficients can be the same or different. For example, during encoding, the bitstream data corresponding to the first transform coefficient is obtained by entropy encoding of the first transform coefficient; the bitstream data corresponding to the second transform coefficient is obtained by (based on the correlation between groups) first performing inter-group run-length encoding on the second transform coefficient, and then performing intra-group run-length encoding on the target group (such as a group containing non-zero second transform coefficients); then during decoding, the computer device performs entropy decoding on the bitstream data corresponding to the first transform coefficient to obtain the first transform coefficient of each point; and performs entropy decoding on the second transform coefficient... The code stream data corresponding to the coefficients undergoes inter-group run-length decoding, and the results of inter-group run-length decoding are further subjected to intra-group run-length decoding to obtain the second transform coefficients at each point. For example, during encoding, the code stream data corresponding to the first transform coefficient is obtained by entropy encoding of the first transform coefficient; the code stream data corresponding to the second transform coefficient is obtained by intra-group run-length encoding of the second transform coefficient. Therefore, during decoding, the computer device performs entropy decoding on the code stream data corresponding to the first transform coefficient to obtain the shift and remainder processing result, and then calculates the first transform coefficient at each point based on the shift and remainder processing result; intra-group run-length decoding is then performed on the code stream data corresponding to the second transform coefficient to obtain the second transform coefficient at each point.

[0158] In one embodiment, direct encoding includes entropy encoding of the attribute information of the point cloud data to obtain S bits of data; correspondingly, a computer device performs entropy decoding on the S bits of data to obtain the attribute information of each point in the point cloud data. Entropy encoding includes at least one of the following encoding modes: a context-based encoding mode and a bypass encoding mode; similarly, entropy decoding includes at least one of the following decoding modes: a context-based decoding mode and a bypass decoding mode. The specific implementation of entropy decoding is described below:

[0159] In one implementation, the encoding device uses a context-based encoding mode to encode the attribute information of each point in the point cloud data, obtaining S bits of data. In one embodiment, the encoding device encodes the attribute information of each point in the point cloud data using the same context model; correspondingly, the computer device decodes the S bits of data using the same context model to obtain the attribute information of each point in the point cloud data. In another embodiment, the encoding device encodes the attribute information of each point in the point cloud data using different context models; correspondingly, the computer device decodes the S bits of data using different context models to obtain the attribute information of each point in the point cloud data. Specifically, S bits of data are divided into first encoded data and second encoded data; the encoding device can indicate the division method of the S bits of data by at least one preset value during encoding; accordingly, the computer device determines the first encoded data and second encoded data based on at least one preset value; and decodes the first encoded data using a first context model (such as decoding the prefix code of the first encoded data using the first context model corresponding to the prefix code in exponential Golbus decoding, and decoding the suffix code of the first encoded data using the first context model corresponding to the suffix code in exponential Golbus decoding), and decodes the second encoded data using a second context model to obtain the attribute information of each point in the point cloud data. The following example illustrates how to indicate the division of S bits of data using at least one preset value: Assume the number of preset values ​​is 1, represented by k (k can be a preset value or the order value corresponding to the exponential Golomb code), k is greater than or equal to 0, and k is less than S; In the exponential Golomb coding method, the S bits of data include S1 prefix codes and S2 suffix codes, S = S1 + S2; In other coding methods, the S bits of data can be divided by high bits (such as the first S / 2 bits) and low bits (such as the last S / 2 bits); The following example illustrates how the S bits of data include S1 prefix codes and S2 suffix codes: (1) In the S1 prefix codes, the kth prefix code The first target data is encoded using the first context model, the (k+1)th prefix code is encoded using the second context model, and the remaining prefix codes are encoded using the third context model. Of the S2 suffix codes, the kth suffix code is encoded using the fourth context model, the (k+1)th suffix code is encoded using the fifth context model, and the remaining suffix codes are encoded using the sixth context model. Correspondingly, the computer device can decode the S bits using the first context model – the sixth context model.(2) Of the S1 prefix codes, the kth prefix code is obtained by encoding the first target data using the first context model, the (k+1)th prefix code is obtained by encoding the second target data using the second context model, and the remaining prefix codes are obtained by encoding the third target data using the third context model. The S2 suffix codes are obtained by encoding the fourth target data - the sixth target data using the fourth context model. Accordingly, the computer device can use the first context model - the fourth context model to decode the S bits. (3) Of the S1 prefix codes, the kth suffix code is obtained by encoding the first target data - the third target data using the first context model. Of the S2 suffix codes, the kth suffix code is obtained by encoding the fourth target data using the fourth context model, the (k+1)th suffix code is obtained by encoding the fifth target data using the fifth context model, and the remaining suffix codes are obtained by encoding the sixth target data using the sixth context model. Accordingly, the computer device can use the first context model, the fourth context model - the sixth context model to decode the S bits. (4) Of the S1 prefix codes, the k-th prefix code is obtained by encoding the first target data using the first context model, the (k+1)-th prefix code is obtained by encoding the second target data using the second context model, and the remaining prefix codes are obtained by encoding the third target data using the third context model; Of the S2 suffix codes, the k-th suffix code is obtained by encoding the fourth target data using the fourth context model, the (k-1)-th suffix code is obtained by encoding the fifth target data using the fifth context model, and the remaining suffix codes are obtained by encoding the sixth target data using the sixth context model; Accordingly, the computer device can use the first context model - the sixth context model to decode the S bits. See Table 1 for details.

[0160] Table 1

[0161]

[0162]

[0163] Where k can be a preset value or the order corresponding to the exponent Golomb; m and n are both greater than 1, and m and n can be the same or different. The prefix code bits are s1, and the suffix code bits are s2. The S bits of data include S1 prefix codes and S2 suffix codes, where S = S1 + S2.

[0164] For example, suppose the number of preset values ​​is 2, represented as k1 and k2; among the S1 prefix codes, the k1th prefix code is obtained by encoding the first target data using the first context model, and the remaining prefix codes are obtained by encoding the second target data using the second context model; among the S2 suffix codes, the k2th suffix code is obtained by encoding the third target data using the third context model, and the remaining suffix codes are obtained by encoding the fourth target data using the fourth context model.

[0165] For example, in the S1 prefix codes, the prefix codes belonging to the first set (such as the 1st to m1th prefix codes, where m1 is a positive integer less than S1) are obtained by encoding the first target data using the first context model; the prefix codes belonging to the second set (such as the m1st to m2th prefix codes, where m2 is a positive integer greater than m1 and less than S1) are obtained by encoding the second target data using the second context model; and the remaining prefix codes are obtained by encoding the third target data using the third context model. In the S2 suffix codes, the prefix codes belonging to the third set... The suffix codes (such as the 1st to m3th suffix codes, where m3 is a positive integer less than S2) are obtained by encoding the fourth target data using the fourth context model. The suffix codes belonging to the fourth set (such as the m3rd to m4th prefix codes, where m4 is a positive integer greater than m3 and less than S2) are obtained by encoding the fifth target data using the fifth context model. The remaining suffix codes are obtained by encoding the sixth target data using the sixth context model. Correspondingly, the computer device can use the first to sixth context models to decode the S bits. See Table 2 for details.

[0166] Table 2

[0167] s1+1-s2 bits Second Context Model Other positions Third Context Model suffix code 1-s3 bits Fourth Context Model s3+1-s4 bits Fifth Context Model Other positions Sixth Context Model

[0168] Here, s1-s4 can be preset values, where s1 can be the order value corresponding to the exponential Golomb code, and s3 can be obtained from the length of the prefix code. It should be noted that any of the above context-based encoding / decoding modes can also be replaced with bypass encoding / decoding modes, which will not be elaborated upon here.

[0169] Then, the bits with a median less than k among the S bits are classified as the first coded data, and the bits with a median greater than or equal to k among the S bits are classified as the second coded data; or, the bits with values ​​less than k among the S bits are classified as the first coded data, and the bits with values ​​greater than or equal to k among the S bits are classified as the second coded data. Assuming the preset number of values ​​is 2, denoted as k1 and k2 (k1 and k2 can be preset values ​​or values ​​corresponding to the order of the exponent Columbus), and both k1 and k2 are greater than or equal to 0; then, the bits with a median less than k1 and a value less than k2 among the S bits are classified as the first coded data, and the remaining bits are classified as the second coded data.

[0170] In another implementation, the encoding device uses a bypass encoding mode to encode the attribute information of each point in the point cloud data, obtaining S bits of data. Correspondingly, the computer device uses a bypass decoding mode to decode the S bits of data, obtaining the attribute information of each point in the point cloud data.

[0171] In one implementation, the attribute information of the point cloud data includes a first attribute information set and a second attribute information set. During encoding, the encoding device can encode the first and second attribute information sets using any two of the following: an encoding mode based on a first context model, an encoding mode based on a second context model, and a bypass encoding mode, to obtain first encoded data and second encoded data. Correspondingly, the computer device can decode the first and second encoded data using corresponding decoding modes to obtain the first and second attribute information sets. For example, if the encoding device uses an encoding mode based on the first context model to encode the attribute information in the first attribute information set to obtain first encoded data, and uses a bypass encoding mode to encode the attribute information in the second attribute information set to obtain second encoded data, then the computer device uses a decoding mode based on the first context model to decode the first encoded data to obtain the first attribute information set, and uses a bypass decoding mode to decode the second encoded data to obtain the second attribute information set.

[0172] In one embodiment, the encoding device first performs binarization processing (such as exponential Golomb coding) on ​​the attribute information of each point to obtain a binarization result, and then performs entropy coding on the binarization result to obtain S bits of data. Correspondingly, the computer device performs entropy decoding on the S bits of data to obtain a binarization result, and then restores the attribute information of each point in the point cloud data based on the binarization result. For example, assuming that the computer device performs entropy decoding on the S bits of data to obtain the binarization result of the residual value of point A: 011, then the computer device restores the residual value of point A: 3 based on the binarization result of the residual value of point A.

[0173] In another embodiment, the encoding device first performs shift and remainder processing on the attribute information of each point to obtain the shift and remainder result, then performs binarization processing on the shift and remainder result to obtain the binarized result, and then performs entropy encoding on the binarized result to obtain S bits of data; correspondingly, the computer device performs entropy decoding on the S bits of data to obtain the binarized result, and then restores the shift processing result based on the binarized result; the shift and remainder processing result includes the quotient and remainder of the attribute information of each point. After obtaining the shift and remainder processing result, the computer device calculates the attribute information of each point in the point cloud data based on the quotient and remainder of the attribute information of each point. The entropy encoding includes at least one of the following encoding modes: context-based encoding mode and bypass encoding mode; similarly, the entropy decoding includes at least one of the following decoding modes: context-based decoding mode and bypass decoding mode. For example, suppose the quotient of the residual value of point A in the binarization result is 010 and the remainder is 001; then after restoring the binarization result, the shifted result of the residual value of point A is: quotient is 2 and remainder is 1. Based on the shifted result of the residual value of point A, the residual value of point A is calculated to be 5.

[0174] In another implementation, the encoding device performs run-length encoding on the attribute information of each point, obtaining S bits of data, where S is a positive integer. These S bits are used to indicate run-length encoding parameters, including the run-length and the encoding of non-preset values ​​(such as 0) in the attribute information of each point. The encoding methods for non-preset value attribute information can include, but are not limited to, residual-based encoding, transform-coefficient-based encoding, and entropy encoding. Correspondingly, the computer device performs run-length decoding on the S bits of data to obtain the attribute information of each point.

[0175] In one embodiment, the encoding device can also perform a shift and remainder operation on the run-length encoding result to obtain S bits of data, where S is a positive integer. Accordingly, when decoding, the computer device first calculates the run-length encoding parameters using the shift and remainder operation result, and then determines the attribute information of each point based on the run-length encoding parameters.

[0176] In another embodiment, the encoding device can also binarize the run-length encoding result to obtain S bits of data, where S is a positive integer; accordingly, when decoding, the computer device first restores the run-length encoding parameters through the binarization result, and then determines the attribute information of each point based on the run-length encoding parameters.

[0177] In another embodiment, the encoding device can also perform shift and remainder processing on the run-length encoding result to obtain the shift and remainder result, and then perform binarization processing on the shift and remainder result to obtain S bits of data, where S is a positive integer; accordingly, when the computer device decodes, it first restores the shift and remainder result through the binarization processing result, then calculates the run-length encoding parameters through the shift and remainder processing result, and then determines the attribute information of each point based on the run-length encoding parameters.

[0178] In another embodiment, the transform coefficients (R*M) in the i-th group are subjected to inter-group run-length encoding by the encoding device. Accordingly, if the run-length of the i-th group is not zero, the computer device performs inter-group run-length decoding on the transform coefficients in the i-th group; if the run-length of the i-th group is zero, the computer device decodes the transform coefficients in the i-th group one by one (e.g., direct decoding, intra-group run-length decoding, etc.).

[0179] In another embodiment, the first transform coefficients (R coefficients) in the i-th group are directly encoded by the encoding device, and the second transform coefficients (R*(M-1) coefficients) in the i-th group are group-run encoded by the encoding device. Correspondingly, the computer device directly decodes the first transform coefficients in the i-th group. If the group run length corresponding to the i-th group is not zero, the computer device performs inter-group run-run decoding on the second transform coefficients in the i-th group; if the group run length corresponding to the i-th group is zero, the computer device decodes the second transform coefficients in the i-th group one by one (e.g., direct decoding, intra-group run-run decoding, etc.).

[0180] In another embodiment, the first transform coefficients (R coefficients) in the i-th group are encoded using a shift-and-remainder method, and the second transform coefficients (R*(M-1) coefficients) in the i-th group are directly encoded using the same method. Correspondingly, the computer device performs shift-and-remainder decoding on the first transform coefficients in the i-th group and directly decodes the second transform coefficients in the i-th group. Specifically, the shift-and-remainder encoding of the target encoded information is implemented as follows: the target encoded information is shifted to obtain shifted data, which may include the shift quotient and the shift remainder. After obtaining the shifted data, the computer device encodes it. Taking the residual value of the current point in the k-th direction as the target encoded information as an example, Table 3 shows an example of shift-and-remainder encoding of the residual value of the current point in the k-th direction.

[0181] Table 3

[0182]

[0183]

[0184] In Table 3, `nodeIdx` indicates the current point in the point cloud. `k` indicates the k-th direction of the current point. `ptn_residual_abs_eq0_flag` (encoding information flag field) indicates whether the target encoding information is 0; when this field is set to 1, it indicates that the target encoding information is 0, and when this field is set to 0, it indicates that the target encoding information is not 0 (and is greater than 0). `ptn_residual_abs_remaining ptn` indicates the encoding shift remainder obtained by shifting the target encoding information. `ptn_residual_abs_half_eq0_flag` (shift quotient flag field) indicates whether the encoding shift quotient obtained by shifting the target encoding information is 0; when this field is set to 1, it indicates that the encoding shift quotient is 0, and when this field is set to 0, it indicates that the encoding shift quotient is not 0 (and is greater than 0). `ptn_residual_abs_numbits` (occupied bit count field) indicates the number of bits occupied by the current value to be encoded. ptn_numbits_remaining (placeholder remainder field) is used to indicate the remainder obtained by shifting the number of occupied bits. ptn_residual_abs_value_per (bit value field) is used to indicate the value of each bit in the number of occupied bits. The corresponding decoding process is as follows: (1) Analyze whether ptn_residual_abs_eq0_flag[k] is 0. If it is 0, continue parsing. If it is not 0, it can be determined that the residual value A[k] of the current point in the k-th direction is 0. (2) If ptn_residual_abs_eq0_flag[k] is 0, then parse ptn_residual_abs_remaining[k] to obtain the encoded shift remainder A2[k] of the residual value of the current point in the k-th direction. (3) Check if ptn_residual_abs_half_eq0_flag[k] is 0. If it is 0, continue parsing. If it is not 0, it can be determined that the encoding shift quotient A1[k] of the residual value in the k-th direction of the current point is 0. (4) If ptn_residual_abs_half_eq0_flag[k] is 0, then parse the encoding shift quotient A1[k] of the residual value in the k-th direction of the current point based on the bit bit encoding method. That is, by parsing ptn_residual_abs_numbits[k] and ptn_residual_abs_value_per[k][j], the encoding shift quotient A1[k] of the residual value in the k-th direction of the current point is obtained.(5) Reconstruct the residual value A[k] of the current point in the k-th direction using the encoded shift quotient A1[k] and the encoded shift remainder A2[k]: A[k] = A1[k] < <s1+A2[k]。

[0185] In another implementation, the points in the point cloud data are divided into K groups, with the i-th group containing M points, and each point associated with R attributes, each attribute corresponding to M transform coefficients; R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K. If M is less than a third quantity threshold, the encoding device performs shift-and-remainder encoding on the non-zero transform coefficients in the i-th group; correspondingly, the computer device performs shift-and-remainder decoding on the non-zero transform coefficients in the i-th group. If M is greater than or equal to the third quantity threshold, the encoding device directly encodes the non-zero transform coefficients in the i-th group; correspondingly, the computer device directly decodes the non-zero transform coefficients in the i-th group.

[0186] In another implementation, the points in the point cloud data are divided into K groups. The attribute of the j-th point in the i-th group consists of Q component attributes, each corresponding to a transform coefficient; Q is a positive integer, K is an integer greater than 1, i is a positive integer less than or equal to K, and j is a positive integer. The encoding device calculates the sum of the absolute values ​​of at least two component attributes among the Q component attributes to obtain a combined component attribute value. If the combined component attribute value is less than a numerical threshold, the encoding device directly encodes the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes; correspondingly, the computer device directly decodes the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes. If the combined component attribute value is greater than or equal to the numerical threshold, the encoding device performs shift-and-remainder encoding on the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes; correspondingly, the computer device performs shift-and-remainder decoding on the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes.

[0187] S304. Present the point cloud data according to its attribute information.

[0188] In one implementation, the computer device renders the reconstructed points based on the attribute information of the point cloud data to obtain and present the point cloud frame corresponding to the point cloud data.

[0189] In this embodiment, the bitstream data of point cloud data is acquired, and the bitstream data is parsed to obtain the grouping information of the point cloud data. Based on the correlation between each group, the grouping prediction process is optimized to obtain the attribute information of the point cloud data. The point cloud data is then presented according to this attribute information. It is evident that optimizing the grouping prediction process through the correlation between each group can make the prediction results closer to the actual results, reduce the amount of data to be decoded in the decoding stage, and thus improve the decoding efficiency of point cloud attribute information.

[0190] Please see Figure 4 , Figure 4 Another point cloud processing method provided in this application embodiment can be executed by a computer device, specifically, the computer device may be... Figure 2 The encoding device 201 in the point cloud processing system 20 shown. For example... Figure 4 As shown, the point cloud processing method may include the following steps S401-S404:

[0191] S401. Obtain the point cloud data to be encoded.

[0192] From the perspective of acquisition methods, the acquisition of point cloud data to be encoded can be divided into two types: acquisition through capturing real-world sound-visual scenes using capture devices and computer-generated methods. In one implementation, the capture device can refer to a hardware component located in the content production equipment, such as a microphone, camera, or sensor on a terminal. In another implementation, the capture device can also be a hardware device connected to the content production equipment, such as a camera connected to a server; used to provide media content acquisition services for the content production equipment by providing point cloud data. The capture device can include, but is not limited to, audio devices, camera devices, and sensing devices. Audio devices can include audio sensors, microphones, etc. Camera devices can include ordinary cameras, stereo cameras, light field cameras, etc. Sensing devices can include laser devices, radar devices, etc. The number of capture devices can be multiple, and these capture devices are deployed at specific locations in the real space to simultaneously capture audio and video content from different angles within that space, with the captured audio and video content remaining synchronized in both time and space.

[0193] S402. Divide the points in the point cloud data to obtain the grouping information of the point cloud data.

[0194] In one implementation, a computer device divides the points in a point cloud data according to a space-filling curve to obtain grouping information for the point cloud data. The coordinate order of the space-filling curves is determined based on the 3D bounding box to which the point cloud data belongs. For example, assuming the coordinates of the space-filling curves are arranged in ascending order of the side length of the 3D bounding box to which the point cloud data belongs, and the side length of the 3D bounding box to which the point cloud data belongs is 4m along the x-axis, 3m along the y-axis, and 5m along the z-axis, then the coordinate order of the space-filling curves is (y, x, z). As another example, assuming the coordinates of the space-filling curves are arranged in descending order of the side length of the 3D bounding box to which the point cloud data belongs, and the side length of the 3D bounding box to which the point cloud data belongs is 4m along the x-axis, 3m along the y-axis, and 5m along the z-axis, then the coordinate order of the space-filling curves is (z, x, y).

[0195] During the point cloud grouping process, the computer device can also control the number of points in each group using a first quantity threshold and a second quantity threshold, where the first quantity threshold is greater than the second quantity threshold. The specific quantity control method can be any of the following embodiments:

[0196] In one embodiment, it is assumed that the points in the point cloud data have been divided into K groups, where K is a positive integer. If the number of points in the i-th group is greater than a first quantity threshold, then the points in the i-th group whose coordinate values ​​in the target direction belong to the numerical range are grouped into a separate group, and the other points in the i-th group other than those whose coordinate values ​​in the target direction belong to the numerical range are grouped into another group. For example, assuming the target direction is the direction corresponding to the y-axis and the numerical range is [40, 41], the computer device will group the points in the i-th group whose values ​​in the y-axis direction are [40, 41] into a separate group, and the other points in the i-th group other than those whose values ​​in the y-axis direction are [40, 41] into another group. As another example, assuming the target direction is the direction corresponding to the z-axis and the numerical range is [5, 5], the computer device will group the points in the i-th group whose values ​​in the z-axis direction are 5 into a separate group, and the other points in the i-th group other than those whose values ​​in the z-axis direction are 5 into another group.

[0197] Here, the target direction is the direction corresponding to the shortest side of the 3D bounding box containing the points in the i-th group, where i is a positive integer less than or equal to K. For example, suppose the values ​​of the points in the i-th group are [20, 30] in the x-axis direction, [5, 40] in the y-axis direction, and [45, 50] in the z-axis direction. Then, the side length of the 3D bounding box containing the points in the i-th group is 10 in the x-axis direction, 30 in the y-axis direction, and 5 in the z-axis direction; in this case, the target direction is the z-axis direction.

[0198] Understandably, in the above scenario, the points in the point cloud data are divided into K+1 groups. If the number of points in the i-th group other than those whose coordinate values ​​in the target direction belong to the numerical range is still greater than the first threshold, the computer device can repeat the implementation method in the above embodiments until the number of points in each group after further division is less than the first threshold.

[0199] In another embodiment, assume that the points in the point cloud data have been divided into K groups, and these K groups of points are arranged in encoding order, where K is a positive integer. If the number of points in the i-th group is less than a second quantity threshold N, then the i-th group is merged with its adjacent groups to obtain a merged group. The adjacent groups of the i-th group can be either the (i-1)-th or the (i+1)-th group, where N is a positive integer. After obtaining the merged group, the computer device can perform an average division process on the merged group to obtain an updated i-th group and its updated adjacent groups; alternatively, it can divide the N points in the merged group into the i-th group, and divide the other points in the merged group (excluding the N points) into adjacent groups of the i-th group. For example, assuming i = 4, N = 50, the number of points in group 4 is 40, the number of points in group 3 is 80, and the number of points in group 5 is 90, the computer device can merge the points in group 4 with the points in group 3 to obtain a merged group, or it can merge the points in group 4 with the points in group 5 to obtain a merged group. Taking merging the points in group 4 with the points in group 3 as an example, the number of points in the merged group is 120. The computer device can use the first 50 points of the 120 points as the updated group 3 and the last 70 points as the updated group 4; it can also use the first 60 points of the 120 points as the updated group 3 and the last 60 points as the updated group 4 (i.e., average division); or it can use the first 70 points of the 120 points as the updated group 3 and the last 50 points as the updated group 4.

[0200] It should be noted that if the number of points in a merged group is less than 2N, the computer can continue to merge the merged group with its adjacent groups until, after S merges, the number of points in the merged group is greater than or equal to (S+1)*N, where S is a positive integer. For example, suppose i=4, N=50, the number of points in group 4 is 30, the number of points in group 5 is 60, and the number of points in group 6 is 90. The computer can then merge the points of group 4 with the points of group 5 for the first time, resulting in merged group 1 with a number of points = 90 < 2N. The computer can then merge the points of merged group 1 with the points of group 6 for the second time, resulting in merged group 2 with a number of points = 180 > 3N. Furthermore, after obtaining merged group 2, the computer device can perform an average division process on merged group 2 to obtain updated group 4-6 (each group in updated group 4-6 has 60 points); the computer device can also use the first 50 points in merged group 2 as the updated group 4, the 51st-100th points in merged group 2 as the updated group 5, and the last 80 points in merged group 2 as the updated group 6.

[0201] In another embodiment, assume that the points in the point cloud data have been divided into K groups, and these K groups of points are arranged in encoding order, where K is a positive integer. If the number of points in the i-th group is odd, then the number of points in the i-th group is incremented or decremented by one, where i is a positive integer less than or equal to K. Incrementing by one can be understood as taking a point from the adjacent groups (e.g., the (i-1)-th group, the (i+1)-th group) and adding it to the i-th group; similarly, decrementing by one can be understood as taking a point from the adjacent groups (e.g., the (i-1)-th group, the (i+1)-th group) of a point in the i-th group and adding it to the adjacent group of the i-th group. It is understood that if the i-th group is the last group in the K groups (i.e., i = K), and the number of points in the first K-1 groups is even, then the number of points in the i-th group remains unchanged.

[0202] S403. Optimize the group prediction process based on the correlation between each group to obtain the attribute information of the point cloud data.

[0203] The method for predicting the attributes of points in point cloud data using computer equipment can be found in the implementation methods described above for attribute prediction, and will not be repeated here. It is understood that different encoding methods correspond to different attribute information in the point cloud data; for example, when encoded using attribute prediction, the attribute information of the corresponding point cloud data includes residual values; when encoded using attribute transformation, the attribute information of the corresponding point cloud data includes transformation coefficient values; and when encoded using a fusion of attribute prediction and attribute transformation, the attribute information of the corresponding point cloud data includes both residual values ​​and transformation coefficient values.

[0204] In one implementation, the points in the point cloud data are divided into K groups, where the points in the K groups are arranged in encoding order, and K is an integer greater than 1. The computer device optimizes the group prediction process based on the correlation between the groups, including optimizing the selection of prediction candidate points during the group prediction process. Specifically, assuming the computer device is decoding the (P+1)th group in the K groups, the method for optimizing the selection of prediction candidate points for the (P+1)th group includes at least one of the following embodiments:

[0205] In one embodiment, in addition to using the first N points of group P+1 as the prediction candidate points for group P+1, the computer device can also add P prediction points corresponding to the first P groups to the prediction candidate points for group P+1, with one prediction point per group. The attribute value of the j-th prediction point is the attribute reconstruction value of the points in group j, and the coordinates of the j-th prediction point are the average geometric coordinates of the points in group j. P is a positive integer less than K, j is a positive integer less than or equal to P, and N is determined by the maximum number of neighbors field (maxNumOfNeighbours). For example, assuming each group has 100 points, N=50, and P=10, then the prediction candidate points for group 11 include not only the first 50 points of group 11 but also the 10 prediction points corresponding to the first 10 groups. Among them, the attribute value of the prediction point corresponding to group 5 is the attribute reconstruction value of the points in group 5, and the set coordinates of the prediction point corresponding to group 5 are the average geometric coordinates of the points in group 5.

[0206] In another embodiment, in addition to using the first N points of group P+1 as prediction candidate points for group P+1, the computer device can also add a target prediction point to the prediction candidate points of group P+1. The value of the target prediction point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups. It is understood that when the weight of the attribute reconstruction value of the current P group points is all 1, the value of the target prediction point is the average of the attribute reconstruction values ​​of the previous P group points. The coordinates of the target prediction point are the average geometric coordinates of the points in the previous P groups. For example, assuming each group has 100 points, N=50, P=10; then the prediction candidate points of group 11 may include the target prediction point in addition to the first 50 points of group 11. The attribute value of the target prediction point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups. It is understood that when the weight of the attribute reconstruction value of the current P group points is all 1, the value of the target prediction point is the average of the attribute reconstruction values ​​of the previous P group points. The coordinates of the target prediction point are the average geometric coordinates of the previous 10 groups.

[0207] In another embodiment, in addition to using the first N points of group P+1 as predicted candidate points for group P+1, the computer device also replaces the points to be replaced in the predicted candidate points of group P+1 with target predicted points. The value of the target predicted point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups. It can be understood that when the weight of the attribute reconstruction value of the current P group points is 1, the value of the target predicted point is the average of the attribute reconstruction values ​​of the points in the previous P groups. The coordinates of the target predicted point are the average geometric coordinates of the points in the previous P groups. The points to be replaced may include, but are not limited to: the first point in the predicted candidate points, the last point in the predicted candidate points, and the farthest point in the predicted candidate points (the point farthest from the average geometric coordinates of group P+1, or the point farthest from the first point in group P+1).

[0208] It should be noted that when the number of points preceding the P+1th group is less than N (i.e., the sum of the number of points in the first P groups is less than the neighbor number threshold (maxNumOfNeighbours)), the computer device can use the P predicted points corresponding to the first P groups or the target predicted point as the candidate predicted points for the P+1th group; or it can use the P predicted points corresponding to the first P groups or the target predicted point as the neighbor points of the P+1th group for prediction.

[0209] In another implementation, the points in the point cloud data are divided into K groups, with the points in each group arranged in encoding order, where K is an integer greater than 1. The computer device optimizes the group prediction process based on the correlation between the groups, including optimizing the acquisition of secondary prediction residual values. Specifically, the computer device acquires the residual values ​​of each point in the i-th group. The residual value of each point is calculated based on the primary prediction residual value and the attribute reconstruction value of that point, where i is a positive integer less than or equal to K. Based on the residual values ​​of at least one point in the first i groups, the computer device determines the reference residual value corresponding to the i-th group, and calculates the secondary prediction residual value based on the residual values ​​of each point in the i-th group and the reference residual value corresponding to the i-th group.

[0210] The specific implementation method for the computer device to determine the reference residual value corresponding to the i-th group based on the residual value of at least one point in the first i-th group can be any of the following:

[0211] In one embodiment, the computer device can determine the residual value of any point in the first i groups as the reference residual value corresponding to the i-th group; for example, the computer device can use the residual value of the third point in the (i-2)-th group as the reference residual value corresponding to the i-th group. As another example, the computer device can use the residual value of the first point in the i-th group as the reference residual value corresponding to the i-th group.

[0212] In another embodiment, the computer device can calculate the reference residual value corresponding to the i-th group based on the residual values ​​of at least two points in the first i groups. Specifically, the computer device can determine the reference residual value corresponding to the i-th group as the weighted average of the residual values ​​of at least two points in the first i groups (where the weight of the residual values ​​of all points in the current i-th group is 1, i.e., the average of the residual values ​​of at least two points in the first i-th group); for example, the computer device can use the weighted average of the residual values ​​of the first point in each of the first i-1 groups as the reference residual value corresponding to the i-th group; for another example, the computer device can use the weighted average of the residual values ​​of the first point in the (i-1)-th group and the residual values ​​of the last point in the (i-1)-th group as the reference residual value corresponding to the i-th group; for yet another example, the computer device can determine the reference residual value corresponding to the i-th group as the weighted average of the maximum and minimum residual values ​​among all the residual values ​​of the i-th group.

[0213] Optionally, the computer device may also calculate the reference residual value corresponding to the i-th group based on the sum of the residual values ​​of at least two points in the first i-th group, or based on the difference of the residual values ​​of at least two points in the first i-th group, or based on the logarithmic sum of the residual values ​​of at least two points in the first i-th group, etc., which will not be elaborated here.

[0214] In another implementation, the points in the point cloud data are divided into K groups, with the points in the K groups arranged in encoding order, where K is an integer greater than 1. The computer device optimizes the group prediction process based on the correlation between the groups, including optimizing the acquisition of the secondary transformation coefficient values ​​during the group prediction process. Specifically, the computer device acquires the transformation coefficient values ​​of each point in the i-th group, where i is a positive integer less than or equal to K. Based on the transformation coefficient values ​​of at least one point in the first i groups, the computer device determines a reference transformation coefficient value for the i-th group, and calculates the secondary transformation coefficient value based on the transformation coefficient values ​​of each point in the i-th group and the reference transformation coefficient value for the i-th group.

[0215] The specific implementation method for the computer device to determine the reference transformation coefficient value corresponding to the i-th group based on the transformation coefficient value of at least one point in the first i-th group can be any of the following:

[0216] In one embodiment, the computer device can determine the transformation coefficient value of any point in the first i-1 groups as the reference transformation coefficient value corresponding to the i-th group; for example, the computer device can take the transformation coefficient value of the third point in the i-2 groups as the reference transformation coefficient value corresponding to the i-th group; or, for another example, the computer device can take the transformation coefficient value of the first point in the i-th group as the reference transformation coefficient value corresponding to the i-th group.

[0217] In another embodiment, the computer device can calculate the reference transformation coefficient value of the i-th group based on the transformation coefficient values ​​of at least two points in the first i groups. Specifically, the computer device can determine the reference transformation coefficient value corresponding to the i-th group as the weighted average of the transformation coefficient values ​​of at least two points in the first i groups (when the weight of the transformation coefficient value of each point in the first i groups is 1, the reference transformation coefficient value corresponding to the i-th group is the average of the transformation coefficient values ​​of at least two points in the first i groups); for example, the computer device can use the weighted average of the transformation coefficient values ​​of the first point in each of the first i-1 groups as the reference transformation coefficient value corresponding to the i-th group; for another example, the computer device can use the average of the transformation coefficient values ​​of the first point in the (i-1)-th group and the transformation coefficient values ​​of the last point in the (i-1)-th group as the reference transformation coefficient value corresponding to the i-th group; for yet another example, the computer device can determine the reference transformation coefficient value corresponding to the i-th group as the weighted average of the maximum and minimum transformation coefficient values ​​among all the transformation coefficient values ​​of the i-th group.

[0218] Optionally, the computer device may also calculate the reference transformation coefficient value corresponding to the i-th group based on the sum of the transformation coefficient values ​​of at least two points in the first i-th group, or based on the difference of the transformation coefficient values ​​of at least two points in the first i-th group, or based on the logarithmic sum of the transformation coefficient values ​​of at least two points in the first i-th group, etc., which will not be elaborated here.

[0219] S404. Encode the attribute information of the point cloud data to obtain the bitstream data of the point cloud data.

[0220] The complete implementation method for encoding the attribute information of point cloud data by computer equipment to obtain the bitstream data of point cloud data can be referred to the above-described example of point cloud encoding, and will not be repeated here.

[0221] In one implementation, the computer device encodes the attribute information of the point cloud data based on the correlation between various groups and the distribution characteristics of the point cloud data's attribute information, thereby obtaining the bitstream data of the point cloud data. It is understood that different encoding methods correspond to different attribute information in the point cloud data; for example, encoding via attribute prediction results in residual values; encoding via attribute transformation results in transformation coefficient values; and encoding via a fusion of attribute prediction and attribute transformation results in both residual values ​​and transformation coefficient values. The different scenarios are described in detail below:

[0222] In one embodiment, the attribute information of the point cloud data is transform coefficients. The points in the point cloud data are divided into K groups. Each point in the i-th group is associated with R attributes, and each attribute corresponds to M transform coefficients. The M transform coefficients corresponding to each attribute include one first transform coefficient (e.g., DC transform coefficient) and M-1 second transform coefficients (e.g., AC transform coefficient). R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K. The process by which the computer device encodes the attribute information of the point cloud data based on the association relationships between the groups and the distribution characteristics of the attribute information includes any of the following implementation methods:

[0223] In one implementation, the computer device directly encodes the first transform coefficients (R coefficients) in the i-th group and performs run-length encoding on the second transform coefficients (R*(M-1) coefficients) in the i-th group.

[0224] The attribute information of point cloud data can include at least one of the following: residual values ​​of each point, and transform coefficient values ​​of each point. In this specific solution, the attribute information of point cloud data can also be obtained by optimizing the prediction process based on the correlation between groups (such as secondary prediction residual values, reference residual values, secondary transform coefficient values ​​and reference transform coefficient values, run length, and the result after shift and remainder processing). Taking the attribute information of point cloud data including the first and second transform coefficients of each point as an example, the encoding and decoding methods of the first and second transform coefficients can be the same or different. For example, during encoding, the bitstream data corresponding to the first transform coefficient is obtained by entropy encoding of the first transform coefficient; the bitstream data corresponding to the second transform coefficient is obtained by (based on the correlation between groups) first performing inter-group run-length encoding on the second transform coefficient, and then performing intra-group run-length encoding on the target group (such as a group containing non-zero second transform coefficients); then during decoding, the computer device performs entropy decoding on the bitstream data corresponding to the first transform coefficient to obtain the first transform coefficient of each point; and performs entropy decoding on the second transform coefficient... The code stream data corresponding to the coefficients undergoes inter-group run-length decoding, and the results of inter-group run-length decoding are further subjected to intra-group run-length decoding to obtain the second transform coefficients at each point. For example, during encoding, the code stream data corresponding to the first transform coefficient is obtained by entropy encoding of the first transform coefficient; the code stream data corresponding to the second transform coefficient is obtained by intra-group run-length encoding of the second transform coefficient. Therefore, during decoding, the computer device performs entropy decoding on the code stream data corresponding to the first transform coefficient to obtain the shift and remainder processing result, and then calculates the first transform coefficient at each point based on the shift and remainder processing result; intra-group run-length decoding is then performed on the code stream data corresponding to the second transform coefficient to obtain the second transform coefficient at each point.

[0225] In one embodiment, direct encoding includes entropy encoding of the attribute information of the point cloud data. A computer device performs entropy encoding on the attribute information of each point in the point cloud data to obtain S bits of data. Entropy encoding includes at least one of the following encoding modes: context-based encoding mode and bypass encoding mode. The specific implementation of entropy encoding is described below:

[0226] In one implementation, the computer device encodes the attribute information of each point in the point cloud data using a context-based encoding mode, obtaining S bits of data. In one embodiment, the computer device encodes the attribute information of each point in the point cloud data using the same context model, obtaining S bits of data. In another embodiment, the computer device encodes the attribute information of each point in the point cloud data using different context models, obtaining S bits of data. Further, the S bits of data are divided into a first bit set and a second bit set; the computer device can indicate the division method of the S bits of data using at least one preset value during encoding, and different bit sets correspond to different decoding methods; for example, the first bit set corresponds to an encoding mode based on a first context model, and the second bit set corresponds to an encoding mode based on a second context model.

[0227] In another implementation, the computer device uses a bypass coding mode to encode the attribute information of each point in the point cloud data, obtaining S bits of data. In one embodiment, the attribute information of the point cloud data includes a first attribute information set and a second attribute information set. When encoding, the computer device can use any two of the following: an encoding mode based on a first context model, an encoding mode based on a second context model, and a bypass coding mode to encode the first attribute information set and the second attribute information set to obtain first encoded data and second encoded data. For example, the computer device uses an encoding mode based on the first context model to encode the attribute information in the first attribute information set to obtain first encoded data, and uses a bypass coding mode to encode the attribute information in the second attribute information set to obtain second encoded data.

[0228] In another embodiment, the computer device first performs binarization processing (such as exponential Golomb coding) on ​​the attribute information of each point to obtain the binarization result, and then performs entropy coding on the binarization result to obtain S bits of data.

[0229] In one implementation, the computer device first performs a shift and remainder operation on the attribute information of each point to obtain the shift and remainder operation result, then performs binarization on the shift and remainder operation result to obtain the binarization result, and finally performs entropy encoding on the binarization result to obtain S bits of data.

[0230] In another implementation, the computer device performs run-length encoding on the attribute information of each point to obtain S bits of data, where S is a positive integer. The S bits of data are used to indicate run-length encoding parameters, which include: run length, and encoding information of non-preset values ​​(such as 0) in the attribute information of each point. The encoding method for non-preset value attribute information may include, but is not limited to: encoding method based on residual value, encoding method based on transform coefficient, and entropy encoding.

[0231] In another implementation, the computer device can also perform a shift and remainder operation on the run-length encoding result to obtain S bits of data, where S is a positive integer.

[0232] In another embodiment, the computer device may also binarize the run-length encoding result to obtain S bits of data, where S is a positive integer.

[0233] In another embodiment, the computer device may further perform a shift and remainder operation on the run-length encoding result to obtain a shift and remainder result, and then perform binarization on the shift and remainder result to obtain S bits of data, where S is a positive integer.

[0234] In another implementation, if the values ​​of the transformation coefficients (R*M) in the i-th group are all preset values ​​(such as 0), the computer device increments the group travel length corresponding to the K-th group point by one. For example, assuming the current group travel length is 3 and the values ​​of the transformation coefficients in the i-th group are all preset values, the computer device increments the current group travel length by one, and the group travel length is 4. The computer device continues to encode the i+1-th group in the above manner. If at least one transformation coefficient in the i-th group has a value that is not a preset value, then the group travel length corresponding to the K-th group point is set to zero, and the transformation coefficients in the i-th group are encoded one by one (e.g., the transformation coefficients in the i-th group are directly encoded, or the transformation coefficients in the i-th group are travel encoded, which can be determined according to actual needs, and this application does not impose any restrictions on this); for example, assuming that the current group travel length is 3, and at least one transformation coefficient in the i-th group has a value that is not a preset value, then the computer device sets the current group travel length to zero, at this time the group travel length is 0, and the transformation coefficients in the i-th group are either travel encoded or directly encoded.

[0235] In another implementation, the computer device encodes the first transformation coefficients (R) in the i-th group (e.g., direct encoding, run-length encoding). If the values ​​of the second transformation coefficients (R*(M-1)) in the i-th group are all preset values ​​(e.g., 0), then the run-length of the group corresponding to the K-th point is incremented by one. For example, assuming the current run-length of the group is 5 and the values ​​of the second transformation coefficients in the i-th group are all preset values, the computer device increments the current run-length of the group by one, and the run-length of the group is 6. The computer device continues to encode the (i+1)-th group in the above manner. If at least one of the second transformation coefficients in the i-th group has a value that is not a preset value, the computer device sets the group travel length corresponding to the K-th group point to zero and encodes the second transformation coefficients in the i-th group one by one (such as directly encoding the second transformation coefficients in the i-th group, or performing travel encoding on the second transformation coefficients in the i-th group, which can be determined according to actual needs, and this application does not impose any restrictions on this); for example, assuming that the current group travel length is 5, and at least one of the second transformation coefficients in the i-th group has a value that is not a preset value, the computer device sets the current group travel length to zero, at this time the group travel length is 0, and directly encodes the second transformation coefficients in the i-th group.

[0236] In another implementation, the computer device performs shift-and-remainder encoding on the first transform coefficients (R coefficients) in the i-th group and directly encodes the second transform coefficients (R*(M-1) coefficients) in the i-th group.

[0237] In another embodiment, the points in the point cloud data are divided into K groups, with the i-th group containing M points. Each point is associated with R attributes, and each attribute corresponds to M transformation coefficients; R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K. In one implementation, R is the number of attribute data associated with each point; for example, R = 1 when each point is associated with a single attribute data point, and R > 1 when each point is associated with multiple attribute data points. The specific process by which the computer device encodes the attribute information of the point cloud data based on the association relationships between the groups and the distribution characteristics of the attribute information of the point cloud data is as follows:

[0238] If M is less than the third threshold, the computer device performs shift-and-remainder encoding on the non-zero transform coefficients in the i-th group. If M is greater than or equal to the third threshold, the computer device directly encodes the non-zero transform coefficients in the i-th group. For example, assuming M = 80 and the third threshold is 50, the computer device directly encodes the non-zero transform coefficients in the i-th group; assuming M = 30 and the third threshold is 50, the computer device performs shift-and-remainder encoding on the non-zero transform coefficients in the i-th group.

[0239] In another embodiment, the points in the point cloud data are divided into K groups. The attribute of the j-th point in the i-th group consists of Q component attributes, each component attribute corresponding to a transformation coefficient; Q is a positive integer, K is an integer greater than 1, i is a positive integer less than or equal to K, and j is a positive integer. The specific process by which the computer device encodes the attribute information of the point cloud data based on the correlation between the various groups and the distribution characteristics of the attribute information of the point cloud data is as follows:

[0240] The computer device calculates the sum of the absolute values ​​of at least two of the Q component attributes to obtain a combined component attribute value. For example, assuming Q = 3, the combined component attribute value can be the sum of the absolute values ​​of the first and second component attributes, the sum of the absolute values ​​of the second and third component attributes, the sum of the absolute values ​​of the first and third component attributes, or the sum of the absolute values ​​of the first, second, and third component attributes. After obtaining the combined component attribute value, the computer device compares it with a numerical threshold. If the combined component attribute value is less than the numerical threshold, the computer device directly encodes the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes. For example, assuming the combined component attribute value is 3, the numerical threshold is 10, Q = 3, and the transform coefficients corresponding to the first and second component attributes are both zero, while the transform coefficient corresponding to the third component attribute is non-zero, the computer device directly encodes the transform coefficient corresponding to the third component attribute. If the combined value of the component attributes is greater than or equal to the numerical threshold, the computer device performs shift-and-remainder encoding on the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes. For example, assuming the combined value of the component attributes is 58, the numerical threshold is 10, Q = 3, and the transform coefficients corresponding to the first and third component attributes are both zero, while the transform coefficient corresponding to the second component attribute is non-zero, the computer device performs shift-and-remainder encoding on the transform coefficient corresponding to the second component attribute.

[0241] In this embodiment, point cloud data to be encoded is acquired, the points in the point cloud data are divided to obtain grouping information, the grouping prediction process is optimized based on the correlation between each group to obtain attribute information of the point cloud data, and the attribute information of the point cloud data is encoded according to the correlation between each group to obtain the bitstream data of the point cloud data. It can be seen that optimizing the grouping prediction process through the correlation between each group can make the prediction results closer to the actual results, reduce the amount of data to be encoded in the encoding stage, and thus improve the encoding efficiency of point cloud attribute information.

[0242] Please see Figure 5 , Figure 5This application provides another point cloud processing method, which can be executed by a computer device. Specifically, the computer device may be... Figure 2 The decoding device 202 in the point cloud processing system 20 shown. For example... Figure 5 As shown, the point cloud processing method may include the following steps S501 and S502:

[0243] S501, Obtain the bitstream data of point cloud data.

[0244] The computer device can acquire the bitstream data of point cloud data in two ways: either by acquiring it in real time from the encoding device, or by downloading the complete bitstream data from the encoding device or server.

[0245] S502. Decode the bitstream data and present point cloud data based on the decoding result.

[0246] In one implementation, the bitstream data includes S bits of data, which are obtained by encoding the attribute information of each point in the point cloud data, where S is a positive integer. The attribute information of each point includes at least one of the following: residual value and transform coefficient value. The process of decoding the bitstream data by the computer device includes decoding the bitstream data to obtain the attribute information of each point in the point cloud data. A specific implementation is described in any of the following embodiments:

[0247] In one embodiment, the encoding device performs entropy encoding on the attribute information of each point in the point cloud data to obtain S bits of data; correspondingly, the computer device performs entropy decoding on the S bits of data to obtain the attribute information of each point in the point cloud data. The entropy encoding includes at least one of the following encoding modes: a context-based encoding mode and a bypass encoding mode; similarly, the entropy decoding includes at least one of the following decoding modes: a context-based decoding mode and a bypass decoding mode. The specific implementation of entropy decoding is described below:

[0248] In one implementation, the encoding device uses a context-based encoding mode to encode the attribute information of each point in the point cloud data, obtaining S bits of data. In one embodiment, the encoding device uses the same context model to encode the attribute information of each point in the point cloud data; correspondingly, the computer device uses the same context model to decode the S bits of data to obtain the attribute information of each point in the point cloud data.

[0249] In another embodiment, the encoding device encodes the attribute information of each point in the point cloud data using different context models; correspondingly, the computer device decodes the S bits of data using different context models to obtain the attribute information of each point in the point cloud data. Specifically, the S bits of data are divided into first encoded data and second encoded data; the encoding device can indicate the division method of the S bits of data by at least one preset value during encoding; correspondingly, the computer device determines the first encoded data and the second encoded data based on at least one preset value; and decodes the first encoded data using the first context model (e.g., decoding the prefix code of the first encoded data using the first context model corresponding to the prefix code in exponential Golbus decoding, and decoding the suffix code of the first encoded data using the first context model corresponding to the suffix code in exponential Golbus decoding), and decodes the second encoded data using the second context model to obtain the attribute information of each point in the point cloud data. The following is an example of how to indicate the division of S bits of data by using at least one preset value: Assume that the number of preset values ​​is 1, represented by k (k can be a preset value or the order value corresponding to the exponential Golomb code), k is greater than or equal to 0, and k is less than S; In the exponential Golomb code, the S bits of data include S1 prefix codes and S2 suffix codes, S = S1 + S2; In other encoding methods, the S bits of data can be distinguished by high bits (such as the first S / 2 bits) and low bits (such as the last S / 2 bits); The following is an explanation of the S bits of data including S1 prefix codes and S2 suffix codes: (1) In the S1 prefix codes, the kth prefix code The first target data is encoded using the first context model, the (k+1)th prefix code is encoded using the second context model, and the remaining prefix codes are encoded using the third context model. Of the S2 suffix codes, the kth suffix code is encoded using the fourth context model, the (k+1)th suffix code is encoded using the fifth context model, and the remaining suffix codes are encoded using the sixth context model. Correspondingly, the computer device can decode the S bits using the first context model – the sixth context model. (2) Among the S1 prefix codes, the kth prefix code is obtained by encoding the first target data using the first context model, the (k+1)th prefix code is obtained by encoding the second target data using the second context model, the remaining prefix codes are obtained by encoding the third target data using the third context model, and the S2 suffix codes are obtained by encoding the fourth target data to the sixth target data using the fourth context model; accordingly, the computer device can use the first context model to the fourth context model to decode the S bits.(3) The S1 prefix codes are obtained by encoding the first target data and the third target data using the first context model. Among the S2 suffix codes, the kth suffix code is obtained by encoding the fourth target data using the fourth context model, the (k+1)th suffix code is obtained by encoding the fifth target data using the fifth context model, and the remaining suffix codes are obtained by encoding the sixth target data using the sixth context model. Accordingly, the computer device can use the first context model, the fourth context model and the sixth context model to decode the S bits. (4) Of the S1 prefix codes, the k-th prefix code is obtained by encoding the first target data using the first context model, the (k+1)-th prefix code is obtained by encoding the second target data using the second context model, and the remaining prefix codes are obtained by encoding the third target data using the third context model; Of the S2 suffix codes, the k-th suffix code is obtained by encoding the fourth target data using the fourth context model, the (k-1)-th suffix code is obtained by encoding the fifth target data using the fifth context model, and the remaining suffix codes are obtained by encoding the sixth target data using the sixth context model; Accordingly, the computer device can use the first context model - the sixth context model to decode the S bits. See Table 4 for details.

[0250] Table 4

[0251]

[0252]

[0253] Where k can be a preset value or the order corresponding to the exponent Golomb; m and n are both greater than 1, and m and n can be the same or different. The prefix code bits are s1, and the suffix code bits are s2. The S bits of data include S1 prefix codes and S2 suffix codes, where S = S1 + S2.

[0254] For example, suppose the number of preset values ​​is 2, represented as k1 and k2; among the S1 prefix codes, the k1th prefix code is obtained by encoding the first target data using the first context model, and the remaining prefix codes are obtained by encoding the second target data using the second context model; among the S2 suffix codes, the k2th suffix code is obtained by encoding the third target data using the third context model, and the remaining suffix codes are obtained by encoding the fourth target data using the fourth context model.

[0255] For another example, among the S1 prefix codes, the prefix codes belonging to the first set (such as the 1st - m1th prefix codes, where m1 is a positive integer less than S1) are obtained by encoding the first target data using the first context model, the prefix codes belonging to the second set (such as the (m1 + 1)th - m2th prefix codes, where m2 is a positive integer greater than m1 and less than S1) are obtained by encoding the second target data using the second context model, and the remaining prefix codes are obtained by encoding the third target data using the third context model; among the S2 suffix codes, the suffix codes belonging to the third set (such as the 1st - m3th suffix codes, where m3 is a positive integer less than S2) are obtained by encoding the fourth target data using the fourth context model, the suffix codes belonging to the fourth set (such as the (m3 + 1)th - m4th prefix codes, where m4 is a positive integer greater than m3 and less than S2) are obtained by encoding the fifth target data using the fifth context model, and the remaining suffix codes are obtained by encoding the sixth target data using the sixth context model; correspondingly, the computer device can use the first context model - the sixth context model to decode the S bit positions. See Table 5:

[0256] Table 5

[0257] m1+1-m2 position Second Context Model Other positions Third Context Model suffix code 1-m3 positions Fourth Context Model m3+1-m4 position Fifth Context Model Other positions Sixth Context Model

[0258] Among them, m1 - m4 can be preset values, m1 can also be the corresponding order value of exponential Golomb, and m3 can also be obtained from the length of the prefix code. It should be noted that any of the above context model - based encoding and decoding modes can also be replaced with a bypass encoding and decoding mode, which will not be elaborated here.

[0259] In another embodiment, the encoding device can also indicate different combinations of context models (which can include at least one context model) used during encoding through the exponential Golomb order; for example, let the target order be k1 and the starting order be k0, and the attribute information in the point cloud data includes: the first attribute information set, the second attribute information set, and the third attribute information set. When k0 < k1, the encoding device can use the first context model combination to encode the first attribute information set; when k0 = k1, the encoding device can use the second context model combination to encode the second attribute information set; when k0 > k1, the encoding device can use the third context model combination to encode the third attribute information set. Correspondingly, the computer device can determine the context model combination used for decoding different bitstream data according to the exponential Golomb order; for example, when decoding the bitstream data corresponding to the third attribute information set, based on k0 > k1, the computer device determines to use the third context model combination to decode the bitstream data corresponding to the third attribute information set.

[0260] In another implementation, the encoding device uses a bypass encoding mode to encode the attribute information of each point in the point cloud data, obtaining S bits of data. Correspondingly, the computer device uses a bypass decoding mode to decode the S bits of data, obtaining the attribute information of each point in the point cloud data.

[0261] In one implementation, the attribute information of the point cloud data includes a first attribute information set and a second attribute information set. During encoding, the encoding device can encode the first and second attribute information sets using any two of the following: an encoding mode based on a first context model, an encoding mode based on a second context model, and a bypass encoding mode, to obtain first encoded data and second encoded data. Correspondingly, the computer device can decode the first and second encoded data using corresponding decoding modes to obtain the first and second attribute information sets. For example, if the encoding device uses an encoding mode based on the first context model to encode the attribute information in the first attribute information set to obtain first encoded data, and uses a bypass encoding mode to encode the attribute information in the second attribute information set to obtain second encoded data, then the computer device uses a decoding mode based on the first context model to decode the first encoded data to obtain the first attribute information set, and uses a bypass decoding mode to decode the second encoded data to obtain the second attribute information set.

[0262] In one embodiment, the encoding device first performs binarization processing (such as exponential Golomb coding) on ​​the attribute information of each point to obtain a binarization result, and then performs entropy coding on the binarization result to obtain S bits of data. Correspondingly, the computer device performs entropy decoding on the S bits of data to obtain a binarization result, and then restores the attribute information of each point in the point cloud data based on the binarization result. For example, assuming that the computer device performs entropy decoding on the S bits of data to obtain the binarization result of the residual value of point A: 011, then the computer device restores the residual value of point A: 3 based on the binarization result of the residual value of point A.

[0263] In another embodiment, the encoding device first performs shift and remainder processing on the attribute information of each point to obtain the shift and remainder result, then performs binarization processing on the shift and remainder result to obtain the binarized result, and then performs entropy encoding on the binarized result to obtain S bits of data; correspondingly, the computer device performs entropy decoding on the S bits of data to obtain the binarized result, and then restores the shift processing result based on the binarized result; the shift and remainder processing result includes the quotient and remainder of the attribute information of each point. After obtaining the shift and remainder processing result, the computer device calculates the attribute information of each point in the point cloud data based on the quotient and remainder of the attribute information of each point. The entropy encoding includes at least one of the following encoding modes: context-based encoding mode and bypass encoding mode; similarly, the entropy decoding includes at least one of the following decoding modes: context-based decoding mode and bypass decoding mode. For example, suppose the quotient of the residual value of point A in the binarization result is 010 and the remainder is 001; then after restoring the binarization result, the shifted result of the residual value of point A is: quotient is 2 and remainder is 1. Based on the shifted result of the residual value of point A, the residual value of point A is calculated to be 5.

[0264] In another implementation, the encoding device performs run-length encoding on the attribute information of each point, obtaining S bits of data, where S is a positive integer. These S bits are used to indicate run-length encoding parameters, including the run-length and the encoding of non-preset values ​​(such as 0) in the attribute information of each point. The encoding methods for non-preset value attribute information can include, but are not limited to, residual-based encoding, transform-coefficient-based encoding, and entropy encoding. Correspondingly, the computer device performs run-length decoding on the S bits of data to obtain the attribute information of each point.

[0265] In one embodiment, the encoding device can also perform a shift and remainder operation on the run-length encoding result to obtain S bits of data, where S is a positive integer. Accordingly, when decoding, the computer device first calculates the run-length encoding parameters using the shift and remainder operation result, and then determines the attribute information of each point based on the run-length encoding parameters.

[0266] In another embodiment, the encoding device can also binarize the run-length encoding result to obtain S bits of data, where S is a positive integer; accordingly, when decoding, the computer device first restores the run-length encoding parameters through the binarization result, and then determines the attribute information of each point based on the run-length encoding parameters.

[0267] In another embodiment, the encoding device can also perform shift and remainder processing on the run-length encoding result to obtain the shift and remainder result, and then perform binarization processing on the shift and remainder result to obtain S bits of data, where S is a positive integer; accordingly, when the computer device decodes, it first restores the shift and remainder result through the binarization processing result, then calculates the run-length encoding parameters through the shift and remainder processing result, and then determines the attribute information of each point based on the run-length encoding parameters.

[0268] In this embodiment, bitstream data of point cloud data is acquired, the bitstream data is decoded, and the point cloud data is presented based on the decoding result. During the decoding process, the bitstream data can be decoded using one or more decoding methods corresponding to the encoding method; the decoding method can also be determined by the indication information corresponding to the preprocessing method; or the bitstream data can be divided by at least one preset value. Through the above methods, the decoding efficiency of point cloud attribute information can be improved.

[0269] Please see Figure 6 , Figure 6 This application provides another point cloud processing method, which can be executed by a computer device. Specifically, the computer device may be... Figure 2 The encoding device 201 in the point cloud processing system 20 shown. For example... Figure 6 As shown, the point cloud processing method may include the following steps S601 and S602:

[0270] S601. Obtain the point cloud data to be encoded.

[0271] For a detailed implementation of step S601, please refer to Figure 4 The implementation method of step S401 will not be described in detail here.

[0272] S602. Encode the point cloud data to be encoded to obtain the bit stream data of the point cloud data.

[0273] In one implementation, the process of encoding point cloud data to be encoded by a computer device includes encoding the attribute information of each point in the point cloud data to obtain S bits of data, where S is a positive integer. The attribute information of each point includes at least one of the following: residual value and transform coefficient value. A specific implementation is described in any of the following embodiments:

[0274] In one embodiment, a computer device performs entropy encoding on the attribute information of each point in point cloud data to obtain S bits of data. Entropy encoding includes at least one of the following encoding modes: context-based encoding mode and bypass encoding mode. The specific implementation of entropy encoding is described below:

[0275] In one implementation, the computer device encodes the attribute information of each point in the point cloud data using a context-based encoding mode, obtaining S bits of data. In one embodiment, the computer device encodes the attribute information of each point in the point cloud data using the same context model, obtaining S bits of data. In another embodiment, the computer device encodes the attribute information of each point in the point cloud data using different context models, obtaining S bits of data. Further, the S bits of data are divided into a first bit set and a second bit set; the computer device can indicate the division method of the S bits of data using at least one preset value during encoding, and different bit sets correspond to different decoding methods; for example, the first bit set corresponds to an encoding mode based on a first context model, and the second bit set corresponds to an encoding mode based on a second context model.

[0276] In another implementation, the computer device uses a bypass coding mode to encode the attribute information of each point in the point cloud data, obtaining S bits of data. In one embodiment, the attribute information of the point cloud data includes a first attribute information set and a second attribute information set. When encoding, the computer device can use any two of the following: an encoding mode based on a first context model, an encoding mode based on a second context model, and a bypass coding mode to encode the first attribute information set and the second attribute information set to obtain first encoded data and second encoded data. For example, the computer device uses an encoding mode based on the first context model to encode the attribute information in the first attribute information set to obtain first encoded data, and uses a bypass coding mode to encode the attribute information in the second attribute information set to obtain second encoded data.

[0277] In another embodiment, the computer device first performs binarization processing (such as exponential Golomb coding) on ​​the attribute information of each point to obtain the binarization result, and then performs entropy coding on the binarization result to obtain S bits of data.

[0278] In one implementation, the computer device first performs a shift and remainder operation on the attribute information of each point to obtain the shift and remainder operation result, then performs binarization on the shift and remainder operation result to obtain the binarization result, and finally performs entropy encoding on the binarization result to obtain S bits of data.

[0279] In another implementation, the computer device performs run-length encoding on the attribute information of each point to obtain S bits of data, where S is a positive integer. The S bits of data are used to indicate run-length encoding parameters, which include: run length, and encoding information of non-preset values ​​(such as 0) in the attribute information of each point. The encoding method for non-preset value attribute information may include, but is not limited to: encoding method based on residual value, encoding method based on transform coefficient, and entropy encoding.

[0280] In another implementation, the computer device can also perform a shift and remainder operation on the run-length encoding result to obtain S bits of data, where S is a positive integer.

[0281] In another embodiment, the computer device may also binarize the run-length encoding result to obtain S bits of data, where S is a positive integer.

[0282] In another embodiment, the computer device may further perform a shift and remainder operation on the run-length encoding result to obtain a shift and remainder result, and then perform binarization on the shift and remainder result to obtain S bits of data, where S is a positive integer.

[0283] In this embodiment, point cloud data to be encoded is acquired, and the point cloud data to be encoded is encoded to obtain bitstream data of the point cloud data. During the encoding process, one or more encoding methods can be used to encode the attribute information of the point cloud data; alternatively, the attribute information of the point cloud data can be preprocessed first, and then the preprocessing result can be encoded; furthermore, the preprocessing method can be indicated by indication information, and the bitstream data division method can be indicated by at least one preset value; through the above methods, the encoding efficiency of point cloud attribute information can be improved.

[0284] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below.

[0285] Please see Figure 7 , Figure 7 This is a schematic diagram of a point cloud processing device provided in an embodiment of this application; the point cloud processing device can be a computer program (including program code) running in a decoding device, for example, the point cloud processing device can be application software in the decoding device. Figure 7 As shown, the point cloud processing device includes an acquisition unit 701 and a processing unit 702.

[0286] Please see Figure 7 In one exemplary embodiment, the various units are described in detail below:

[0287] Acquisition unit 701 is used to acquire the bitstream data of point cloud data;

[0288] The processing unit 702 is used to parse the bit stream data to obtain the grouping information of the point cloud data. The grouping information is used to indicate the association relationship between the various groups corresponding to the point cloud data.

[0289] And it is used to optimize the grouping prediction process based on the correlation between each group to obtain the attribute information of the point cloud data;

[0290] And it is used to present point cloud data according to the attribute information of point cloud data.

[0291] In one implementation, the points in the point cloud data are divided into K groups, and the points in the K groups are arranged in decoding order, where K is an integer greater than 1; the optimization includes optimizing the selection of prediction candidate points in the group prediction process; the method of optimization by the processing unit 702 includes at least one of the following:

[0292] Add the target predicted point to the prediction candidate points of group P+1. The value of the target predicted point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups, and the coordinates of the target predicted point are the average geometric coordinates of the points in the previous P groups; or...

[0293] Replace the points to be replaced in the predicted candidate points of group P+1 with the target predicted points; or...

[0294] If the sum of the points in the first P groups is less than the threshold for the number of neighbors, then the target prediction point will be used as the prediction candidate point in the P+1 group.

[0295] In one implementation, the points in the point cloud data are divided into K groups, where K is an integer greater than 1; the optimization includes optimizing the acquisition of residual values ​​during the group prediction process, which includes:

[0296] Obtain the secondary prediction residual value and reference residual value corresponding to the i-th group, where i is a positive integer less than or equal to K;

[0297] The residual value of each point in the i-th group is calculated based on the secondary prediction residual value and the reference residual value.

[0298] In one implementation, the K groups of points are arranged in decoding order, and the processing unit 702 is used to obtain the secondary prediction residual value and reference residual value corresponding to the i-th group, specifically for:

[0299] The bitstream data is parsed to obtain the secondary prediction residual value and reference residual value corresponding to the i-th group; or,

[0300] The bitstream data is parsed to obtain the secondary prediction residual value corresponding to the i-th group, and the reference residual value corresponding to the i-th group is determined based on the residual value of at least one point in the first i groups.

[0301] In one implementation, the processing unit 702 is configured to determine the reference residual value corresponding to the i-th group based on the residual values ​​of at least one point in the first i-1 groups, specifically configured to:

[0302] The residual value at any point in the first i groups is determined as the reference residual value for the i-th group; or,

[0303] Calculate the reference residual value for the i-th group based on the residual values ​​of at least two points in the first i-th group.

[0304] In one implementation, the points in the point cloud data are divided into K groups, where K is an integer greater than 1; the optimization includes optimizing the acquisition of transformation coefficient values ​​during the group prediction process, which includes:

[0305] Obtain the quadratic transformation coefficient value and the reference transformation coefficient value corresponding to the i-th group, where i is a positive integer less than or equal to K;

[0306] The transformation coefficient values ​​of each point in the i-th group are calculated based on the quadratic transformation coefficient values ​​and the reference transformation coefficient values.

[0307] In one implementation, the K groups of points are arranged in decoding order, and the processing unit 702 is used to obtain the quadratic transform coefficient value and the reference transform coefficient value corresponding to the i-th group, specifically for:

[0308] The bitstream data is parsed to obtain the quadratic transform coefficient values ​​and reference transform coefficient values ​​corresponding to the i-th group; or,

[0309] The bitstream data is parsed to obtain the second transformation coefficient values ​​corresponding to the i-th group, and the reference transformation coefficient values ​​corresponding to the i-th group are determined based on the coefficient values ​​of at least one point in the first i groups.

[0310] In one embodiment, the processing unit 702 is configured to determine the reference transformation coefficient value corresponding to the i-th group based on the coefficient values ​​of at least one point in the first i-th group, specifically configured to:

[0311] The transformation coefficient value of any point in the first i groups is determined as the reference transformation coefficient value of the i-th group; or...

[0312] Based on the transformation coefficient values ​​of at least two points in the first i groups, calculate the reference transformation coefficient value for the i-th group.

[0313] In one implementation, the points in the point cloud data are divided into K groups, each point in the i-th group is associated with R attributes, each attribute corresponds to M transformation coefficients, and the M transformation coefficients corresponding to each attribute include 1 first transformation coefficient and M-1 second transformation coefficients; R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K; the processing unit 702 is further configured to:

[0314] Directly decode the first transform coefficient in the i-th group, and perform intra-group run-length decoding on the second transform coefficient in the i-th group; or,

[0315] If the group run length corresponding to the i-th group is not zero, then perform inter-group run decoding on the transform coefficients in the i-th group; if the group run length corresponding to the i-th group is zero, then decode the transform coefficients in the i-th group one by one; or...

[0316] For the first transform coefficient in the i-th group, perform direct decoding. If the group run length corresponding to the i-th group is not zero, then perform inter-group run decoding for the second transform coefficient in the i-th group; if the group run length corresponding to the i-th group is zero, then decode the second transform coefficient in the i-th group one by one; or...

[0317] The first transform coefficient in the i-th group is decoded by shifting and taking the remainder, and the second transform coefficient in the i-th group is decoded directly.

[0318] In one implementation, the points in the point cloud data are divided into K groups, the i-th group includes M points, and each point is associated with R attributes, each attribute corresponding to M transformation coefficients; R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K; the processing unit 702 is further configured to:

[0319] Perform shift-and-remainder decoding on the non-zero transform coefficients in the i-th group; or,

[0320] The non-zero transform coefficients in the i-th group are directly decoded.

[0321] In one implementation, the points in the point cloud data are divided into K groups, and the attribute of the j-th point in the i-th group consists of Q component attributes, each component attribute corresponding to a transformation coefficient; Q is a positive integer, K is an integer greater than 1, i is a positive integer less than or equal to K, and j is a positive integer; the processing unit 702 is further configured to:

[0322] Directly decode the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes; or,

[0323] Perform shift-and-remainder decoding on the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes.

[0324] In one implementation, the bitstream data includes S bits of data, where S is a positive integer; the processing unit 702 is further configured to:

[0325] Entropy decoding is performed on S bits of data to obtain the attribute information of the point cloud data; or,

[0326] Run-length decoding is performed on S bits of data to obtain the attribute information of the point cloud data;

[0327] Among them, entropy decoding includes at least one of the following decoding modes: context-based decoding mode and bypass decoding mode; the attribute information of point cloud data includes at least one of the following: residual value of each point and transformation coefficient of each point.

[0328] In one implementation, entropy decoding includes a context-based decoding mode and a bypass decoding mode. The context-based decoding mode includes a decoding mode based on a first context model and a decoding mode based on a second context model. The S bits of data include first encoded data and second encoded data. The process of entropy decoding of the S bits of data by the processing unit 702 includes:

[0329] The first encoded data is decoded using a context-based decoding mode, and the second encoded data is decoded using a bypass decoding mode; or,

[0330] The first encoded data is decoded using a decoding mode based on a first context model, and the second encoded data is decoded using a decoding mode based on a second context model.

[0331] In one implementation, the first encoded data and the second encoded data are obtained by dividing S bits of data based on at least one preset value.

[0332] In one embodiment, the processing unit 702 is configured to perform entropy decoding on S bits of data to obtain attribute information of the point cloud data, specifically for:

[0333] Entropy decoding is performed on S bits of data to obtain the binarized result;

[0334] Based on the binarization results, the attribute information of the point cloud data is restored.

[0335] In one embodiment, the processing unit 702 is used to restore the attribute information of the point cloud data based on the binarization processing result, specifically for:

[0336] Based on the binarization result, restore the shift and remainder result;

[0337] Based on the result of the shift and remainder processing, calculate the attribute information of the point cloud data.

[0338] According to one embodiment of this application, Figure 3 and Figure 5 The point cloud processing method shown can be partially implemented by... Figure 7 The point cloud processing apparatus shown is executed by individual units. For example, Figure 3 Step S301 shown can be performed by Figure 7 The acquisition unit 701 shown is executed, and steps S302-S304 can be performed by... Figure 7The processing unit 702 shown executes the operation; Figure 5 Step S501 shown can be performed by Figure 7 The acquisition unit 701 shown is executed, and step S502 can be performed by... Figure 7 The processing unit 702 shown is executed. Figure 7 The units in the point cloud processing device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the point cloud processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0339] According to another embodiment of this application, the following can be executed by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). Figure 3 and Figure 5 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 7 The point cloud processing apparatus shown herein, and the point cloud processing method for implementing the embodiments of this application, are described. A computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and executed therein.

[0340] Based on the same inventive concept, the principle and beneficial effects of the point cloud processing device provided in the embodiments of this application are similar to the principle and beneficial effects of the point cloud processing method in the embodiments of this application. For details, please refer to the principle and beneficial effects of the method implementation. For the sake of brevity, these will not be repeated here.

[0341] Please see Figure 8 , Figure 8 This is a schematic diagram of another point cloud processing device provided in an embodiment of this application; the point cloud processing device can be a computer program (including program code) running in an encoding device, for example, the point cloud processing device can be application software in the encoding device. Figure 8 As shown, the point cloud processing device includes an acquisition unit 801 and a processing unit 802. Please refer to [link / reference]. Figure 8 The detailed descriptions of each unit are as follows:

[0342] Acquisition unit 801 is used to acquire point cloud data to be encoded;

[0343] The processing unit 802 is used to divide the points in the point cloud data to obtain the grouping information of the point cloud data. The grouping information is used to indicate the relationship between the various groups corresponding to the point cloud data.

[0344] And it is used to optimize the grouping prediction process based on the correlation between each group to obtain the attribute information of the point cloud data;

[0345] And it is used to encode the attribute information of point cloud data to obtain the bitstream data of point cloud data.

[0346] In one embodiment, the processing unit 802 is used to divide the points in the point cloud data to obtain grouping information of the point cloud data, specifically for:

[0347] The points in the point cloud data are divided according to the space filling curve to obtain the grouping information of the point cloud data;

[0348] The coordinate arrangement order of the space filling curves is determined according to the 3D bounding box to which the point cloud data belongs.

[0349] In one embodiment, the processing unit 802 is configured to divide the points in the point cloud data according to the space-filling curve to obtain the grouping information of the point cloud data, specifically for:

[0350] The points in the point cloud data are divided into K groups according to the space filling curve, where K is an integer greater than 1;

[0351] If the number of points in the i-th group is greater than the first quantity threshold, then the points in the i-th group whose coordinate values ​​in the target direction belong to the numerical range are divided into a group. The target direction is the direction corresponding to the shortest side in the three-dimensional bounding box to which the points in the i-th group belong, where i is a positive integer less than or equal to K.

[0352] If the number of points in the i-th group is odd, then the number of points in the i-th group is incremented or decremented by one.

[0353] In one implementation, the K groups of points are arranged in coding order, and the processing unit 802 is further configured to:

[0354] If the number of points in the i-th group is less than the second quantity threshold N, then the i-th group is merged with the adjacent groups of the i-th group to obtain a merged group. The second quantity threshold N is less than the first quantity threshold, and N is a positive integer.

[0355] The merged group is divided into an average value to obtain the updated i-th group and the updated adjacent groups of the i-th group; or, the N points in the merged group are divided into the i-th group, and the other points in the merged group other than the N points are divided into the adjacent groups of the i-th group.

[0356] In one implementation, the points in the point cloud data are divided into K groups, where K is an integer greater than 1; optimization includes optimizing the selection of prediction candidate points during the group prediction process; the method by which the processing unit 802 performs selection optimization includes at least one of the following:

[0357] Add the target predicted point to the prediction candidate points of group P+1. The value of the target predicted point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups, and the coordinates of the target predicted point are the average geometric coordinates of the points in the previous P groups; or...

[0358] Replace the points to be replaced in the predicted candidate points of group P+1 with the target predicted points; or...

[0359] If the sum of the points in the first P groups is less than the threshold for the number of neighbors, then the target prediction point will be used as the prediction candidate point in the P+1 group.

[0360] In one implementation, the points in the point cloud data are divided into K groups, and the points in the K groups are arranged in encoding order, where K is an integer greater than 1; the optimization includes optimizing the acquisition of the secondary prediction residual value in the group prediction process, and the optimization of the acquisition of the secondary prediction residual value by the processing unit 802 includes:

[0361] Obtain the residual value of each point in the i-th group. The residual value of each point is calculated based on the prediction residual value and the attribute reconstruction value of that point. i is a positive integer less than or equal to K.

[0362] Based on the residual value of at least one point in the first i groups, determine the reference residual value corresponding to the i-th group;

[0363] The secondary prediction residual is calculated based on the residual values ​​of each point in the i-th group and the corresponding reference residual value of the i-th group.

[0364] In one implementation, the processing unit 802 is configured to determine a reference residual value corresponding to the i-th group based on the residual values ​​of at least one point in the first i groups, specifically configured to:

[0365] The residual value at any point in the first i groups is determined as the reference residual value for the i-th group; or,

[0366] Calculate the reference residual value for the i-th group based on the residual values ​​of at least two points in the first i-th group.

[0367] In one implementation, the points in the point cloud data are divided into K groups, and the points in the K groups are arranged in encoding order, where K is an integer greater than 1; the optimization includes optimizing the acquisition of the secondary transformation coefficient values ​​in the group prediction process, and the optimization of the acquisition of the secondary prediction transformation coefficient values ​​by the processing unit 802 includes:

[0368] Obtain the transformation coefficient values ​​for each point in the i-th group, where i is a positive integer less than or equal to K;

[0369] Based on the transformation coefficient values ​​of at least one point in the first i groups, determine the reference transformation coefficient values ​​for the i-th group;

[0370] The secondary transformation coefficient values ​​are calculated based on the transformation coefficient values ​​of each point in the i-th group and the reference transformation coefficient values ​​of the i-th group.

[0371] In one embodiment, the processing unit 802 is configured to determine the reference transformation coefficient value of the i-th group based on the transformation coefficient value of at least one point in the first i-th group, specifically configured to:

[0372] The transformation coefficient value of any point in the first i groups is determined as the reference transformation coefficient value of the i-th group; or...

[0373] Based on the transformation coefficient values ​​of at least two points in the first i groups, calculate the reference transformation coefficient value for the i-th group.

[0374] In one embodiment, the processing unit 802 is configured to encode the attribute information of the point cloud data to obtain the bitstream data of the point cloud data, specifically for:

[0375] Based on the correlation between each group and the distribution characteristics of the attribute information of the point cloud data, the attribute information of the point cloud data is encoded to obtain the bitstream data of the point cloud data.

[0376] In one implementation, the attribute information of the point cloud data is transformation coefficients. The points in the point cloud data are divided into K groups. Each point in the i-th group is associated with R attributes, and each attribute corresponds to M transformation coefficients. The M transformation coefficients corresponding to each attribute include one first transformation coefficient and M-1 second transformation coefficients. R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K. The process by which the processing unit 802 encodes the attribute information of the point cloud data according to the association relationship between the groups and the distribution characteristics of the attribute information of the point cloud data includes:

[0377] The first transform coefficient in the i-th group is directly encoded, and the second transform coefficient in the i-th group is run-length encoded; or,

[0378] If all the transformation coefficients in group i are preset values, then the group travel length corresponding to the point in group K is incremented by one; if at least one transformation coefficient in group i has a value other than the preset value, then the group travel length corresponding to the point in group K is set to zero, and the transformation coefficients in group i are encoded one by one; or,

[0379] The first transformation coefficient in the i-th group is directly encoded. If all the values ​​of the second transformation coefficients in the i-th group are preset values, the group travel length corresponding to the K-th group point is incremented by one. If at least one of the second transformation coefficients in the i-th group has a value that is not a preset value, the group travel length corresponding to the K-th group point is set to zero, and the second transformation coefficients in the i-th group are encoded one by one. Or,

[0380] The first transform coefficient in the i-th group is encoded by shift and remainder, and the second transform coefficient in the i-th group is encoded directly.

[0381] In one implementation, the points in the point cloud data are divided into K groups, with the i-th group containing M points. Each point is associated with R attributes, and each attribute corresponds to M transformation coefficients. R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K. The process by which the processing unit 802 encodes the attribute information of the point cloud data according to the association between the groups and the distribution characteristics of the attribute information of the point cloud data includes:

[0382] If M is less than the third quantity threshold, then the non-zero transform coefficients in the i-th group are subjected to shift-and-remainder encoding.

[0383] If M is greater than or equal to the third quantity threshold, then the non-zero transform coefficients in the i-th group are directly encoded.

[0384] In one implementation, the points in the point cloud data are divided into K groups, and the attribute of the j-th point in the i-th group consists of Q component attributes, each component attribute corresponding to a transformation coefficient; K is an integer greater than 1, Q is a positive integer, i is a positive integer less than or equal to K, and j is a positive integer; the process by which the processing unit 802 encodes the attribute information of the point cloud data according to the correlation between the groups and the distribution characteristics of the attribute information of the point cloud data includes:

[0385] Calculate the sum of the absolute values ​​of at least two component attributes among the Q component attributes to obtain the combined component attribute value;

[0386] If the combined value of the component attributes is less than the numerical threshold, then the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes are directly encoded;

[0387] If the combined value of the component attributes is greater than or equal to the numerical threshold, then the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes are subjected to shift-and-remainder encoding.

[0388] In one implementation, the attribute information of the point cloud data includes at least one of the following: the residual value of each point, and the transformation coefficient of each point; the process by which the processing unit 802 encodes the attribute information of the point cloud data includes:

[0389] Entropy encoding is performed on the attribute information of the point cloud data to obtain S bits of data, where S is a positive integer; or,

[0390] Run-length encoding is performed on the attribute information of the point cloud data to obtain S bits of data.

[0391] Entropy coding includes at least one of the following coding modes: context-based coding mode and bypass coding mode.

[0392] In one implementation, entropy encoding includes a context-based encoding mode and a bypass encoding mode; the context-based decoding mode includes a decoding mode based on a first context model and a decoding mode based on a second context model; the attribute information of the point cloud data includes a first attribute information set and a second attribute information set; the process of entropy encoding the attribute information of the point cloud data by the processing unit 802 includes:

[0393] The attribute information in the first attribute information set is encoded using a context-based encoding scheme, and the attribute information in the second attribute information set is encoded using a bypass encoding scheme; or,

[0394] The attribute information in the first attribute information set is encoded using an encoding mode based on the first context model, and the attribute information in the second attribute information set is encoded using an encoding mode based on the second context model.

[0395] In one implementation, the S bits of data include first encoded data and second encoded data; the first encoded data and the second encoded data are obtained by dividing the S bits of data based on at least one preset value.

[0396] In one implementation, the processing unit 802 is used to perform entropy encoding on the attribute information of the point cloud data to obtain S bits of data, specifically for:

[0397] The attribute information of the point cloud data is binarized to obtain the binarized result;

[0398] Entropy encoding is performed on the binarization result to obtain S bits of data.

[0399] In one embodiment, the processing unit 802 is configured to perform binarization processing on the attribute information of the point cloud data to obtain a binarization processing result, specifically for:

[0400] The attribute information of the point cloud data is shifted and moduloed to obtain the shift and modulo result;

[0401] The result of the shift and remainder processing is binarized to obtain the binarized result.

[0402] According to one embodiment of this application, Figure 4 and Figure 6 The point cloud processing method shown can be partially implemented by... Figure 8 The point cloud processing apparatus shown is executed by individual units. For example, Figure 4 Step S401 shown can be performed by Figure 8 The acquisition unit 801 shown is executed, and steps S402-S404 can be performed by... Figure 8 The processing unit 802 shown executes this; Figure 6 Step S601 shown can be performed by Figure 8 The acquisition unit 801 shown is executed, and step S602 can be performed by... Figure 8 The processing unit 802 shown executes. Figure 8 The units in the point cloud processing device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the point cloud processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0403] According to another embodiment of this application, the following can be executed by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). Figure 4 and Figure 6 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 8 The point cloud processing apparatus shown herein, and the point cloud processing method for implementing the embodiments of this application, are described. A computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and executed therein.

[0404] Based on the same inventive concept, the principle and beneficial effects of the point cloud processing device provided in the embodiments of this application are similar to the principle and beneficial effects of the point cloud processing method in the embodiments of this application. For details, please refer to the principle and beneficial effects of the method implementation. For the sake of brevity, these will not be repeated here.

[0405] Figure 9This is a schematic diagram of a decoding device provided in an embodiment of this application; the decoding device can be a computer device used by a user of point cloud media, and the computer device can be a terminal (such as a PC, a smart mobile device (such as a smartphone), a VR device (such as a VR headset, VR glasses, etc.)). Figure 9 As shown, the decoding device includes a receiver 901, a processor 902, a memory 903, and a display / playback device 904.

[0406] in:

[0407] Receiver 901 is used to enable transmission and interaction between the decoder and other devices, specifically to facilitate the transmission of point cloud media between the encoding and decoding devices. That is, the decoding device receives the relevant media resources of the point cloud media transmitted by the encoding device through receiver 901.

[0408] Processor 902 (or CPU (Central Processing Unit)) is the processing core of the encoding device. Processor 902 is adapted to implement one or more program instructions, specifically to load and execute one or more program instructions to achieve... Figure 3 and Figure 5 The flowchart of the point cloud processing method is shown.

[0409] Memory 903 is a memory device in the decoding device used to store programs and media resources. It is understood that memory 903 here can include the built-in storage medium of the decoding device, or it can include extended storage media supported by the decoding device. It should be noted that memory 903 can be high-speed RAM, or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one memory located remotely from the aforementioned processor. Memory 903 provides storage space for storing the operating system of the decoding device. Furthermore, this storage space is also used to store computer programs, which include program instructions adapted to be called and executed by the processor to perform the various steps of the point cloud processing method. In addition, memory 903 can also be used to store a 3D image of the point cloud media formed after processor processing, the audio content corresponding to the 3D image, and information required for rendering the 3D image and audio content.

[0410] Display / playback device 904 is used to output rendered sound and 3D images.

[0411] Please see again Figure 9 The processor 902 may include a parser 921, a decoder 922, a converter 923, and a renderer 924; wherein:

[0412] The parser 921 is used to decapsulate the rendering media encapsulation file from the encoding device. Specifically, it decapsulates the media file resources according to the file format requirements of point cloud media to obtain audio and video streams, and provides the audio and video streams to the decoder 922.

[0413] Decoder 922 decodes the audio stream to obtain the audio content and provides it to the renderer for audio rendering. Additionally, decoder 922 decodes the video stream to obtain a 2D image. Based on the metadata provided by the media presentation description information, if the metadata indicates that the point cloud media has undergone a region encapsulation process, the 2D image refers to an encapsulated image; if the metadata indicates that the point cloud media has not undergone a region encapsulation process, the planar image refers to a projected image.

[0414] Converter 923 is used to convert 2D images into 3D images. If the point cloud media has undergone a region encapsulation process, converter 923 will first decapsulate the encapsulated image to obtain a projected image. Then, the projected image is reconstructed to obtain a 3D image. If the rendering media has not undergone a region encapsulation process, converter 923 will directly reconstruct the projected image to obtain a 3D image.

[0415] Renderer 924 is used to render the audio content and 3D images of point cloud media. Specifically, it renders the audio content and 3D images based on the metadata related to rendering and viewport in the media presentation description information, and then outputs the rendered content to the display / playback device.

[0416] In one exemplary embodiment, the processor 902 (specifically, the devices included in the processor) executes instructions by calling one or more instructions stored in memory. Figure 3 or Figure 5 The illustrated point cloud processing method includes the following steps. Specifically, the memory stores one or more first instructions, which are adapted to be loaded by the processor 902 and executed as follows:

[0417] Obtain the bitstream data of point cloud data;

[0418] The bitstream data is parsed to obtain the grouping information of the point cloud data. The grouping information is used to indicate the relationship between the various groups of the point cloud data.

[0419] The grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data;

[0420] The point cloud data is presented according to its attribute information.

[0421] In one implementation, the points in the point cloud data are divided into K groups, and the points in the K groups are arranged in decoding order, where K is an integer greater than 1; the optimization includes optimizing the selection of prediction candidate points in the group prediction process; the method by which the processor 902 performs the selection optimization includes at least one of the following:

[0422] Add the target predicted point to the prediction candidate points of group P+1. The value of the target predicted point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups, and the coordinates of the target predicted point are the average geometric coordinates of the points in the previous P groups; or...

[0423] Replace the points to be replaced in the predicted candidate points of group P+1 with the target predicted points; or...

[0424] If the sum of the points in the first P groups is less than the threshold for the number of neighbors, then the target prediction point will be used as the prediction candidate point in the P+1 group.

[0425] In one implementation, the points in the point cloud data are divided into K groups, where K is an integer greater than 1; the optimization includes optimizing the acquisition of residual values ​​during the group prediction process, and the processor 902 optimizes the acquisition of residual values ​​by:

[0426] Obtain the secondary prediction residual value and reference residual value corresponding to the i-th group, where i is a positive integer less than or equal to K;

[0427] The residual value of each point in the i-th group is calculated based on the secondary prediction residual value and the reference residual value.

[0428] In one implementation, the K groups of points are arranged in decoding order. A specific embodiment of the processor 902 obtaining the secondary prediction residual value and reference residual value corresponding to the i-th group is as follows:

[0429] The bitstream data is parsed to obtain the secondary prediction residual value and reference residual value corresponding to the i-th group; or,

[0430] The bitstream data is parsed to obtain the secondary prediction residual value corresponding to the i-th group, and the reference residual value corresponding to the i-th group is determined based on the residual value of at least one point in the first i groups.

[0431] In one embodiment, the processor 902 determines the reference residual value corresponding to the i-th group based on the residual value of at least one point in the first i-1 groups. A specific example of this is as follows:

[0432] The residual value at any point in the first i groups is determined as the reference residual value for the i-th group; or,

[0433] Calculate the reference residual value for the i-th group based on the residual values ​​of at least two points in the first i-th group.

[0434] In one implementation, the points in the point cloud data are divided into K groups, where K is an integer greater than 1; the optimization includes optimizing the acquisition of transformation coefficient values ​​during the group prediction process, and the processor 902 optimizes the acquisition of transformation coefficient values ​​by:

[0435] Obtain the quadratic transformation coefficient value and the reference transformation coefficient value corresponding to the i-th group, where i is a positive integer less than or equal to K;

[0436] The transformation coefficient values ​​of each point in the i-th group are calculated based on the quadratic transformation coefficient values ​​and the reference transformation coefficient values.

[0437] In one embodiment, the K groups of points are arranged in decoding order, and the processor 902 obtains the quadratic transform coefficient value and the reference transform coefficient value corresponding to the i-th group as follows:

[0438] The bitstream data is parsed to obtain the quadratic transform coefficient values ​​and reference transform coefficient values ​​corresponding to the i-th group; or,

[0439] The bitstream data is parsed to obtain the second transformation coefficient values ​​corresponding to the i-th group, and the reference transformation coefficient values ​​corresponding to the i-th group are determined based on the coefficient values ​​of at least one point in the first i groups.

[0440] In one embodiment, the processor 902 determines the reference transformation coefficient value corresponding to the i-th group based on the coefficient value of at least one point in the first i groups. A specific example of this is as follows:

[0441] The transformation coefficient value of any point in the first i groups is determined as the reference transformation coefficient value of the i-th group; or...

[0442] Based on the transformation coefficient values ​​of at least two points in the first i groups, calculate the reference transformation coefficient value for the i-th group.

[0443] In one implementation, the points in the point cloud data are divided into K groups. Each point in the i-th group is associated with R attributes, each attribute corresponds to M transformation coefficients, and the M transformation coefficients corresponding to each attribute include 1 first transformation coefficient and M-1 second transformation coefficients; R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K; the computer program in memory 903 is loaded by processor 902 and also performs the following steps:

[0444] Directly decode the first transform coefficient in the i-th group, and perform intra-group run-length decoding on the second transform coefficient in the i-th group; or,

[0445] If the group run length corresponding to the i-th group is not zero, then perform inter-group run decoding on the transform coefficients in the i-th group; if the group run length corresponding to the i-th group is zero, then decode the transform coefficients in the i-th group one by one; or...

[0446] For the first transform coefficient in the i-th group, perform direct decoding. If the group run length corresponding to the i-th group is not zero, then perform inter-group run decoding for the second transform coefficient in the i-th group; if the group run length corresponding to the i-th group is zero, then decode the second transform coefficient in the i-th group one by one; or...

[0447] The first transform coefficient in the i-th group is decoded by shifting and taking the remainder, and the second transform coefficient in the i-th group is decoded directly.

[0448] In one implementation, the points in the point cloud data are divided into K groups, the i-th group includes M points, and each point is associated with R attributes, each attribute corresponding to M transformation coefficients; R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K; the computer program in memory 903 is loaded by processor 902 and also performs the following steps:

[0449] Perform shift-and-remainder decoding on the non-zero transform coefficients in the i-th group; or,

[0450] The non-zero transform coefficients in the i-th group are directly decoded.

[0451] In one implementation, the points in the point cloud data are divided into K groups, and the attribute of the j-th point in the i-th group consists of Q component attributes, each component attribute corresponding to a transformation coefficient; Q is a positive integer, K is an integer greater than 1, i is a positive integer less than or equal to K, and j is a positive integer; the computer program in memory 903 is loaded by processor 902 and also performs the following steps:

[0452] Directly decode the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes; or,

[0453] Perform shift-and-remainder decoding on the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes.

[0454] In one embodiment, the bitstream data includes S bits of data, where S is a positive integer; the computer program in memory 903 is loaded by processor 902 and also performs the following steps:

[0455] Entropy decoding is performed on S bits of data to obtain the attribute information of the point cloud data; or,

[0456] Run-length decoding is performed on S bits of data to obtain the attribute information of the point cloud data;

[0457] Among them, entropy decoding includes at least one of the following decoding modes: context-based decoding mode and bypass decoding mode; the attribute information of point cloud data includes at least one of the following: residual value of each point and transformation coefficient of each point.

[0458] In one implementation, entropy decoding includes a context-based decoding mode and a bypass decoding mode. The context-based decoding mode includes a decoding mode based on a first context model and a decoding mode based on a second context model. The S bits of data include first encoded data and second encoded data. The process of entropy decoding of the S bits of data by the processor 902 includes:

[0459] The first encoded data is decoded using a context-based decoding mode, and the second encoded data is decoded using a bypass decoding mode; or,

[0460] The first encoded data is decoded using a decoding mode based on a first context model, and the second encoded data is decoded using a decoding mode based on a second context model.

[0461] In one embodiment, the first encoded data and the second encoded data are obtained by dividing S bits of data based on at least one preset value.

[0462] In one embodiment, the processor 902 performs entropy decoding on S bits of data to obtain the attribute information of the point cloud data. A specific example is as follows:

[0463] Entropy decoding is performed on S bits of data to obtain the binarized result;

[0464] Based on the binarization results, the attribute information of the point cloud data is restored.

[0465] In one embodiment, the processor 902 restores the attribute information of the point cloud data based on the binarization processing result. A specific example is as follows:

[0466] Based on the binarization result, restore the shift and remainder result;

[0467] Based on the result of the shift and remainder processing, calculate the attribute information of the point cloud data.

[0468] Based on the same inventive concept, the principle and beneficial effects of the decoding device provided in the embodiments of this application are similar to the principle and beneficial effects of the point cloud processing method in the embodiments of this application. For details, please refer to the principle and beneficial effects of the method implementation. For the sake of brevity, these will not be repeated here.

[0469] Figure 10 This is a schematic diagram of the structure of an encoding device provided in an embodiment of this application; the encoding device may be a computer device used by a provider of point cloud media, which may be a terminal (such as a PC, a smart mobile device (such as a smartphone) or a server. Figure 10As shown, the encoding device includes a capture device 1001, a processor 1002, a memory 1003, and a transmitter 1004. Wherein:

[0470] The capture device 1001 is used to acquire raw data (including audio and video content synchronized in time and space) of point cloud media from real-world sound-visual scenes. The capture device 1001 may include, but is not limited to, audio devices, camera devices, and sensing devices. Audio devices may include audio sensors, microphones, etc. Camera devices may include ordinary cameras, stereo cameras, light field cameras, etc. Sensing devices may include laser devices, radar devices, etc.

[0471] Processor 1002 (or CPU (Central Processing Unit)) is the processing core of the encoding device. Processor 1002 is adapted to implement one or more program instructions, specifically to load and execute one or more program instructions to achieve... Figure 4 or Figure 6 The flowchart of the point cloud processing method is shown.

[0472] Memory 1003 is a memory device in the encoding apparatus used to store programs and media resources. It is understood that memory 1003 here can include the built-in storage medium of the encoding apparatus, or it can include extended storage media supported by the encoding apparatus. It should be noted that the memory can be high-speed RAM, or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one memory located remotely from the aforementioned processor. The memory provides storage space for storing the operating system of the encoding apparatus. Furthermore, this storage space is also used to store computer programs, which include program instructions adapted to be called and executed by the processor to perform the various steps of the point cloud processing method. In addition, memory 1003 can also be used to store point cloud media files formed after processing by the processor, which include media file resources and media presentation description information.

[0473] The transmitter 1004 is used to enable transmission and interaction between the encoding device and other devices, specifically to enable the transmission of point cloud media between the encoding device and the content playback device. That is, the encoding device transmits relevant media resources of point cloud media to the content playback device through the transmitter 1004.

[0474] Please see again Figure 10 The processor 1002 may include a converter 1021, an encoder 1022, and a packager 1023; wherein:

[0475] Converter 1021 performs a series of conversion processes on captured video content to make it suitable for video encoding of point cloud media. The conversion processes may include stitching and projection; optionally, they may also include region encapsulation. Converter 1021 can convert captured 3D video content into 2D images and provide them to the encoder for video encoding.

[0476] Encoder 1022 is used to encode the captured audio content to form an audio bitstream of point cloud media. It is also used to encode the 2D image obtained by converter 1021 to obtain a video bitstream.

[0477] The encapsulator 1023 encapsulates audio and video streams into a file container according to the point cloud media file format (such as ISOBMFF) to form a point cloud media file resource. This media file resource can be a media file or a media segment forming a point cloud media file. It also records the metadata of the point cloud media file resource using media presentation description information according to the point cloud media file format requirements. The encapsulated point cloud media file obtained by the encapsulator is stored in memory and provided to the content playback device as needed for point cloud media presentation.

[0478] The processor 1002 (specifically, the various components within the processor) executes instructions by calling one or more instructions from memory. Figure 4 or Figure 6 The illustrated point cloud processing method includes the following steps. Specifically, memory 1003 stores one or more first instructions, which are adapted to be loaded by processor 1002 and executed as follows:

[0479] Obtain the point cloud data to be encoded;

[0480] The points in the point cloud data are divided to obtain the grouping information of the point cloud data. The grouping information is used to indicate the relationship between the corresponding groups of the point cloud data.

[0481] The grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data;

[0482] The attribute information of the point cloud data is encoded to obtain the bitstream data of the point cloud data.

[0483] In one embodiment, the processor 1002 divides the points in the point cloud data to obtain the grouping information of the point cloud data. A specific example is as follows:

[0484] The points in the point cloud data are divided according to the space filling curve to obtain the grouping information of the point cloud data;

[0485] The coordinate arrangement order of the space filling curves is determined according to the 3D bounding box to which the point cloud data belongs.

[0486] In one embodiment, the processor 1002 divides the points in the point cloud data according to the space-filling curve to obtain the grouping information of the point cloud data. A specific example is as follows:

[0487] The points in the point cloud data are divided into K groups according to the space filling curve, where K is an integer greater than 1;

[0488] If the number of points in the i-th group is greater than the first quantity threshold, then the points in the i-th group whose coordinate values ​​in the target direction belong to the numerical range are divided into a group. The target direction is the direction corresponding to the shortest side in the three-dimensional bounding box to which the points in the i-th group belong, where i is a positive integer less than or equal to K.

[0489] If the number of points in the i-th group is odd, then the number of points in the i-th group is incremented or decremented by one.

[0490] In one embodiment, the K groups of points are arranged in coding order, and the computer program in memory 1003 is loaded by processor 1002 and also performs the following steps:

[0491] If the number of points in the i-th group is less than the second quantity threshold N, then the i-th group is merged with the adjacent groups of the i-th group to obtain a merged group. The second quantity threshold N is less than the first quantity threshold, and N is a positive integer.

[0492] The merged group is divided into an average value to obtain the updated i-th group and the updated adjacent groups of the i-th group; or, the N points in the merged group are divided into the i-th group, and the other points in the merged group other than the N points are divided into the adjacent groups of the i-th group.

[0493] In one implementation, the points in the point cloud data are divided into K groups, where K is an integer greater than 1; the optimization includes optimizing the selection of prediction candidate points during the group prediction process; specific embodiments of the processor 1002 performing selection optimization include at least one of the following:

[0494] Add the target predicted point to the prediction candidate points of group P+1. The value of the target predicted point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups, and the coordinates of the target predicted point are the average geometric coordinates of the points in the previous P groups; or...

[0495] Replace the points to be replaced in the predicted candidate points of group P+1 with the target predicted points; or...

[0496] If the sum of the points in the first P groups is less than the threshold for the number of neighbors, then the target prediction point will be used as the prediction candidate point in the P+1 group.

[0497] In one implementation, the points in the point cloud data are divided into K groups, and the points in the K groups are arranged in encoding order, where K is an integer greater than 1; the optimization includes optimizing the acquisition of the secondary prediction residual value in the group prediction process. A specific embodiment of the processor 1002 optimizing the acquisition of the secondary prediction residual value is as follows:

[0498] Obtain the residual value of each point in the i-th group. The residual value of each point is calculated based on the prediction residual value and the attribute reconstruction value of that point. i is a positive integer less than or equal to K.

[0499] Based on the residual value of at least one point in the first i groups, determine the reference residual value corresponding to the i-th group;

[0500] The secondary prediction residual is calculated based on the residual values ​​of each point in the i-th group and the corresponding reference residual value of the i-th group.

[0501] In one embodiment, the processor 1002 determines the reference residual value corresponding to the i-th group based on the residual value of at least one point in the first i groups. A specific example of this is:

[0502] The residual value at any point in the first i groups is determined as the reference residual value for the i-th group; or,

[0503] Calculate the reference residual value for the i-th group based on the residual values ​​of at least two points in the first i-th group.

[0504] In one implementation, the points in the point cloud data are divided into K groups, and the points in the K groups are arranged in encoding order, where K is an integer greater than 1; the optimization includes optimizing the acquisition of the secondary transformation coefficient values ​​in the group prediction process. A specific embodiment of the processor 1002 optimizing the acquisition of the secondary prediction transformation coefficient values ​​is as follows:

[0505] Obtain the transformation coefficient values ​​for each point in the i-th group, where i is a positive integer less than or equal to K;

[0506] Based on the transformation coefficient values ​​of at least one point in the first i groups, determine the reference transformation coefficient values ​​for the i-th group;

[0507] The secondary transformation coefficient values ​​are calculated based on the transformation coefficient values ​​of each point in the i-th group and the reference transformation coefficient values ​​of the i-th group.

[0508] In one embodiment, the processor 1002 determines the reference transformation coefficient value of the i-th group based on the transformation coefficient value of at least one point in the first i-th group. A specific example of this is as follows:

[0509] The transformation coefficient value of any point in the first i groups is determined as the reference transformation coefficient value of the i-th group; or...

[0510] Based on the transformation coefficient values ​​of at least two points in the first i groups, calculate the reference transformation coefficient value for the i-th group.

[0511] In one embodiment, the processor 1002 encodes the attribute information of the point cloud data to obtain the bitstream data of the point cloud data. A specific example is as follows:

[0512] Based on the correlation between each group and the distribution characteristics of the attribute information of the point cloud data, the attribute information of the point cloud data is encoded to obtain the bitstream data of the point cloud data.

[0513] In one implementation, the attribute information of the point cloud data is transform coefficients. The points in the point cloud data are divided into K groups. Each point in the i-th group is associated with R attributes, and each attribute corresponds to M transform coefficients. The M transform coefficients corresponding to each attribute include one first transform coefficient and M-1 second transform coefficients. R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K. The processor 1002 encodes the attribute information of the point cloud data according to the association relationship between the groups and the distribution characteristics of the attribute information of the point cloud data. A specific embodiment of this implementation is as follows:

[0514] The first transform coefficient in the i-th group is directly encoded, and the second transform coefficient in the i-th group is run-length encoded; or,

[0515] If all the transformation coefficients in group i are preset values, then the group travel length corresponding to the point in group K is incremented by one; if at least one transformation coefficient in group i has a value other than the preset value, then the group travel length corresponding to the point in group K is set to zero, and the transformation coefficients in group i are encoded one by one; or,

[0516] The first transformation coefficient in the i-th group is directly encoded. If all the values ​​of the second transformation coefficients in the i-th group are preset values, the group travel length corresponding to the K-th group point is incremented by one. If at least one of the second transformation coefficients in the i-th group has a value that is not a preset value, the group travel length corresponding to the K-th group point is set to zero, and the second transformation coefficients in the i-th group are encoded one by one. Or,

[0517] The first transform coefficient in the i-th group is encoded by shift and remainder, and the second transform coefficient in the i-th group is encoded directly.

[0518] In one implementation, the points in the point cloud data are divided into K groups, with the i-th group containing M points. Each point is associated with R attributes, and each attribute corresponds to M transformation coefficients; R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K. A specific embodiment of the processor 1002 encoding the attribute information of the point cloud data based on the association relationships between the groups and the distribution characteristics of the attribute information is as follows:

[0519] If M is less than the third quantity threshold, then the non-zero transform coefficients in the i-th group are subjected to shift-and-remainder encoding.

[0520] If M is greater than or equal to the third quantity threshold, then the non-zero transform coefficients in the i-th group are directly encoded.

[0521] In one implementation, the points in the point cloud data are divided into K groups, and the attribute of the j-th point in the i-th group consists of Q component attributes, each component attribute corresponding to a transformation coefficient; K is an integer greater than 1, Q is a positive integer, i is a positive integer less than or equal to K, and j is a positive integer; the processor 1002 encodes the attribute information of the point cloud data according to the correlation between the groups and the distribution characteristics of the attribute information of the point cloud data. A specific embodiment of this implementation is as follows:

[0522] Calculate the sum of the absolute values ​​of at least two component attributes among the Q component attributes to obtain the combined component attribute value;

[0523] If the combined value of the component attributes is less than the numerical threshold, then the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes are directly encoded;

[0524] If the combined value of the component attributes is greater than or equal to the numerical threshold, then the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes are subjected to shift-and-remainder encoding.

[0525] In one embodiment, the attribute information of the point cloud data includes at least one of the following: the residual value of each point, and the transformation coefficient of each point; a specific embodiment of the processor 1002 encoding the attribute information of the point cloud data is as follows:

[0526] Entropy encoding is performed on the attribute information of the point cloud data to obtain S bits of data, where S is a positive integer; or,

[0527] Run-length encoding is performed on the attribute information of the point cloud data to obtain S bits of data.

[0528] Entropy coding includes at least one of the following coding modes: context-based coding mode and bypass coding mode.

[0529] In one embodiment, entropy encoding includes a context-based encoding mode and a bypass encoding mode; the context-based decoding mode includes a decoding mode based on a first context model and a decoding mode based on a second context model; the attribute information of the point cloud data includes a first attribute information set and a second attribute information set; a specific embodiment of the processor 1002 performing entropy encoding on the attribute information of the point cloud data is as follows:

[0530] The attribute information in the first attribute information set is encoded using a context-based encoding scheme, and the attribute information in the second attribute information set is encoded using a bypass encoding scheme; or,

[0531] The attribute information in the first attribute information set is encoded using an encoding mode based on the first context model, and the attribute information in the second attribute information set is encoded using an encoding mode based on the second context model.

[0532] In one embodiment, the S bits of data include first encoded data and second encoded data; the first encoded data and the second encoded data are obtained by dividing the S bits of data based on at least one preset value.

[0533] In one embodiment, the processor 1002 performs entropy encoding on the attribute information of the point cloud data to obtain S bits of data. A specific example is as follows:

[0534] The attribute information of the point cloud data is binarized to obtain the binarized result;

[0535] Entropy encoding is performed on the binarization result to obtain S bits of data.

[0536] In one embodiment, the processor 1002 performs binarization processing on the attribute information of the point cloud data to obtain the binarization result. A specific example is as follows:

[0537] The attribute information of the point cloud data is shifted and moduloed to obtain the shift and modulo result;

[0538] The result of the shift and remainder processing is binarized to obtain the binarized result.

[0539] Based on the same inventive concept, the principle and beneficial effects of the coding device provided in the embodiments of this application in solving the problem are similar to the principle and beneficial effects of the point cloud processing method in the embodiments of this application in solving the problem. For the sake of brevity, the principle and beneficial effects of the method implementation can be referred to.

[0540] This application also provides a computer-readable storage medium storing one or more instructions, which are adapted to be loaded by a processor and executed by the point cloud processing method described in the above method embodiments.

[0541] This application also provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the point cloud processing method described in the above method embodiments.

[0542] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the point cloud processing method described above.

[0543] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0544] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0545] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0546] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that all or part of the processes for implementing the above embodiments and equivalent variations made in accordance with the claims of this application are still within the scope of this application.

Claims

1. A point cloud processing method, characterized in that, The method includes: Obtain the bitstream data of point cloud data; The bitstream data is parsed to obtain the grouping information of the point cloud data, and the grouping information is used to indicate the association relationship between the various groups corresponding to the point cloud data. The grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data; The point cloud data is presented according to its attribute information; In the point cloud data, points are divided into K groups, and each point in the i-th group is associated with R attributes. Each attribute corresponds to M transformation coefficients, and the M transformation coefficients corresponding to each attribute include one first transformation coefficient and M-1 second transformation coefficients. The method further includes: if the group travel length corresponding to the i-th group is not zero, then performing inter-group travel decoding on the transformation coefficients in the i-th group; if the group travel length corresponding to the i-th group is zero, then decoding the transformation coefficients in the i-th group one by one; or, directly decoding the first transformation coefficient in the i-th group; if the group travel length corresponding to the i-th group is not zero, then performing inter-group travel decoding on the second transformation coefficients in the i-th group; if the group travel length corresponding to the i-th group is zero, then decoding the second transformation coefficients in the i-th group one by one. Where R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K.

2. The method as described in claim 1, characterized in that, The points in the point cloud data are divided into K groups, and the points in the K groups are arranged in decoding order, where K is an integer greater than 1; the optimization includes optimizing the selection of prediction candidate points in the group prediction process; the optimization method includes at least one of the following: Add the target predicted point to the predicted candidate points of the (P+1)th group. The value of the target predicted point is the weighted average of the attribute reconstruction values ​​of the points in the previous P groups, and the coordinates of the target predicted point are the average geometric coordinates of the points in the previous P groups; or... Replace the points to be replaced in the predicted candidate points of group P+1 with the target predicted point; or... If the sum of the points in the first P groups is less than the neighbor number threshold, then the target prediction point is used as the prediction candidate point in the P+1 group.

3. The method as described in claim 1, characterized in that, The points in the point cloud data are divided into K groups, where K is an integer greater than 1; the optimization includes optimizing the acquisition of residual values ​​during the group prediction process, and the optimization of acquiring residual values ​​includes: Obtain the secondary prediction residual value and reference residual value corresponding to the i-th group, where i is a positive integer less than or equal to K; The residual value of each point in the i-th group is calculated based on the secondary prediction residual value and the reference residual value.

4. The method as described in claim 3, characterized in that, The K groups of points are arranged in decoding order. Obtaining the secondary prediction residual value and reference residual value corresponding to the i-th group includes: The bitstream data is parsed to obtain the secondary prediction residual value and reference residual value corresponding to the i-th group; or, The bitstream data is parsed to obtain the secondary prediction residual value corresponding to the i-th group, and the reference residual value corresponding to the i-th group is determined based on the residual value of at least one point in the first i groups.

5. The method as described in claim 4, characterized in that, Determining the reference residual value corresponding to the i-th group based on the residual value of at least one point in the first i-1 groups includes: The residual value at any point in the first i groups is determined as the reference residual value corresponding to the i-th group; or, Based on the residual values ​​of at least two points in the first i groups, calculate the reference residual value corresponding to the i-th group.

6. The method as described in claim 1, characterized in that, The points in the point cloud data are divided into K groups, where K is an integer greater than 1; the optimization includes optimizing the acquisition of transformation coefficient values ​​during the group prediction process, and the optimization of acquiring transformation coefficient values ​​includes: Obtain the quadratic transformation coefficient value and the reference transformation coefficient value corresponding to the i-th group, where i is a positive integer less than or equal to K; The transformation coefficient values ​​of each point in the i-th group are calculated based on the second transformation coefficient values ​​and the reference transformation coefficient values.

7. The method as described in claim 6, characterized in that, The K groups of points are arranged in decoding order. Obtaining the quadratic transform coefficient value and reference transform coefficient value corresponding to the i-th group includes: The bitstream data is parsed to obtain the second transform coefficient values ​​and reference transform coefficient values ​​corresponding to the i-th group; or, The bitstream data is parsed to obtain the secondary transform coefficient values ​​corresponding to the i-th group, and the reference transform coefficient values ​​corresponding to the i-th group are determined based on the coefficient values ​​of at least one point in the first i groups.

8. The method as described in claim 7, characterized in that, Determining the reference transformation coefficient value corresponding to the i-th group based on the coefficient value of at least one point in the first i groups includes: The transformation coefficient value of any point in the first i groups is determined as the reference transformation coefficient value of the i-th group; or, The reference transformation coefficient value of the i-th group is calculated based on the transformation coefficient values ​​of at least two points in the first i-th group.

9. The method as described in claim 1, characterized in that, The points in the point cloud data are divided into K groups. The attribute of the j-th point in the i-th group is composed of Q component attributes, and each component attribute corresponds to a transformation coefficient. Q is a positive integer, K is an integer greater than 1, i is a positive integer less than or equal to K, and j is a positive integer; the method further includes: The non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes are directly decoded; or, The non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes are subjected to shift-and-remainder decoding.

10. The method as described in claim 1, characterized in that, The bitstream data includes S bits of data, where S is a positive integer; the method further includes: Entropy decoding is performed on the S bits of data to obtain the attribute information of the point cloud data; or, Run-length decoding is performed on the S bits of data to obtain the attribute information of the point cloud data; The entropy decoding includes at least one of the following decoding modes: a context-based decoding mode and a bypass decoding mode; the attribute information of the point cloud data includes at least one of the following: the residual value of each point and the transformation coefficient of each point.

11. The method as described in claim 10, characterized in that, Entropy decoding includes a context-based decoding mode and a bypass decoding mode. The context-based decoding mode includes a decoding mode based on a first context model and a decoding mode based on a second context model. The S bits of data include first encoded data and second encoded data; the process of entropy decoding of the S bits of data includes: The first encoded data is decoded using a context-based decoding mode, and the second encoded data is decoded using a bypass decoding mode. or, The first encoded data is decoded using a decoding mode based on a first context model, and the second encoded data is decoded using a decoding mode based on a second context model.

12. The method as described in claim 11, characterized in that, The first encoded data and the second encoded data are obtained by dividing the S bits of data based on at least one preset value.

13. The method as described in claim 10, characterized in that, The entropy decoding of the S bits of data to obtain the attribute information of the point cloud data includes: Entropy decoding is performed on the S bits of data to obtain the binarization result; Based on the binarization result, the attribute information of the point cloud data is restored.

14. The method as described in claim 13, characterized in that, The process of restoring the attribute information of the point cloud data based on the binarization result includes: Based on the binarization result, the shift-and-remainder processing result is restored; Based on the result of the shift and remainder processing, the attribute information of the point cloud data is calculated.

15. The method according to any one of claims 1 to 14, further comprising: The points in the point cloud data are divided into K groups according to the space filling curve, where K is an integer greater than 1; If the number of points in the i-th group is greater than the first quantity threshold, then the points in the i-th group whose coordinate values ​​in the target direction belong to the numerical range are divided into a group. The target direction is the direction corresponding to the shortest side in the three-dimensional bounding box to which the points in the i-th group belong, and i is a positive integer less than or equal to K. If the number of points in the i-th group is odd, then the number of points in the i-th group is incremented or decremented by one. The coordinate arrangement order of the space filling curves is determined according to the three-dimensional bounding box to which the point cloud data belongs.

16. The method as described in claim 15, characterized in that, The K groups of points are arranged in coding order, and the method further includes: If the number of points in the i-th group is less than the second quantity threshold N, then the i-th group is merged with the adjacent groups of the i-th group to obtain a merged group. The second quantity threshold N is less than the first quantity threshold, and N is a positive integer. The merged group is divided into an average value to obtain the updated i-th group and the updated adjacent groups of the i-th group; or, the N points in the merged group are divided into the i-th group, and the other points in the merged group other than the N points are divided into the adjacent groups of the i-th group.

17. A point cloud processing method, characterized in that, The method includes: Obtain the point cloud data to be encoded; The points in the point cloud data are divided to obtain the grouping information of the point cloud data. The grouping information is used to indicate the association relationship between the various groups corresponding to the point cloud data. The grouping prediction process is optimized based on the correlation between each group to obtain the attribute information of the point cloud data; Based on the correlation between each group and the distribution characteristics of the attribute information of the point cloud data, the attribute information of the point cloud data is encoded to obtain the bitstream data of the point cloud data. Wherein, the attribute information of the point cloud data is transformation coefficients, the points in the point cloud data are divided into K groups, each point in the i-th group is associated with R attributes, each attribute corresponds to M transformation coefficients, and the M transformation coefficients corresponding to each attribute include 1 first transformation coefficient and M-1 second transformation coefficients, the process of encoding the attribute information of the point cloud data according to the association relationship between each group and the distribution characteristics of the attribute information of the point cloud data includes: The first transform coefficients in the i-th group are directly encoded, and the second transform coefficients in the i-th group are run-length encoded; or, If all the transformation coefficients in the i-th group are preset values, then the group travel length corresponding to the K-th group points is incremented by one; if at least one transformation coefficient in the i-th group has a value other than the preset value, then the group travel length corresponding to the K-th group points is set to zero, and the transformation coefficients in the i-th group are encoded one by one; or, The first transformation coefficient in the i-th group is directly encoded. If all the values ​​of the second transformation coefficients in the i-th group are preset values, the group travel length corresponding to the K-group point is incremented by one. If at least one of the second transformation coefficients in the i-th group has a value that is not a preset value, the group travel length corresponding to the K-group point is set to zero, and the second transformation coefficients in the i-th group are encoded one by one. Alternatively, The first transform coefficients in the i-th group are subjected to shift-and-remainder encoding, and the second transform coefficients in the i-th group are directly encoded. Where R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K.

18. The method as described in claim 17, characterized in that, In the point cloud data, the points are divided into K groups, the i-th group includes M points, each point is associated with R attributes, and each attribute corresponds to M transformation coefficients; The process of encoding the attribute information of the point cloud data based on the correlation between each group and the distribution characteristics of the attribute information of the point cloud data includes: If M is less than the third quantity threshold, then the non-zero transform coefficients in the i-th group are subjected to shift-and-remainder encoding. If M is greater than or equal to the third quantity threshold, then the non-zero transform coefficients in the i-th group are directly encoded; Where R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K.

19. The method as described in claim 17, characterized in that, In the point cloud data, the points are divided into K groups, and the attribute of the j-th point in the i-th group is composed of Q component attributes, with each component attribute corresponding to a transformation coefficient. The process of encoding the attribute information of the point cloud data based on the correlation between each group and the distribution characteristics of the attribute information of the point cloud data includes: Calculate the sum of the absolute values ​​of at least two component attributes among the Q component attributes to obtain the combined component attribute value; If the combined value of the component attributes is less than the numerical threshold, then the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes are directly encoded; If the combined value of the component attributes is greater than or equal to the numerical threshold, then the non-zero transform coefficients among the transform coefficients corresponding to the Q component attributes are subjected to shift-and-remainder encoding. Where K is an integer greater than 1, Q is a positive integer, i is a positive integer less than or equal to K, and j is a positive integer.

20. The method as described in claim 17, characterized in that, The attribute information of the point cloud data includes at least one of the following: the residual value of each point, and the transformation coefficient of each point; the process of encoding the attribute information of the point cloud data includes: The attribute information of the point cloud data is entropy encoded to obtain S bits of data, where S is a positive integer; or, Run-length encoding is performed on the attribute information of the point cloud data to obtain S bits of data; Entropy coding includes at least one of the following coding modes: context-based coding mode and bypass coding mode.

21. The method as described in claim 20, characterized in that, Entropy coding includes a context-based coding mode and a bypass coding mode; the context-based decoding mode includes a decoding mode based on a first context model and a decoding mode based on a second context model. The attribute information of the point cloud data includes a first set of attribute information and a second set of attribute information. The process of entropy encoding the attribute information of the point cloud data includes: The attribute information in the first attribute information set is encoded using a context-based encoding mode, and the attribute information in the second attribute information set is encoded using a bypass encoding mode. or, The attribute information in the first attribute information set is encoded using an encoding mode based on a first context model, and the attribute information in the second attribute information set is encoded using an encoding mode based on a second context model.

22. The method as described in claim 21, characterized in that, The S bits of data include first encoded data and second encoded data; the first encoded data and the second encoded data are obtained by dividing the S bits of data based on at least one preset value.

23. The method as described in claim 20, characterized in that, The entropy encoding of the attribute information of the point cloud data to obtain S bits of data includes: The attribute information of the point cloud data is binarized to obtain the binarization result; The binarization result is entropy encoded to obtain S bits of data.

24. The method as described in claim 23, characterized in that, The binarization of the attribute information of the point cloud data to obtain the binarization result includes: The attribute information of the point cloud data is subjected to shift and remainder processing to obtain the shift and remainder processing result; The shift remainder processing result is binarized to obtain the binarized result.

25. A point cloud processing device, characterized in that, The point cloud processing device includes: The acquisition unit is used to acquire the bitstream data of point cloud data. The processing unit is used to parse the bitstream data to obtain the grouping information of the point cloud data, and the grouping information is used to indicate the association relationship between the various groups corresponding to the point cloud data; And to optimize the grouping prediction process based on the correlation between each group, so as to obtain the attribute information of the point cloud data; And for presenting the point cloud data according to the attribute information of the point cloud data; The processing unit is further configured to: if the group travel length corresponding to the i-th group is not zero, perform inter-group travel decoding on the transform coefficients in the i-th group; if the group travel length corresponding to the i-th group is zero, perform sequential decoding on the transform coefficients in the i-th group; or, directly decode the first transform coefficient in the i-th group; if the group travel length corresponding to the i-th group is not zero, perform inter-group travel decoding on the second transform coefficient in the i-th group; if the group travel length corresponding to the i-th group is zero, perform sequential decoding on the second transform coefficient in the i-th group. In this context, the points in the point cloud data are divided into K groups. Each point in the i-th group is associated with R attributes, and each attribute corresponds to M transformation coefficients. The M transformation coefficients corresponding to each attribute include one first transformation coefficient and M-1 second transformation coefficients. R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K.

26. A point cloud processing device, characterized in that, The point cloud processing device includes: The acquisition unit is used to acquire point cloud data to be encoded. The processing unit is used to divide the points in the point cloud data to obtain the grouping information of the point cloud data, and the grouping information is used to indicate the association relationship between the various groups corresponding to the point cloud data. And to optimize the grouping prediction process based on the correlation between each group, so as to obtain the attribute information of the point cloud data; And to encode the attribute information of the point cloud data to obtain the bitstream data of the point cloud data; The processing unit is used to encode the attribute information of the point cloud data according to the correlation between each group and the distribution characteristics of the attribute information of the point cloud data, so as to obtain the bitstream data of the point cloud data. Wherein, the attribute information of the point cloud data is transformation coefficients, the points in the point cloud data are divided into K groups, each point in the i-th group is associated with R attributes, each attribute corresponds to M transformation coefficients, and the M transformation coefficients corresponding to each attribute include 1 first transformation coefficient and M-1 second transformation coefficients, the processing unit is used for The first transform coefficients in the i-th group are directly encoded, and the second transform coefficients in the i-th group are run-length encoded; or used for, If all the transformation coefficients in the i-th group are preset values, then the group travel length corresponding to the K-th group points is incremented by one; if at least one transformation coefficient in the i-th group has a value other than the preset value, then the group travel length corresponding to the K-th group points is set to zero, and the transformation coefficients in the i-th group are encoded one by one; or used for, The first transformation coefficient in the i-th group is directly encoded. If the values ​​of the second transformation coefficients in the i-th group are all preset values, the group travel length corresponding to the K-group point is incremented by one. If at least one of the second transformation coefficients in the i-th group has a value that is not a preset value, the group travel length corresponding to the K-group point is set to zero, and the second transformation coefficients in the i-th group are encoded one by one. Alternatively, it can be used for... The first transform coefficients in the i-th group are subjected to shift-and-remainder encoding, and the second transform coefficients in the i-th group are directly encoded. Where R is a positive integer, K and M are integers greater than 1, and i is a positive integer less than or equal to K.

27. A computer device, characterized in that, include: Storage devices and processors; A memory, wherein a computer program is stored; A processor is configured to load the computer program to implement the point cloud processing method as described in any one of claims 1-16; or to load the computer program to implement the point cloud processing method as described in any one of claims 17-24.

28. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as the point cloud processing method as claimed in any one of claims 1-16; or loaded and executed as the point cloud processing method as claimed in any one of claims 17-24.

29. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the point cloud processing method as described in any one of claims 1-16; or implement the point cloud processing method as described in any one of claims 17-24.

Images