A point cloud processing method, apparatus, device, storage medium, and product

Abstract

Embodiments of the present application provide a point cloud processing method, device, equipment, storage medium and product. The method comprises: when encoding processing is required for a to-be-encoded point in a point cloud, a restriction parameter of the point cloud is set, attribute information of the to-be-encoded point in the point cloud is obtained, the attribute information of the to-be-encoded point is encoded, and the encoding process is controlled according to the restriction parameter; when decoding processing is required for a to-be-decoded point in the point cloud, an encoding code stream of the to-be-decoded point in the point cloud and the restriction parameter of the point cloud are obtained, the encoding code stream of the to-be-decoded point is decoded, and the decoding process is controlled according to the restriction parameter. The restriction parameter of the point cloud makes the encoding and decoding process of the attribute information of each point in the point cloud controllable, thereby improving the encoding and decoding performance of the attribute information of the point cloud.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to the field of coding and decoding technology, and in particular to a point cloud processing method, a point cloud processing device, a computer device, a computer-readable storage medium, and a point cloud processing product. Background Art

[0002] With the continuous development of science and technology, it is now possible to obtain a large number of high-precision point clouds at a low cost and in a short period of time. The point cloud can include multiple points, and each point in the point cloud has geometric information and attribute information. In order to improve the transmission efficiency of the point cloud, it is usually necessary to encode the point cloud before transmitting it; specifically, after the encoding end encodes the geometric information and attribute information of each point in the point cloud, it can transmit the encoded point cloud to the decoding end, and the decoding end can decode the encoded point cloud to reconstruct the geometric information and attribute information of each point in the point cloud. Practice has found that because the encoding process requires the attribute information of each point to be sorted, the waiting time for the attribute information of each point in the point cloud is long during encoding, and the encoding performance is low. Summary of the Invention

[0003] The embodiments of the present application provide a point cloud processing method and related equipment, which can improve the encoding performance of point cloud attribute information.

[0004] In one aspect, an embodiment of the present application provides a point cloud processing method, the point cloud processing method comprising:

[0005] Set the limit parameters of the point cloud, which include one or both of cache limit parameters and delay limit parameters;

[0006] Get the attribute information of the points to be coded in the point cloud;

[0007] Encoding attribute information of the code point; and

[0008] The encoding process is controlled according to the constraint parameters.

[0009] In the embodiment of the present application, when it is necessary to encode the points to be encoded in the point cloud, it is possible to set point cloud restriction parameters, obtain attribute information of the points to be encoded in the point cloud, encode the attribute information of the points to be encoded, and control the encoding process according to the restriction parameters. By setting point cloud restriction parameters, the embodiment of the present application makes the encoding process of the attribute information of the points to be encoded controllable, thereby improving the encoding performance of the point cloud attribute information.

[0010] On the one hand, an embodiment of the present application provides another point cloud processing method, which includes:

[0011] Get the encoded code stream of the point to be decoded in the point cloud;

[0012] Obtaining restriction parameters of the point cloud, where the restriction parameters include one or both of a cache restriction parameter and a delay restriction parameter;

[0013] Decoding the coded code stream of the decoding point; and

[0014] The decoding process is controlled according to the constraint parameters.

[0015] In the embodiment of the present application, when decoding a point to be decoded in a point cloud, the encoding stream of the point to be decoded and the point cloud's restriction parameters can be obtained, the encoding stream of the point to be decoded can be decoded, and the decoding process can be controlled according to the restriction parameters. By obtaining the point cloud's restriction parameters, the embodiment of the present application makes the decoding process of the attribute information of the point to be decoded controllable, thereby improving the decoding performance of the point cloud's attribute information.

[0016] In one aspect, an embodiment of the present application provides a point cloud processing device, comprising:

[0017] A setting unit is used to set the limit parameters of the point cloud, and the limit parameters include one or both of a cache limit parameter and a delay limit parameter;

[0018] An acquisition unit, used to obtain attribute information of the points to be coded in the point cloud;

[0019] The processing unit is used to encode the attribute information of the coding point; and control the encoding process according to the restriction parameters.

[0020] In one aspect, an embodiment of the present application provides another point cloud processing device, the point cloud processing device comprising:

[0021] an acquisition unit, configured to acquire a coded stream of a point to be decoded in the point cloud, and acquire restriction parameters of the point cloud, wherein the restriction parameters include one or both of a buffer restriction parameter and a delay restriction parameter;

[0022] The processing unit is configured to decode the coded code stream of the decoding point; and control the decoding process according to the restriction parameters.

[0023] Accordingly, an embodiment of the present application provides a computer device, which includes a processor and a computer-readable storage medium; wherein the processor is suitable for implementing a computer program, the computer-readable storage medium stores the computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned point cloud processing method.

[0024] Accordingly, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned point cloud processing method.

[0025] Accordingly, embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described point cloud processing method.

[0026] In an embodiment of the present application, when it is necessary to encode the points to be encoded in the point cloud, it is possible to set restriction parameters of the point cloud, obtain the attribute information of the points to be encoded in the point cloud, encode the attribute information of the points to be encoded, and control the encoding process according to the restriction parameters. When it is necessary to decode the points to be decoded in the point cloud, it is possible to obtain the encoding code stream of the points to be decoded in the point cloud and the restriction parameters of the point cloud, decode the encoding code stream of the points to be decoded, and control the decoding process according to the restriction parameters. The embodiment of the present application makes the encoding and decoding process of the attribute information of each point in the point cloud controllable by the restriction parameters of the point cloud, thereby improving the encoding and decoding performance of the point cloud attribute information. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1a A schematic diagram of a coding framework provided in an embodiment of the present application;

[0029] Figure 1b A schematic diagram of the relationship between the block where the current point is located and its parent block neighbors provided in an embodiment of the present application;

[0030] Figure 2 An architectural diagram of a point cloud processing system provided in an embodiment of the present application;

[0031] Figure 3 A point cloud processing method provided in an embodiment of the present application;

[0032] Figure 4 Another point cloud processing method provided in an embodiment of the present application;

[0033] Figure 5 Another point cloud processing method provided in an embodiment of the present application;

[0034] Figure 6 Another point cloud processing method provided in the embodiment of the present application;

[0035] Figure 7 A schematic diagram of the structure of a point cloud processing device provided in an embodiment of the present application;

[0036] Figure 8 A schematic diagram of the structure of another point cloud processing device provided in an embodiment of the present application;

[0037] Figure 9 A schematic diagram of the structure of an encoding device provided in an embodiment of the present application;

[0038] Figure 10 A schematic diagram of the structure of a decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] In order to more clearly understand the technical solutions provided by the embodiments of the present application, the key terms involved in the embodiments of the present application are first introduced here:

[0041] (1) Point Cloud. A point cloud is a set of irregularly distributed discrete points in space that express the spatial structure and surface properties of a three-dimensional object or scene. Point clouds can be divided into different categories based on different classification criteria. For example, based on the acquisition method of the point cloud, they can be divided into dense point clouds and sparse point clouds. For another example, based on the time series type of the point cloud, they can be divided into static point clouds and dynamic point clouds.

[0042] (2) Point Cloud Data. The geometric information and attribute information of each point in the point cloud together constitute point cloud data. Geometric information can also be called three-dimensional position information. The geometric information of a point in the point cloud refers to the spatial coordinates (x, y, z) of the point, which can include the coordinate values ​​of the point in each coordinate axis direction of the three-dimensional coordinate system, for example, the coordinate value x in the X-axis direction, the coordinate value y in the Y-axis direction, and the coordinate value z in the Z-axis direction. The attribute information of a point in the point cloud can include at least one of the following: color information, material information, laser reflection intensity information (also called reflectivity); usually, each point in the point cloud has the same amount of attribute information. For example, each point in the point cloud can have two types of attribute information: color information and laser reflection intensity. For another example, each point in the point cloud can have three types of attribute information: color information, material information, and laser reflection intensity information.

[0043] (3) Point Cloud Compression (PCC). Point cloud coding refers to the process of encoding the geometric information and attribute information of each point in the point cloud to obtain a compressed code stream. Point cloud coding can include two main processes: geometric information encoding and attribute information encoding. The current mainstream point cloud coding technology can be divided into geometric structure-based point cloud coding and projection-based point cloud coding according to different types of point clouds. Here, G-PCC (Geometry-based Point Cloud Compression) in MPEG (Moving Picture Expert Group, international audio and video coding standard) and AVS-PCC in AVS (Audio Video Coding Standard, China's national video coding standard) are used as examples to introduce them.

[0044] The coding frameworks of G-PCC and AVS-PCC are roughly the same, such as Figure 1a As shown in Figure 1, the process can be divided into the geometric information encoding process and the attribute information encoding process. The geometric information encoding process encodes the geometric information of each point in the point cloud data to obtain a geometric bit stream; the attribute information encoding process encodes the attribute information of each point in the point cloud data to obtain an attribute bit stream. The geometric bit stream and the attribute bit stream together constitute the compressed code stream of the point cloud data.

[0045] For the geometric information encoding process, the main operations and processing can be seen in the following description:

[0046] ① Pre-processing: This can include coordinate transformation and voxelization. Through scaling and translation operations, the point cloud data in 3D space is converted into integer form and its minimum geometric position is moved to the coordinate origin.

[0047] ② Geometric Octree Encoding: An octree is a tree-like data structure that evenly divides a pre-defined bounding box into three-dimensional spaces, with each node having eight children. By using "1" and "0" to indicate whether each child node in the octree is occupied, an occupancy code is generated as a bitstream of point cloud geometry information. Bounding box encoding is an algorithm for finding the optimal bounding space for a discrete set of points. The basic idea is to approximate complex geometric objects with a slightly larger geometric object with simpler characteristics (called a bounding box).

[0048] ③ Geometry Entropy Encoding: Statistical compression encoding is performed on the octtree's occupancy code information, ultimately outputting a binary (0 or 1) compressed bitstream. Statistical coding is a lossless encoding method that can effectively reduce the bitrate required to express the same signal. A commonly used statistical coding method is context-based binary arithmetic coding (CABAC).

[0049] For the attribute information encoding process, the main operations and processing can be found in the following description:

[0050] 1. Attribute Recoloring: In lossy encoding, after the geometric information is encoded, the encoder needs to decode and reconstruct the geometric information, that is, restore the geometric information of each point in the point cloud. The attribute information corresponding to one or more neighboring points in the original point cloud is found and used as the attribute information of the reconstructed point.

[0051] ② Attribute transform (Transform attributes) encoding: Use attribute transform algorithms (such as DCT, Haar, etc.) to group and transform attribute information and quantize the transform coefficients; obtain the attribute reconstruction value through inverse quantization and inverse transformation; calculate the difference between the original attribute and the attribute reconstruction value to obtain the attribute residual and quantize it; encode the quantized transform coefficients and attribute residual.

[0052] ③ Attribute Quantization: The degree of quantization is usually determined by the quantization parameter. In attribute prediction coding, entropy coding is performed on the quantized prediction residual information; in attribute transform coding and attribute prediction transform coding, entropy coding is performed on the quantized transform coefficients.

[0053] 4. Attribute Entropy Coding: Quantized prediction residual information or transform coefficients are typically compressed using run-length coding and arithmetic coding. The corresponding coding mode, quantization parameters, and other information are also encoded using an entropy encoder.

[0054] (4) Point cloud decoding. Point cloud decoding refers to the process of decoding the compressed bitstream obtained by point cloud encoding to reconstruct the point cloud; in detail, it refers to the process of reconstructing the geometric information and attribute information of each point in the point cloud based on the geometric bitstream and attribute bitstream in the compressed bitstream. After obtaining the compressed bitstream at the decoding end, the geometric bitstream is first entropy decoded to obtain the quantized geometric information of each point in the point cloud, and then dequantized to reconstruct the geometric information of each point in the point cloud. As for the attribute bitstream, entropy decoding is first performed to obtain the quantized prediction residual information or quantized transform coefficients of each point in the point cloud; then the quantized prediction residual information is dequantized to obtain the reconstructed residual information, and the quantized transform coefficients are dequantized to obtain the reconstructed transform coefficients. The reconstructed transform coefficients are inversely transformed to obtain the reconstructed residual information. Based on the reconstructed residual information of each point in the point cloud, the attribute information of each point in the point cloud can be reconstructed. The reconstructed attribute information of each point in the point cloud is matched one by one with the reconstructed geometric information in order to reconstruct the point cloud.

[0055] In addition, this application also involves the following technologies:

[0056] (1) Attribute prediction:

[0057] ① Point cloud reordering, the specific process is as follows:

[0058] Get the coordinates (x, y, z) of the point cloud, generate the Morton code or Hilbert code of each point based on the space filling curve, and sort them according to the encoding order of each point (such as sorting from small to large) to obtain the Morton order or Hilbert order. Taking the Morton code of geometric coordinates as an example, the geometric position of the point in the point cloud is represented by three-dimensional Cartesian coordinates (X, Y, Z). Use N bits to represent each coordinate value, and the coordinate of the k-th point (X k ,Y k ,Z k ) can be expressed as:

[0059]

[0060] The Morton code corresponding to the k-th point can be expressed as follows:

[0061]

[0062] Represent each three bits in octal. n=0,1,…,N-1, then the Morton code corresponding to the k-th point can be expressed as:

[0063]

[0064] ②Sequence-based neighbor point selection, the specific process is as follows:

[0065] Sequence-based neighbor point selection includes distance-based selection and spatial relationship and distance-based selection. The following describes the above methods:

[0066] i) Distance-based method, the specific steps are as follows:

[0067] Among the first P neighbor candidate points in the Morton order or Hilbert order, P can be determined by the maximum number of neighbors field (maxNumOfNeighbours), and each neighbor candidate point is calculated (the i-th neighbor candidate point is represented by (x i ,y i ,z i )), the Manhattan distance d to the current decoding point (x, y, z) = |xx i |+|yy i |+|zz i After obtaining the Manhattan distance from each neighbor candidate point to the current point to be decoded (x, y, z), the k neighbor candidate points with the shortest Manhattan distances can be used as neighbors of the current point to be decoded; alternatively, the maximum distance value among the k points with the shortest Manhattan distances is determined, and all neighbor candidate points with a Manhattan distance less than or equal to the maximum distance value are used as neighbors of the current point. Finally, the m points with the closest Manhattan distances are determined as the nearest neighbors of the current point to be decoded, where m and k are positive integers and m≤k.

[0068] ii) Selection method based on spatial relationship and distance. The specific steps are as follows:

[0069] First, determine the initial block size, that is, determine the initial right shift number N of the codeword corresponding to each point. Based on this, N+3 is the corresponding parent block range.

[0070] Then, the point cloud is traversed in a certain order, such as Figure 1bAs shown, the current point to be decoded (point P) performs a nearest neighbor search among the decoded points (limited to the range of the first k points) within the parent block of the B block where it is located and the neighboring blocks that are coplanar, colinear, and co-point with it.

[0071] If the number of neighbor points of the current point to be decoded determined by the above method is insufficient, the neighbor point selection is performed according to the distance-based method in i).

[0072] ③ Calculation of predicted value, the specific process is as follows:

[0073] Let the attribute reconstruction value of each neighbor be j=0,1…k, then the attribute prediction value of the current point for:

[0074]

[0075] i) Weighted calculation based on distance:

[0076] In one embodiment, the inverse of the Manhattan distance between the current point to be decoded and its neighboring points is used as the weight, and finally the weighted average of the attribute reconstruction values ​​of the k neighboring points is calculated to obtain the attribute prediction value. Assume that the geometric coordinates of the current point to be decoded are (x i ,y i , z i ), the geometric coordinates of the jth neighbor point are (x ij ,y ij , z ij ), then the weight w of the j-th neighbor point ij for:

[0077]

[0078] In another embodiment, if the components in the x, y, and z directions use different weights, the weight w of the jth neighbor point is ij for:

[0079]

[0080] ii) Weighted calculation based on distance and other parameters:

[0081] The weight of each neighbor point of the current point to be decoded is w = 1 / d, and the optimized weight of the neighbor candidate point whose distance is equal to the maximum distance value is wm = (1 / d) * dwm, where the size of dwm is the minimum value between Qstep (attribute quantization step) and the number of neighbor candidate points whose distance is equal to the maximum distance value.

[0082] (2) Attribute transformation:

[0083] The core of attribute transformation is attribute encoding that combines prediction and transformation. The specific process is as follows:

[0084] First, the point cloud to be encoded, which has undergone color space conversion and recoloring, is reordered from small to large according to the Hilbert code, and then adaptive grouping and corresponding K-ary transformation are used. The specific process is as follows:

[0085] i) Point cloud grouping:

[0086] The point cloud is grouped sequentially based on the order of the space-filling curve, with points whose Hilbert code shares the same first L bits grouped together. When the number of points in a group exceeds a preset limit, the group is further refined. Furthermore, the grouping can be adjusted based on the number of points in the previous group.

[0087] ii) Intra-group prediction:

[0088] The prediction point selection range is the P points preceding the first point in the current group, where P is determined by the maximum number of neighbors field (maxNumOfNeighbours). Predictions are performed on these P points one by one, and a unified prediction value is selected or calculated for each group. For example, the prediction value of the first point in the current group is used as the prediction value for the current group.

[0089] iii) Based on the residual value transformation:

[0090] The predicted attribute residual values ​​of each group of points undergo a K_i-element discrete cosine transform (DCT), where K_i = 2…8, and the transform coefficients are quantized and entropy coded. When K_i = 1, no transform calculation is performed and the attribute residual values ​​are directly quantized and entropy coded. The resulting transform coefficients are quantized.

[0091] (3) Entropy coding:

[0092] Binarize and process the signed attribute prediction residual or transform coefficient after quantization (lossy case). Entropy coding can adopt variable length coding, context-based adaptive binary arithmetic coding, run length coding and other methods. Specifically:

[0093] ①Variable length coding:

[0094] Codewords of varying lengths are used to represent the residuals or coefficients to be encoded. The code length is designed based on the probability of symbol occurrence. Commonly used variable-length codes include exponential Golomb coding and arithmetic coding.

[0095] ②Context-based adaptive binary arithmetic coding (CABAC), which mainly includes the following steps:

[0096] Binarization: CABAC uses binary arithmetic coding, meaning only two digits (1 or 0) are encoded. A non-binary numerical symbol, such as a transform coefficient or motion vector, is first binarized, or converted into a binary codeword, before arithmetic coding. This process is similar to converting a numerical value into a variable-length codeword. In practice, this binary codeword is further encoded using an arithmetic encoder before transmission.

[0097] Context model selection: The context model is a probabilistic model that is selected based on the statistics of the most recent coded data symbols. This model stores the probability of each 'bin' being 1 or 0.

[0098] Arithmetic coding: The arithmetic encoder encodes each 'bin' according to the selected probability model.

[0099] Probability update: The selected context model will be updated according to the actual encoding value. For example, if the value of 'bin' is 1, the frequency count of 1 will be increased.

[0100] ③Run-length coding:

[0101] The number of consecutive points in the statistics that are a specific symbol is recorded as the run length. If the current symbol is zero, the run length value is increased by 1. If it is non-zero, the run length value is first encoded, then the non-zero attribute prediction residual is encoded, and finally the run length value is set to 0 and the count starts again. The residual can be encoded using common entropy coding methods such as variable length coding or CABAC.

[0102] Specifically, run-length encoding is categorized into fixed-length run-length encoding and variable-length run-length encoding. Fixed-length run-length encoding uses a fixed number of binary bits for each run. Variable-length run-length encoding uses different numbers of binary bits for different run-length ranges, requiring a flag bit to indicate the number of binary bits used.

[0103] Based on the above descriptions of point clouds, point cloud data, point cloud encoding, point cloud decoding, attribute prediction, attribute transformation, and entropy coding, an embodiment of the present application provides a point cloud processing solution. When it is necessary to encode the points to be encoded in the point cloud, the restriction parameters of the point cloud can be set, the attribute information of the points to be encoded in the point cloud can be obtained, the attribute information of the points to be encoded can be encoded, and the encoding process can be controlled according to the restriction parameters. When it is necessary to decode the points to be decoded in the point cloud, the encoding code stream of the points to be decoded in the point cloud and the restriction parameters of the point cloud can be obtained, the encoding code stream of the points to be decoded can be decoded, and the decoding process can be controlled according to the restriction parameters. The embodiment of the present application makes the encoding and decoding process of the attribute information of each point in the point cloud controllable through the restriction parameters of the point cloud, thereby improving the encoding and decoding performance of the point cloud attribute information.

[0104] The point cloud processing solution provided in the embodiments of the present application can also be combined with cloud computing, cloud storage, and other cloud technologies. Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. Cloud computing can provide powerful computing support for the encoding and decoding stages of point cloud attributes, which can greatly improve the encoding efficiency and decoding efficiency of point cloud attributes. Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (also known as storage nodes) in the network through application software or application interfaces to work together and provide external data storage and service access functions. Cloud storage can provide powerful storage support for the encoding and decoding stages of point cloud attributes, which can further improve the encoding efficiency and decoding efficiency of point cloud attributes.

[0105] In addition, the point cloud processing solution provided in the embodiment of the present application can also be combined with artificial intelligence. Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0106] AI technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, large-scale application processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0107] Machine learning is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of AI and the fundamental way to make computers intelligent. Its applications span all areas of artificial intelligence. Machine learning / deep learning typically includes techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0108] Deep learning: The concept of deep learning originates from the study of artificial neural networks. A multilayer perceptron with multiple hidden layers is an example of a deep learning architecture. Deep learning discovers distributed feature representations of data by combining lower-level features to form more abstract higher-level representations of attribute categories or features. The embodiments of this application primarily involve optimizing and training the models used in the encoding and decoding process using training datasets. For example, an initial model can be trained using a training dataset to obtain a context model.

[0109] Based on the above description, the following Figure 2 The point cloud processing system suitable for implementing the point cloud processing solution provided in the embodiment of the present application is introduced. Figure 2 As shown, the point cloud processing system 20 may include an encoding device 201 and a decoding device 202. The encoding device 201 may be a terminal or a server, and the decoding device 202 may be a terminal or a server. A communication connection may be established between the encoding device 201 and the decoding device 202. The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, a smart TV, etc., but is not limited thereto. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0110] (1) For the encoding device 201:

[0111] The encoding device 201 can acquire point cloud data (i.e., the geometric and attribute information of each point in the point cloud). Point cloud data can be acquired through scene capture or device generation. Scene capture point cloud data refers to point cloud data obtained by capturing a real-world visual scene using a capture device associated with the encoding device 201. The capture device is used to provide point cloud data acquisition services for the encoding device 201. The capture device may include, but is not limited to, any of the following: a camera device, a sensor device, or a scanning device. Camera devices may include ordinary cameras, stereo cameras, and light field cameras, etc., sensor devices may include laser devices, radar devices, etc., and scanning devices may include 3D laser scanning devices, etc. The capture device associated with the encoding device 201 may refer to a hardware component provided in the encoding device 201, such as a terminal camera or sensor, etc., or a hardware device connected to the encoding device 201, such as a camera connected to a server. Device-generated point cloud data refers to point cloud data generated by the encoding device 201 based on virtual objects (e.g., virtual 3D objects and virtual 3D scenes obtained through 3D modeling).

[0112] The encoding device 201 can encode the geometric information of each point in the point cloud to obtain a geometry bitstream (containing the geometric encoding information of each point in the point cloud), and can also encode the attribute information of each point in the point cloud to obtain an attribute bitstream (containing the attribute encoding information of each point in the point cloud). In particular, when encoding the attribute information of the points to be encoded in the point cloud, the encoding device 201 can set point cloud restriction parameters and encode the acquired attribute information of the points to be encoded based on the point cloud restriction parameters, thereby controlling the encoding process. After obtaining the compressed bitstream of the point cloud, the encoding device 201 transmits the compressed bitstream (i.e., the attribute bitstream and the geometry bitstream) to the decoding device 202.

[0113] (2) For the decoding device 202:

[0114] After receiving the compressed code stream (i.e., the attribute bit stream and the geometry bit stream) transmitted by the encoding device 201, the decoding device 202 can decode the geometry bit stream to reconstruct the geometric information of each point in the point cloud, and can also decode the attribute bit stream to obtain the attribute information of each point in the point cloud. In particular, when decoding the points to be decoded in the point cloud, the decoding device 202 can obtain the restriction parameters of the point cloud and decode the encoded code stream of the obtained points to be decoded based on the restriction parameters of the point cloud, thereby controlling the decoding process. After obtaining the reconstructed geometry information and reconstructed attribute information of each point in the point cloud, the decoding device 202 associates the reconstructed geometry information and the reconstructed attribute information of each point in the point cloud one-to-one to reconstruct the point cloud.

[0115] In the embodiment of the present application, when it is necessary to encode the points to be encoded in the point cloud, the restriction parameters of the point cloud can be set, the attribute information of the points to be encoded in the point cloud can be obtained, the attribute information of the points to be encoded can be encoded, and the encoding process can be controlled according to the restriction parameters. When it is necessary to decode the points to be decoded in the point cloud, the encoding code stream of the points to be decoded in the point cloud and the restriction parameters of the point cloud can be obtained, the encoding code stream of the points to be decoded can be decoded, and the decoding process can be controlled according to the restriction parameters. The embodiment of the present application makes the encoding and decoding process of the attribute information of each point in the point cloud controllable through the restriction parameters of the point cloud, thereby improving the encoding and decoding performance of the point cloud attribute information. It can be understood that the point cloud processing system described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. It can be known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0116] Based on the above description of the point cloud processing system, the point cloud processing solution provided in the embodiment of the present application is introduced in more detail below with reference to the accompanying drawings.

[0117] See also Figure 3 , Figure 3 A point cloud processing method provided in an embodiment of the present application can be executed by a computer device, which can be Figure 2 The encoding device 201 in the point cloud processing system 20 is shown. Figure 3 As shown, the point cloud processing method may include the following steps S301 to S303:

[0118] S301: Set the restriction parameters of the point cloud.

[0119] The restriction parameters are used to set trigger conditions for encoding the attribute information of the points to be encoded in the point cloud. The restriction parameters may include at least one of a cache restriction parameter and a delay restriction parameter. The cache restriction parameter is used to control the cache logic during the encoding process. For example, when the number of intermediate coefficients corresponding to the attribute information of the points to be encoded in the cache queue (such as the residual, transform coefficient, etc. corresponding to the attribute information of the points to be encoded) meets the trigger condition set based on the cache restriction parameter (such as the number of intermediate coefficients corresponding to the attribute information of the points to be encoded in the cache queue reaches the cache threshold indicated by the cache parameter), the computer device (encoding device) encodes the intermediate coefficients corresponding to the attribute information of the points to be encoded in the cache queue. Similarly, the delay restriction parameter is used to control the encoding logic during the encoding process. For example, when the delayed number of intermediate coefficients corresponding to the attribute information of the points to be encoded meets the trigger condition set based on the delay restriction parameter (such as the delayed number of intermediate coefficients corresponding to the attribute information of the points to be encoded reaches the delay threshold indicated by the delay parameter), the computer device (encoding device) encodes the intermediate coefficients corresponding to the attribute information of the points to be encoded. The delay number indicates the number of intermediate coefficients corresponding to the attribute information of the current point to be encoded.

[0120] The setting methods of the restriction parameters include any one or more of the following: setting the restriction parameters to default values; setting the restriction parameters to fixed values ​​and encapsulating the set restriction parameters into the encoding code stream formed by encoding; setting the restriction parameters to variable values ​​and encapsulating the set restriction parameters into the encoding code stream formed by encoding, for example, setting the restriction parameters to a in encoding code stream 1, setting the restriction parameters to b in encoding code stream 2, and setting the restriction parameters to c in encoding code stream 3, and a, b, and c are different from each other; dynamically adjusting the value of the restriction parameters during the encoding process and encapsulating the dynamically adjusted restriction parameters into the encoding code stream formed by encoding; taking the encoding as run-length encoding as an example, the computer device can count the lengths of the first Q runs, and dynamically adjust the value of the restriction parameters according to the lengths of the first Q runs, and encapsulate the dynamically adjusted restriction parameters into the encoding code stream formed by encoding. It is understandable that if the restriction parameters are not set, the restriction parameters are the default values.

[0121] S302: Obtain attribute information of the points to be coded in the point cloud.

[0122] The attribute information of the point to be coded can include at least one of the following: color information, material information, and laser reflection intensity information (also known as reflectivity). Typically, each point in a point cloud has the same amount of attribute information. For example, each point in a point cloud can have two types of attribute information: color information and laser reflection intensity information. Another example is that each point in a point cloud can have three types of attribute information: color information, material information, and laser reflection intensity information. The attribute information of the point to be coded can be generated based on data collected by an acquisition device (such as a camera or 3D laser scanner), or it can be generated directly by a computer based on virtual three-dimensional objects and scenes.

[0123] S303: Encode the attribute information of the point to be encoded, and control the encoding process according to the restriction parameters.

[0124] Methods for encoding the attribute information of a point to be coded include: encoding the attribute information of the point to be coded by point number, and encoding the attribute information of the point to be coded by group number. Encoding by point number means encoding a point in the point cloud as the basic encoding unit; encoding by group number means encoding a group in the point cloud (a group can include at least one point in the point cloud) as the basic encoding unit.

[0125] In one embodiment, a computer device encodes the attribute information of a point to be encoded in units of points as follows: converting the attribute information of the point to be encoded to obtain an intermediate coefficient of the point to be encoded; wherein, converting the attribute information of the point to be encoded may be: predicting the attribute information of the point to be encoded to obtain a residual between the predicted information of the point to be encoded and the attribute information of the point to be encoded; or it may be: after obtaining the residual, transforming the residual to obtain a transformation coefficient; that is, converting the attribute information of the point to be encoded to obtain an intermediate coefficient of the point to be encoded may be a residual or a transformation coefficient.

[0126] After obtaining the intermediate coefficients of the points to be encoded, the computer device can write the intermediate coefficients of the points to be encoded into the cache, and read the intermediate coefficients of the points to be encoded from the cache for encoding; for example, the computer device can write the intermediate coefficients of the points to be encoded into a cache queue set based on the cache limit parameters. When the cache queue is full (such as the intermediate coefficients of the points to be encoded stored in the cache queue reach the cache threshold), the computer device reads the intermediate coefficients of the points to be encoded from the cache queue for encoding. The computer device can also obtain the delayed number of the intermediate coefficients of the current point to be encoded. If the delayed number of the intermediate coefficients of the current point to be encoded reaches the delay threshold, the computer device encodes the intermediate coefficients of the current point to be encoded. The specific implementation method of the computer device encoding the intermediate coefficients of the points to be encoded can refer to the above-mentioned implementation method of point cloud encoding, which will not be repeated here.

[0127] In another embodiment, the computer device encodes the attribute information of the points to be coded in units of groups as follows: the attribute information of the points to be coded is grouped to obtain multiple point cloud groups. The basis for grouping can be set according to the actual coding requirements. In one embodiment, the computer device can group the attribute information of the points to be coded based on the order of the space filling curve; for example, the computer device can group the attribute information of the points to be coded with the same first L bits of the Hilbert code into one group. After obtaining multiple groups of point cloud groups, the computer device can convert the attribute information of the points to be coded in the point cloud groups to obtain converted point cloud groups; the specific implementation method of converting the attribute information of the points to be coded in the point cloud groups can refer to the previous embodiment and will not be repeated here. The converted point cloud groups contain intermediate coefficients of the points to be coded, and the intermediate coefficients include residuals or transformation coefficients.

[0128] After obtaining the converted point cloud grouping, the computer device can write the converted point cloud grouping into the cache, and sequentially read the intermediate coefficients of the points to be encoded in the converted point cloud grouping from the cache for encoding; for example, the computer device can write the converted point cloud grouping into a cache queue set based on the cache limit parameters. When the cache queue is full (such as the converted point cloud groups stored in the cache queue reach the cache threshold), the computer device reads the converted point cloud grouping from the cache queue for encoding. The computer device can also obtain the delay number of the current converted point cloud grouping. If the delay number of the current converted point cloud grouping reaches the delay threshold, the computer device encodes the current converted point cloud grouping. The specific implementation method of the computer device encoding the converted point cloud grouping can refer to the above-mentioned implementation method of point cloud encoding, which will not be repeated here.

[0129] In the embodiment of the present application, when it is necessary to encode the points to be encoded in the point cloud, it is possible to set point cloud restriction parameters, obtain attribute information of the points to be encoded in the point cloud, encode the attribute information of the points to be encoded, and control the encoding process according to the restriction parameters. By setting point cloud restriction parameters, the embodiment of the present application makes the encoding process of the attribute information of the points to be encoded controllable, thereby improving the encoding performance of the point cloud attribute information.

[0130] See also Figure 4 , Figure 4 Another point cloud processing method provided in an embodiment of the present application can be executed by a computer device, which can be Figure 2 The encoding device 201 in the point cloud processing system 20 is shown. Figure 4 As shown, the point cloud processing method may include the following steps S401 to S405:

[0131] S401: Set the restriction parameters of the point cloud.

[0132] The restriction parameters include one or both of a buffer restriction parameter (coeffLengthControl) and a delay restriction parameter (maxNumofCoeff). The restriction parameters are used to determine the encoding restriction point number threshold.

[0133] In one embodiment, when the restriction parameter includes only one parameter, the unit of the restriction parameter includes any of the following: in points, in groups. Restriction in points means limiting the number of points to be encoded when encoding is performed in points; restriction in groups means limiting the number of groups to be encoded when encoding is performed in groups. When the restriction parameter is a cache restriction parameter, the encoding restriction point threshold is the cache restriction point threshold; the unit of the cache restriction parameter can be points or groups. Similarly, when the restriction parameter is a delay restriction parameter, the encoding restriction point threshold is the delay restriction point threshold; the unit of the delay restriction parameter can be points or groups. Specifically:

[0134] If the restriction parameter is in points and the value of the restriction parameter is X1, then the coding restriction point threshold is the actual total number of intermediate coefficients of the X1 points to be encoded; for example, assuming that the restriction parameter is a cache restriction parameter, in points, and the value of the cache restriction parameter is 1024, then the coding restriction point threshold is the actual total number of intermediate coefficients of the 1024 points to be encoded; for another example, assuming that the restriction parameter is a delay restriction parameter, in points, and the value of the delay restriction parameter is 2048, then the coding restriction point threshold is the actual total number of intermediate coefficients of the 2048 points to be encoded.

[0135] If the cache limit parameter is in groups and the value of the limit parameter is X2, then the encoding limit point threshold is the actual total number of intermediate coefficients of the points to be encoded in the X2 groups of point cloud groups; for example, assuming that the limit parameter is the cache limit parameter, in groups, and the value of the cache limit parameter is 64, then the encoding limit point threshold is the actual total number of intermediate coefficients of the points to be encoded in the 64 groups of point cloud groups; assuming that the limit parameter is the delay limit parameter, in groups, and the value of the delay limit parameter is 128, then the encoding limit point threshold is the actual total number of intermediate coefficients of the points to be encoded in the 128 groups of point cloud groups.

[0136] If the limit parameter is in groups, the value is X3, and the number of points in each point cloud group is limited to X4, then the encoding limit point threshold is the total number of intermediate coefficients of the points to be encoded in the X3 point cloud groups, and the total number does not exceed the value of X3 * X4. For example, assuming the limit parameter is a cache limit parameter, in groups, the value is 64, and the number of points in each point cloud group is 4, then the encoding limit point threshold is the actual total number of intermediate coefficients of the points to be encoded in the 64 point cloud groups, and the total number does not exceed 64 * 4 = 256. If the limit parameter is a delay limit parameter, in groups, the value is 128, and the number of points in each point cloud group is 4, then the encoding limit point threshold is the actual total number of intermediate coefficients of the points to be encoded in the 128 point cloud groups, and the total number does not exceed 128 * 4 = 512.

[0137] In another embodiment, when the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: points, groups, and multiples; wherein, restriction parameter 1 being in multiples means that when restriction parameter 2 limits the number of points to be encoded in points, restriction parameter 1 limits the number of points to be encoded in points, and restriction parameter 1 limits the number of points to be encoded by a multiple of restriction parameter 2; specifically, if the cache restriction parameter is in multiples, And the value of the cache limit parameter is Y1, then the cache limit point threshold is Y1 times the delay limit point threshold; if the delay limit parameter is in multiples and the value of the delay limit parameter is Y2, then the delay limit point threshold is Y2 times the cache limit point threshold; for example, assuming that the delay limit parameter (limit parameter 1) is in multiples and the value is 2; when the cache limit parameter (limit parameter 2) is in points and the value of the cache limit parameter is 1024, the delay limit parameter is in points and the value of the delay limit parameter is 1024*2=2048.

[0138] When the limit parameter 2 limits the number of packets to be encoded in units of the number of groups, the limit parameter 1 limits the number of packets to be encoded in units of the number of groups, and the limit parameter 1 limits the number of packets to be encoded as a multiple of the number of packets to be encoded by the limit parameter 2; specifically, if the buffer limit parameter is in units of a multiple relationship and the value of the buffer limit parameter is Y1, then the buffer limit point threshold is Y1 times the delay limit point threshold; if the delay limit parameter is in units of a multiple relationship and the value of the delay limit parameter is Y2, then the delay limit point threshold is Y2 times the buffer limit point threshold; for example, assuming that the delay limit parameter (limit parameter 1) is in units of a multiple and the value is 2; when the buffer limit parameter (limit parameter 2) is in units of the number of groups and the value of the buffer limit parameter is 64, the delay limit parameter is in units of the number of groups, and the value of the delay limit parameter is 64*2=128.

[0139] As can be seen from the above, the combination methods include: both limit parameter 1 and limit parameter 2 are in units of points or groups; limit parameter 1 is in units of points and limit parameter 2 is in units of groups, or limit parameter 2 is in units of multiples; limit parameter 1 is in units of groups and limit parameter 2 is in units of points, or limit parameter 2 is in units of multiples. It is understood that the above limit parameter 1 can be either the cache limit parameter or the delay limit parameter, and the above limit parameter 2 is the other of the cache limit parameter and the delay limit parameter.

[0140] S402: Obtain attribute information of the points to be coded in the point cloud.

[0141] The specific implementation of step S402 can be found in Figure 3 The implementation of step S302 will not be repeated here.

[0142] When the restriction parameter is a buffer restriction parameter, continue to execute step S403; when the restriction parameter is a delay restriction parameter, continue to execute step S404; when the restriction parameter includes a buffer restriction parameter and a delay restriction parameter, continue to execute step S405.

[0143] S403: Encode the attribute information of the point to be coded, and control the cache logic in the encoding process according to the cache limit parameter.

[0144] In one embodiment, a cache limit parameter is used to limit the number of to-be-encoded points allowed to be written into the cache. Each to-be-encoded point may include one or more intermediate coefficients. The computer device determines a cache limit point threshold based on the cache limit parameter. Whenever the number of to-be-encoded points written into the cache reaches the cache limit point threshold, the computer device reads the intermediate coefficients of the to-be-encoded point from the cache for encoding.

[0145] In one embodiment, the encoding method is a group-based encoding method, that is, a group in the point cloud is used as the basic encoding unit for encoding. The computer device counts the total number of points to be encoded contained in each point cloud group in the cache. When the number of groups stored in the cache is M, and the total number of points to be encoded contained in these M point cloud groups reaches the cache limit point threshold, the computer device reads the intermediate coefficients of the points to be encoded contained in these M point cloud groups from the cache for encoding. Furthermore, the computer device clears the cache and continues to write the intermediate coefficients of the points to be encoded into the cache.

[0146] In another embodiment, the encoding method is an encoding method based on the number of groups, that is, a group in the point cloud is encoded as the basic encoding unit. The computer device counts the number of point cloud groups M contained in the cache, and calculates the total number of points to be encoded contained in the cache based on M and the point limit threshold of the point cloud group; for example, assuming M = 64, the point limit threshold is 8, then the total number of points to be encoded contained in the cache = 64*8=512. If the total number of points to be encoded contained in the cache reaches the cache limit threshold, the computer device controls the reading of the intermediate coefficients of the points to be encoded contained in the M groups of point cloud groups from the cache for encoding. Furthermore, the computer device clears the cache and continues to write the intermediate coefficients of the points to be encoded into the cache.

[0147] In another embodiment, the encoding method is an encoding method based on the number of groups, that is, a group in the point cloud (a group may include at least one point in the point cloud) is used as the basic encoding unit for encoding. The computer device determines the group number threshold of the point cloud groups allowed to be written in the cache according to the cache limit parameter; whenever the number of point cloud groups written in the cache reaches the group number threshold, the computer device reads the intermediate coefficients of the points to be encoded contained in the point cloud groups written in the cache from the cache for encoding. For example, if the value of the cache limit parameter is 64, the computer device determines that the number of point cloud groups allowed to be written in the cache is 64 according to the cache limit parameter, and whenever the number of point cloud groups written in the cache is 64, the computer device reads the intermediate coefficients of the points to be encoded contained in these 64 groups of point cloud groups from the cache for encoding.

[0148] Furthermore, the computer device can arrange the point cloud groups written in the cache according to a preset arrangement, and encapsulate the preset arrangement into the coded stream formed by encoding; a specific implementation method for the computer device to encapsulate the intermediate coefficient arrangement of the point cloud groups in the cache into the coded stream formed by encoding is: each arrangement corresponds to an index value, and the computer device writes the index value corresponding to the intermediate coefficient arrangement of the point cloud groups in the cache into the coded stream. The computer device can also arrange the point cloud groups written in the cache according to a preset arrangement, and send the preset arrangement directly to the decoding device; the computer device can also arrange the point cloud groups written in the cache according to a default arrangement. It can be understood that when arranging the point cloud groups written in the cache according to the default arrangement, the computer device does not need to inform the decoding device of the arrangement. Among them, the preset arrangement method includes any one of the following: a fixed arrangement method and a dynamic arrangement method; the fixed arrangement method means that the arrangement method of the intermediate coefficients of the point cloud grouping cached each time (such as according to similarity, according to the number of points to be encoded contained in the group, etc.) is the same, and the dynamic arrangement method means that the arrangement method of the intermediate coefficients of the cached point cloud grouping changes; for example, the arrangement method of the intermediate coefficients of the point cloud grouping cached for the first time is the same as the arrangement method of the intermediate coefficients of the point cloud grouping cached for the second time, and the arrangement method of the intermediate coefficients of the point cloud grouping cached for the third time is different.

[0149] In another embodiment, the encoding includes run length encoding. The computer device determines a cache limit threshold for the number of points to be encoded that are allowed to be written into the cache based on a cache limit parameter. The cache limit threshold is used to limit the number of points to be encoded that are allowed to be written into the cache during the encoding process. The method for determining the cache limit threshold can refer to the method for determining the encoding limit threshold in step S401 and will not be repeated here. During the run length encoding process, the computer device sets a run length for counting consecutive specific symbols. At the beginning, the run length is set to an initial value. When the intermediate coefficients of the points to be encoded are sequentially read from the cache, if an intermediate coefficient with a specific symbol is read, the run length count is incremented by one step. If the number of points to be encoded read equals the cache limit threshold, the current run length is encoded, or the cache limit threshold and the intermediate coefficients of the points to be encoded with the specific symbol are encoded, and the run length is set to the initial value for re-counting. If the intermediate coefficients of the points to be encoded with a non-specific symbol are read, the current run length and the intermediate coefficients of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting.

[0150] For example, assume that the cache limit parameter is in points and the value is 1024 (i.e., the cache limit point number is 1024). If the computer device uses run-length encoding to encode the residual of the attribute information of the to-be-encoded point, the computer device counts the number of points in the residual of the attribute information of the to-be-encoded point that are consecutively specific symbols (such as 0), recorded as run-length. If the residual of the attribute information of the current to-be-encoded point is a specific symbol, the run-length value increases by 1. Whenever the number of points of the to-be-encoded point in the cache reaches 1024, the computer device encodes the run-length value and the residual of the attribute information of the current to-be-encoded point, sets the run-length to the initial value (such as 0), and re-counts the run-length. If the residual of the attribute information of the current to-be-encoded point is a non-specific symbol (such as non-0), the computer device encodes the current run-length value and the residual of the attribute information of the current to-be-encoded point, sets the run-length to the initial value (such as 0), and re-counts the run-length.

[0151] S404: Encode the attribute information of the point to be encoded, and control the delay logic in the encoding process according to the delay limit parameter.

[0152] The specific encoding method used by the computer device to encode the attribute information of the code point is an encoding method that requires statistics on the preceding sequence; for example, the specific encoding method used by the computer device to encode the attribute information of the code point may be run-length encoding. The following description takes the run-length encoding as an example:

[0153] In one embodiment, a computer device determines a delay limit point threshold based on a delay limit parameter. The delay limit point threshold is used to limit the number of points to be encoded that are allowed to be delayed during the encoding process. The method for determining the delay limit point threshold can refer to the method for determining the encoding limit point threshold in step S401 and will not be repeated here. During the run length encoding process, a run length is set for counting consecutive specific symbols. The run length is set to an initial value (e.g., 0). When the computer device sequentially reads the intermediate coefficients of the points to be encoded from the cache, if the computer device reads an intermediate coefficient with a value of a specific symbol, the run length count value is increased by one step. The computer device obtains the run length count value. If the run length count value reaches the delay limit point threshold, the computer device performs parameter encoding and resets the run length to the initial value for re-counting. The computer device performing parameter encoding refers to: the computer device encoding the current run length; or the computer device encoding a limit identifier, the limit identifier being used to set the run length to a preset value (e.g., the preset value can be the delay limit point threshold); or the computer device encoding the delay limit point threshold and the intermediate coefficients of the points to be encoded with a value of the specific symbol. If the computer device reads an intermediate coefficient of a point to be encoded whose value is a non-specific symbol, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0154] For example, assuming that the current run length is 63, the specific symbol is 0, and the delay limit point threshold is 64; if the computer device reads that the value of the intermediate coefficient of the point to be encoded is 0, the current run length is 63+1=64, the computer device encodes the current run length (64), or the computer device encodes the restriction identifier, or the computer device encodes the delay limit point threshold (64) and the intermediate coefficient of the point to be encoded with the value of the specific symbol, and resets the run length to the initial value for recounting; if the computer device reads that the value of the intermediate coefficient of the point to be encoded is 1 (non-0), the computer device encodes the current run length (63) and the intermediate coefficient of the point to be encoded with the value of 1, and resets the run length to the initial value for recounting.

[0155] In another embodiment, the computer device determines a delay limit point threshold based on a delay limit parameter. The delay limit point threshold is used to limit the number of points to be encoded that are allowed to be delayed during the encoding process. The method for determining the delay limit point threshold can refer to the method for determining the encoding limit point threshold in step S401 and will not be repeated here. During the run encoding process, a run length for counting consecutive specific symbols is set, and the run length is set to an initial value (such as 0). During the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache, if the computer device reads an intermediate coefficient with a value of a specific symbol, the count value of the run length is increased by one step; if the number of points to be encoded reaches the delay limit point threshold, the computer device encodes the current run length, or the computer device encodes the delay limit point threshold and the intermediate coefficients of the points to be encoded with a value of a specific symbol, and resets the run length to the initial value for re-counting. If the computer device reads an intermediate coefficient of the point to be encoded with a value of a non-specific symbol, the computer device encodes the current run length and the intermediate coefficients of the non-specific symbol, and resets the run length to the initial value for re-counting.

[0156] For example, assuming that the number of currently delayed intermediate coefficients is 58, the specific symbol is 0, and the threshold value of the number of points to be encoded that are allowed to be delayed is 64; if the computer device reads that the value of the intermediate coefficient of the point to be encoded is 0, the current run length is 58+1=59, and the computer device continues to read the next intermediate coefficient; if the computer device reads that the value of the intermediate coefficient of the point to be encoded is 1 (non-0), the computer device encodes the current run length (59) and the intermediate coefficient of the point to be encoded with a value of 1, and resets the run length to the initial value to recount.

[0157] It can be understood that in the above embodiment, the run length in the run encoding is less than or equal to the delay limit point threshold; based on this, the computer device can also set the number of bits of the run length according to the delay limit point threshold.

[0158] For example, assuming that the delay limit parameter is in points and the value is 1024 (i.e., the delay limit point number is 1024), if the computer device uses run-length encoding to encode the residual of the attribute information of the to-be-encoded point, the computer device counts the number of points in the residual of the attribute information of the to-be-encoded point that are consecutively specific symbols (such as 0), recorded as run-length. If the residual of the attribute information of the current to-be-encoded point is a specific symbol, the run-length value increases by 1. Whenever the run-length value reaches 1024, the computer device encodes the run-length value and the residual of the attribute information of the current to-be-encoded point, sets the run-length to the initial value (such as 0), and re-counts the run-length. If the residual of the attribute information of the current to-be-encoded point is a non-specific symbol (such as non-0), the computer device encodes the current run-length value and the residual of the attribute information of the current to-be-encoded point, sets the run-length to the initial value (such as 0), and re-counts the run-length.

[0159] S405 : Encode the attribute information of the point to be coded, and control the coding logic in the coding process according to the buffer limit parameter and the delay limit parameter.

[0160] In one embodiment, the cache limit parameter and the delay limit parameter are independent of each other. Taking the intermediate coefficients of the attribute information of the point to be encoded as the transformation coefficients as an example, the computer device first controls the cache logic in the encoding process according to the cache limit parameter; the specific implementation method can refer to the implementation method in step S403, which will not be repeated here. If encoding is performed in groups, after caching the intermediate coefficients of the point cloud grouping, the computer device can arrange the intermediate coefficients of the point cloud grouping, and can write the arrangement method into the encoded code stream; or, directly specify the arrangement method to the decoding device. The computer device clears the cache and repeats the above steps until the intermediate coefficients of all point cloud groups are encoded. During the encoding process, the computer device determines the run length during run encoding according to the delay limit parameter, and controls the encoding logic in the encoding process based on the run length; the specific implementation method can refer to the implementation method in step S404, which will not be repeated here.

[0161] For example, assuming that both the cache limit parameter and the delay limit parameter are set in units of points, the cache limit parameter is 1024, the delay limit parameter is 2048, and encoding is performed in groups; the computer device counts the number of points to be encoded contained in each point cloud group to be encoded in the cache, and determines the number of groups M based on the cache limit parameter and the number of points to be encoded contained in each point cloud group to be encoded in the cache; where M is a positive integer, the total number of points to be encoded contained in the M groups of point cloud groups to be encoded is less than or equal to 1024, and the total number of points to be encoded contained in the M+1 groups of point cloud groups to be encoded is greater than 1024. The computer device arranges the transform coefficients in the M groups of point cloud groups to be encoded one by one (e.g., in a fixed arrangement, a random arrangement, or a specified arrangement). After obtaining the transform coefficients in the M groups of point cloud groups to be encoded, the computer device clears the current cache and restarts caching the transform coefficients of the groups to be encoded until the transform coefficients in all point cloud groups are encoded. In a specific implementation, the transform coefficients of each point cloud group include a first transform coefficient DC and at least one second transform coefficient {AC}. The arrangement of transform coefficients is determined by the computer (encoding device). For example, transform coefficients within even-numbered cache-limited parameter points, or in other words, the number of point cloud groups in the cache is even, are arranged as [DC1,...,DCM,{AC1},...,{ACM}]. Transform coefficients within odd-numbered cache-limited parameter points, or in other words, the number of point cloud groups in the cache is odd, are arranged as [{AC1},...,{ACM},DC1,...,DCM]. During the encoding process, the computer may employ run-length encoding to encode the residual of the attribute information of the points to be encoded. The run-length is calculated by counting the number of consecutive points in the residual of the attribute information of the points to be encoded that contain a specific symbol. If the residual symbol of the attribute information of the current point to be encoded is a specific value (e.g., 0), the run-length value is incremented by 1. Whenever the run-length reaches the delay-limited number of points indicated by the delay limit parameter, the run-length value and the residual of the attribute information of the current point to be encoded are encoded, and the run-length value is reset to 0, restarting the count. If the residual symbol of the attribute information of the current point to be encoded is a non-specific value (such as non-0), the computer device first encodes the run-length value, then encodes the residual of the attribute information of the current point to be encoded, sets the run-length to 0, and restarts counting.

[0162] In another embodiment, the buffer limit parameter and the delay limit parameter are interrelated; that is, the buffer limit parameter is determined based on the delay limit parameter, or vice versa. For example, if the intermediate coefficients of the attribute information of the point to be encoded are transform coefficients, the computer device determines the buffer limit parameter and the delay limit parameter based on their correlation, and then controls the caching logic during the encoding process according to the buffer limit parameter. The specific implementation method is described in step S403 and is not further described here. If encoding is performed in groups, after caching the intermediate coefficients of the point cloud group, the computer device can arrange the intermediate coefficients of the point cloud group and write the arrangement into the encoded bitstream; alternatively, the arrangement can be directly specified to the decoding device. The computer device clears the buffer and repeats the above steps until the intermediate coefficients of all point cloud groups are encoded. During the encoding process, the computer device determines the run length for run encoding according to the delay limit parameter and controls the encoding logic based on the run length. The specific implementation method is described in step S404 and is not further described here.

[0163] For example, let the delay limit parameter be in multiples and have a value of 2; let the cache limit parameter be in groups and have a value of 64. Then the cache limit points determined based on the cache limit parameter are the total number of points to be encoded in the 64 point cloud groups, and the delay limit points determined based on the delay limit parameter are the total number of points to be encoded in 64*2=128 point cloud groups. For another example, let the cache limit parameter be in groups and have a value of 64, and the second transform coefficients of the first point cloud group in the 64 point cloud groups are k. Then the delay limit parameter is in points and has a value of 64+k to ensure real-time decoding of the first point cloud group. If encoding is performed in groups, the computer device counts the number of points to be encoded contained in the 64 point cloud groups, and arranges the transform coefficients in the 64 point cloud groups one by one (such as a fixed arrangement, a random arrangement, or a specified arrangement). After obtaining the transform coefficients for 64 groups of point cloud groups to be encoded, the computer device clears the current cache and restarts caching the transform coefficients for the groups to be encoded until the transform coefficients for all point cloud groups are encoded. In a specific implementation, the transform coefficients for each point cloud group include a first transform coefficient DC and at least one second transform coefficient {AC}. The arrangement of the transform coefficients is determined by the computer device (encoding device). For example, for transform coefficients within an even-numbered cache-restricted parameter point, or when the number of groups in the cache is even, the transform coefficients are arranged as [DC1,...,DCM,{AC1},...,{ACM}]. For transform coefficients within an odd-numbered cache-restricted parameter point, or when the number of groups in the cache is odd, the transform coefficients are arranged as [{AC1},...,{ACM},DC1,...,DCM]. During the encoding process, the computer device may employ run-length encoding to encode the residual of the attribute information of the points to be encoded. The number of consecutive points in the residual of the attribute information of the points to be encoded that are a specific symbol is counted as the run-length. If the residual symbol of the attribute information of the current point to be coded is a specific value (such as 0), the run-length value is increased by 1. Whenever the run-length reaches the delay threshold indicated by the delay limit parameter, the run-length value and the residual of the attribute information of the current point to be coded are encoded, and the run-length is set to 0, and the counting is restarted. If the residual symbol of the attribute information of the current point to be coded is a non-specific value (such as non-0), the computer device first encodes the run-length value, then encodes the residual of the attribute information of the current point to be coded, and sets the run-length to 0, and restarts the counting.

[0164] The embodiments of this application are Figure 3On the basis of the embodiment, on the one hand, the encoding process of point cloud attribute information is controlled by one or more of the cache limit parameters and the delay limit parameters; on the other hand, the attribute information of multiple points to be encoded or multiple point cloud groups to be encoded are arranged according to a preset arrangement method, thereby improving the encoding performance of point cloud attribute information.

[0165] See also Figure 5 , Figure 5 Another point cloud processing method provided in an embodiment of the present application can be executed by a computer device, which can be Figure 2 The decoding device 202 in the point cloud processing system 20 is shown. Figure 5 As shown, the point cloud processing method may include the following steps S501 to S503:

[0166] S501: Obtain the encoded code stream of the point to be decoded in the point cloud.

[0167] The coded code stream of the point to be decoded is obtained by encoding the code point by the encoding device. For details, see Figure 3 and Figure 4 The encoding stream of the point to be decoded in the point cloud can be obtained by the computer device in real time from the encoding device, or it can be obtained by the computer device from a downloaded encoding stream.

[0168] S502: Obtain restriction parameters of the point cloud.

[0169] The computer device obtains the point cloud constraint parameters by any one or more of the following: parsing the constraint parameters set to fixed values ​​from the encoded bitstream, parsing the constraint parameters set to variable values ​​from the encoded bitstream, and parsing the constraint parameters set to dynamically adjusted values ​​from the encoded bitstream. It is understood that if the point cloud constraint parameters are not parsed from the encoded bitstream, the computer device determines the point cloud constraint parameters to be default values.

[0170] In one embodiment, when the restriction parameter includes only one parameter, the unit of the restriction parameter includes any of the following: points or groups. Restriction in points means limiting the number of points to be decoded when decoding is performed in points; restriction in groups means limiting the number of groups to be decoded when decoding is performed in groups. When the restriction parameter is a cache restriction parameter, the decoding restriction point threshold is the cache restriction point threshold; the unit of the cache restriction parameter can be points or groups. Similarly, when the restriction parameter is a delay restriction parameter, the decoding restriction point threshold is the delay restriction point threshold; the unit of the delay restriction parameter can be points or groups.

[0171] In another embodiment, when the restriction parameter includes two parameters, the units of the restriction parameter include any one or more of the following: points, groups, and multiples; wherein, restriction parameter 1 being in multiples means that: when restriction parameter 2 limits the number of points to be decoded in points, restriction parameter 1 limits the number of points to be decoded in points, and restriction parameter 1 limits the number of points to be decoded to a multiple of the number of points to be decoded by restriction parameter 2. When restriction parameter 2 limits the number of groups to be decoded in groups, restriction parameter 1 limits the number of groups to be decoded in groups, and restriction parameter 1 limits the number of groups to be decoded to a multiple of the number of groups to be decoded by restriction parameter 2.

[0172] As can be seen from the above, the combination methods include: both limit parameter 1 and limit parameter 2 are in units of points or groups; limit parameter 1 is in units of points and limit parameter 2 is in units of groups, or limit parameter 2 is in units of multiples; limit parameter 1 is in units of groups and limit parameter 2 is in units of points, or limit parameter 2 is in units of multiples. It is understood that the above limit parameter 1 can be either the cache limit parameter or the delay limit parameter, and the above limit parameter 2 is the other of the cache limit parameter and the delay limit parameter.

[0173] S503: Decode the coded code stream of the decoding point, and control the decoding process according to the restriction parameters.

[0174] The decoding method includes run-length decoding. If the encoding device encodes in units of points, the computer device decodes in units of points. If the encoding device encodes in units of groups, the computer device decodes in units of groups. The restriction parameters may include at least one of a buffer restriction parameter and a delay restriction parameter.

[0175] In a first implementation, the computer device controls the decoding process according to the restriction parameter by: determining a restriction point threshold according to the restriction parameter, the restriction point threshold being used to limit the number of points to be decoded; for example, if the restriction parameter is a cache restriction parameter, the computer device determines the number of points to be decoded that can be cached in the cache according to the cache restriction parameter; if the restriction parameter is a delay restriction parameter, the computer device determines the number of points to be decoded that can be delayed during the decoding process according to the delay restriction parameter. In one implementation, during run decoding, if the computer device reads a restriction flag, the count value of the first run length can be set to a preset value (such as the restriction point threshold); if the count value of the first run length is greater than zero and not equal to the restriction point threshold, the intermediate coefficient of the current point to be decoded is set to a specific value (such as 0), and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero; the computer device obtains the intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero and performs decoding. In another implementation, during run decoding, if the computer device reads a restriction flag, it can parse the first run length to obtain a count value of the first run length. If the count value of the first run length is greater than zero and not equal to the restriction point threshold, the intermediate coefficient of the current point to be decoded is set to a specific value (e.g., 0), and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero. The computer device then obtains the intermediate coefficient of the point to be decoded that is read when the count value of the first run length is equal to zero and performs decoding. The restriction flag is used to indicate whether the restriction point threshold is used. If the restriction flag is read, it indicates that the restriction point threshold is used; if the restriction flag is not read, it indicates that the restriction point threshold is not used. The computer device can read the restriction flag in any of the following ways: reading from the encoded bitstream, setting by default, or obtaining it through conditional determination, such as the condition that the first run length is equal to a preset value, i.e., when the first run length is equal to the preset value, the restriction flag is set.

[0176] In a second implementation, a computer device controls the decoding process according to a restriction parameter by determining a restriction point threshold based on the restriction parameter, where the restriction point threshold is used to limit the number of points to be decoded. During run decoding, a first run length count value is read from the coded bitstream. If the first run length count value is greater than zero and equal to the restriction point threshold, the computer device sets the intermediate coefficient of the current point to be decoded to a specific value and decreases the first run length count value by one step until the first run length count value equals zero. A second run length count value is read from the coded bitstream. If the second run length count value is greater than zero and not equal to the restriction point threshold, the computer device sets the intermediate coefficient of the current point to be decoded to a preset value and decreases the second run length count value by one step until the second run length count value equals zero. The intermediate coefficient of the point to be decoded read when the second run length count value equals zero is obtained and decoded. It should be noted that the second run length is the next run length after the first run length. If the count value of the second run length is greater than zero and equal to the limit point threshold, the computer device sets the intermediate coefficient of the current point to be decoded to a specific value and reduces the count value of the second run length by one step until the count value of the second run length is equal to zero. The count value of the third run length is read from the encoded bitstream and the judgment is continued. In other words, if the count value of the read run length is greater than zero and equal to the limit point threshold, then when the count value of the run length is 0, the computer device will continue to read the count value of the next run length. If the count value of the read run length is greater than zero and not equal to the limit point threshold, then when the count value of the run length is 0, the computer device will obtain the intermediate coefficient of the point to be decoded read when the count value of the run length is equal to zero and perform decoding.

[0177] In implementation method three, during run-length decoding, the computer device reads a first run-length count value from the encoded bitstream. If the first run-length count value is greater than zero, the computer device sets the intermediate coefficient of the current point to be decoded to a specific value and decrements the first run-length count value by one step until the first run-length count value equals zero. The intermediate coefficient of the point to be decoded read when the first run-length count value equals zero is obtained and decoded. It should be noted that in this implementation method, the intermediate coefficient of the point to be decoded read when the first run-length count value equals zero may be a specific value. In one possible implementation method, the intermediate coefficients with the specific value may be accumulated over multiple run lengths.

[0178] It is understood that when encoding is performed in groups, the decoding process is similar to the above-mentioned implementation methods 1 to 3 and will not be repeated here. The difference is that when the restriction parameter is a cache restriction parameter, the computer device can cache the intermediate parameters of the read point cloud group and then arrange the intermediate coefficients of each point cloud group in the cache according to the obtained arrangement method, and then decode according to the above-mentioned implementation methods 1 to 3.

[0179] In one embodiment, the decoding method is a decoding method based on the number of points, that is, a point in the point cloud is decoded as the basic decoding unit. The specific implementation method of the computer device decoding the coded code stream of the point to be decoded is: the coded code stream of the point to be decoded is parsed to obtain the intermediate coefficients of the point to be decoded, the intermediate coefficients include residuals or transform coefficients, the computer device writes the intermediate coefficients of the point to be decoded into the cache, and reads the intermediate coefficients of the point to be decoded from the cache for decoding. During the decoding process, the computer device can control the decoding process based on the restriction parameters; for example, the computer device can control the decoding process based on the delay restriction parameters; for another example, the computer device can control the decoding process based on the cache restriction parameters; for another example, the computer device can control the decoding process based on the cache restriction parameters and the delay restriction parameters. The specific implementation method of the computer device decoding the coded code stream can refer to the above-mentioned implementation method of point cloud decoding, which will not be repeated here.

[0180] In another embodiment, the decoding method is a decoding method based on the number of groups, that is, a group in the point cloud is decoded as the basic decoding unit. The specific implementation method of the computer device decoding the coded code stream of the point to be decoded is: the coded code stream of the point to be decoded is parsed to obtain multiple groups of point cloud groups, each group of point cloud groups contains the intermediate coefficients of the point to be decoded, and the intermediate coefficients include residuals or transformation coefficients. The computer device writes the point cloud group into the cache, and sequentially reads the intermediate coefficients of the point to be decoded in the point cloud group from the cache for decoding. During the decoding process, the computer device can control the decoding process based on the restriction parameters; for example, the computer device can control the decoding process based on the delay restriction parameters; for another example, the computer device can control the decoding process based on the cache restriction parameters; for another example, the computer device can control the decoding process based on the cache restriction parameters and the delay restriction parameters. The specific implementation method of the computer device decoding the coded code stream can refer to the above-mentioned implementation method of point cloud decoding, which will not be repeated here.

[0181] In the embodiment of the present application, when decoding a point to be decoded in a point cloud, the encoding stream of the point to be decoded and the point cloud's restriction parameters can be obtained, the encoding stream of the point to be decoded can be decoded, and the decoding process can be controlled according to the restriction parameters. By obtaining the point cloud's restriction parameters, the embodiment of the present application makes the decoding process of the attribute information of the point to be decoded controllable, thereby improving the decoding performance of the point cloud's attribute information.

[0182] See also Figure 6 , Figure 6 Another point cloud processing method provided in the embodiment of the present application can be executed by a computer device, which can be Figure 2 The decoding device 202 in the point cloud processing system 20 is shown. Figure 5 As shown, the point cloud processing method may include the following steps S601 to S605:

[0183] S601: Obtain the encoded code stream of the point to be decoded in the point cloud.

[0184] S602: Obtain restriction parameters of the point cloud.

[0185] The specific implementation of step S601 and step S602 can be referred to Figure 5 The implementation of step S501 and step S502 will not be repeated here.

[0186] When the restriction parameter is a buffer restriction parameter, continue to execute step S603; when the restriction parameter is a delay restriction parameter, continue to execute step S604; when the restriction parameter includes a buffer restriction parameter and a delay restriction parameter, continue to execute step S605.

[0187] S603: Decode the attribute information of the point to be decoded, and control the cache logic in the decoding process according to the cache limit parameter.

[0188] In one embodiment, a cache limit parameter is used to limit the number of points to be decoded that can be written to the cache. Each point to be decoded may include one or more intermediate coefficients. The computer device determines a cache limit threshold based on the cache limit parameter. Whenever the number of points to be decoded written to the cache reaches the cache limit threshold, the computer device reads the intermediate coefficients of the point to be decoded from the cache for decoding.

[0189] In one embodiment, the decoding method is a group-based decoding method, that is, a group in the point cloud is used as the basic decoding unit for decoding. The computer device counts the total number of points to be decoded contained in each point cloud group in the cache. When the number of groups stored in the cache is M, and the total number of points to be decoded contained in these M point cloud groups reaches the cache limit point threshold, the computer device reads the intermediate coefficients of the points to be decoded contained in these M point cloud groups from the cache for decoding. Furthermore, the computer device clears the cache and continues to write the intermediate coefficients of the points to be decoded into the cache.

[0190] In another embodiment, the decoding method is a group-based decoding method, that is, a group in the point cloud is used as the basic decoding unit for decoding. The computer device counts the total number of target points to be decoded contained in each point cloud group in the cache, and the number of target points to be decoded in each point cloud group is less than or equal to the point limit threshold of each point cloud group. If the cache includes M point cloud groups and the total number of target points to be decoded contained in the M point cloud groups reaches the cache limit threshold, the intermediate coefficients of the points to be decoded contained in the M point cloud groups are read from the cache for decoding. Furthermore, the computer device clears the cache and continues to write the intermediate coefficients of the points to be decoded into the cache.

[0191] In another embodiment, the decoding method is a decoding method based on the number of groups, that is, a group in the point cloud (a group may include at least one point in the point cloud) is used as the basic decoding unit for decoding. The computer device counts the number of point cloud groups M contained in the cache, and calculates the total number of points to be decoded contained in the cache based on M and the point limit threshold of the point cloud group; for example, assuming M = 64, the point limit threshold is 8, then the total number of points to be decoded contained in the cache = 64*8 = 512. If the total number of points to be decoded contained in the cache reaches the cache limit point threshold, the computer device controls the reading of the intermediate coefficients of the points to be decoded contained in the M groups of point cloud groups from the cache for decoding. Furthermore, the computer device clears the cache and continues to write the intermediate coefficients of the points to be decoded into the cache.

[0192] In one embodiment, during the decoding process, the computer device can also obtain the preset arrangement of the point cloud groups in the cache. The preset arrangement includes any one of the following: a fixed arrangement and a dynamic arrangement; a fixed arrangement means that the arrangement of the intermediate coefficients of the point cloud groups cached each time (such as according to similarity, according to the number of points to be encoded contained in the group, etc.) is the same, and a dynamic arrangement means that the arrangement of the intermediate coefficients of the cached point cloud groups changes; for example, the arrangement of the intermediate coefficients of the point cloud groups cached for the first time is the same as the arrangement of the intermediate coefficients of the point cloud groups cached for the second time, and the arrangement of the intermediate coefficients of the point cloud groups cached for the third time is different. The specific implementation method of the computer device obtaining the preset arrangement of the point cloud groups in the cache is: each arrangement corresponds to an index value, the computer device can parse the index value from the encoded code stream, and determine the arrangement of the point cloud groups based on the index value.

[0193] S604: Decode the attribute information of the point to be decoded, and control the delay logic in the decoding process according to the delay limit parameter.

[0194] The specific implementation of step S604 can be found in Figure 5 The implementation of step S503 is not described here in detail.

[0195] S605: Decode the attribute information of the to-be-decoded point, and control the decoding logic in the decoding process according to the buffer limit parameter and the delay limit parameter.

[0196] In one embodiment, the cache limit parameter and the delay limit parameter are independent of each other. The computer device first reads the intermediate parameters of the point to be decoded, or the intermediate parameters of the point cloud group, into the cache according to the delay limit parameter; the specific implementation method can refer to the implementation method in step S604, which will not be repeated here. If the encoding device encodes in groups, then after caching the intermediate parameters of the read point cloud group, the computer device can also arrange the intermediate coefficients of each point cloud group in the cache according to the obtained arrangement method. The computer device can then control the decoding logic in the decoding process according to the delay limit parameter; the specific implementation method can refer to the implementation method in step S503, which will not be repeated here.

[0197] In another embodiment, the cache limit parameter and the delay limit parameter are related to each other; that is, the cache limit parameter is determined based on the delay limit parameter, or the delay limit parameter is determined based on the cache limit parameter. The computer device first reads the intermediate parameters of the point to be decoded, or the intermediate parameters of the point cloud grouping, into the cache according to the delay limit parameter; the specific implementation method can refer to the implementation method in step S604, which will not be repeated here. If the encoding device encodes in groups, after caching the intermediate parameters of the read point cloud grouping, the computer device can also arrange the intermediate coefficients of each point cloud grouping in the cache according to the obtained arrangement method. The computer device can then control the decoding logic in the decoding process according to the delay limit parameter; the specific implementation method can refer to the implementation method in step S503, which will not be repeated here.

[0198] The embodiments of this application are Figure 5 On the basis of the embodiment, on the one hand, the decoding process of point cloud attribute information is controlled by one or more of the cache limit parameters and the delay limit parameters; on the other hand, the attribute information of multiple points to be decoded, or multiple point cloud groups to be decoded, are arranged according to the arrangement indicated by the encoding code stream, thereby improving the decoding performance of the point cloud attribute information.

[0199] The above describes in detail the method of the embodiment of the present application. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0200] See Figure 7 , Figure 7 This is a schematic diagram of the structure of a point cloud processing device provided in an embodiment of the present application; the point cloud processing device can be a computer program (including program code) running in an encoding device, for example, the point cloud processing device can be an application software in the encoding device. Figure 7 As shown, the point cloud processing device includes a setting unit 701, an acquisition unit 702 and a processing unit 703. Figure 7 , the detailed description of each unit is as follows:

[0201] A setting unit 701 is used to set limit parameters of the point cloud, where the limit parameters include one or both of a cache limit parameter and a delay limit parameter;

[0202] An acquisition unit 702 is used to acquire attribute information of a point to be encoded in the point cloud;

[0203] The processing unit 703 is configured to encode the attribute information of the point to be encoded; and control the encoding process according to the restriction parameters.

[0204] In one embodiment, the encoding method includes an encoding method using points as units; the processing unit 703 is configured to encode the attribute information of the coding point, specifically to:

[0205] The attribute information of the to-be-coded point is converted to obtain the intermediate coefficients of the to-be-coded point; the intermediate coefficients include residuals or transform coefficients;

[0206] Write the intermediate coefficients of the point to be encoded into the cache;

[0207] The intermediate coefficients of the point to be encoded are read from the cache for encoding.

[0208] In one embodiment, the encoding method includes an encoding method based on the number of groups; the processing unit 703 is configured to encode the attribute information of the code point, specifically to:

[0209] The attribute information of the coded points is grouped and processed to obtain multiple point cloud groups;

[0210] The attribute information of the points to be coded in the point cloud group is converted to obtain a converted point cloud group, wherein the converted point cloud group contains intermediate coefficients of the points to be coded; the intermediate coefficients include residuals or transformation coefficients;

[0211] Write the converted point cloud groups into the cache;

[0212] The intermediate coefficients of the points to be encoded in the converted point cloud group are read from the cache in sequence for encoding.

[0213] In one embodiment, the restriction parameter includes a cache restriction parameter; the processing unit 703 is configured to control the encoding process according to the restriction parameter, specifically to:

[0214] Control the caching logic during encoding according to the cache limit parameters.

[0215] In one embodiment, the processing unit 703 is configured to control the cache logic during the encoding process according to the cache limit parameter, specifically to:

[0216] Determine the cache limit point threshold according to the cache limit parameter. The cache limit point threshold is used to limit the number of points to be encoded that are allowed to be written into the cache.

[0217] Whenever the number of points to be encoded written into the cache reaches a cache limit point number threshold, the intermediate coefficients of the points to be encoded are read from the cache for encoding.

[0218] In one embodiment, the encoding method includes an encoding method based on the number of groups; the processing unit 703 is configured to control reading the intermediate coefficients of the to-be-encoded points from the cache for encoding whenever the number of to-be-encoded points written into the cache reaches a cache limit point threshold, specifically for:

[0219] Count the total number of points to be encoded contained in each point cloud group in the cache. If the cache includes M point cloud groups and the total number of points to be encoded contained in the M point cloud groups reaches the cache limit point threshold, then control the reading of the intermediate coefficients of the points to be encoded contained in the M point cloud groups from the cache for encoding; or,

[0220] The number of point cloud groups M contained in the cache is counted, and the total number of points to be encoded contained in the cache is calculated based on M and the point limit threshold of the point cloud group. If the total number of points to be encoded contained in the cache reaches the cache limit threshold, the intermediate coefficients of the points to be encoded contained in the M groups of point cloud groups are read from the cache for encoding.

[0221] In one embodiment, the encoding method includes an encoding method based on the number of groups; the processing unit 703 is configured to control the cache logic in the encoding process according to the cache limit parameter, specifically to:

[0222] Determine the threshold number of point cloud groups allowed to be written into the cache according to the cache limit parameter;

[0223] Whenever the number of point cloud groups written in the cache reaches a group number threshold, the intermediate coefficients of the points to be encoded contained in the point cloud groups written in the cache are read from the cache for encoding.

[0224] In one embodiment, the processing unit 703 is further configured to:

[0225] Arrange the point cloud groups written into the cache according to the preset arrangement;

[0226] The preset arrangement mode includes any of the following: fixed arrangement mode and dynamic arrangement mode; the fixed arrangement mode means that the arrangement mode of the intermediate coefficients of the point cloud grouping in the cache is fixed, and the dynamic arrangement mode means that the arrangement mode of the intermediate coefficients of the point cloud grouping in the cache is variable.

[0227] In one embodiment, each arrangement corresponds to an index value; the processing unit 703 is further configured to:

[0228] Write the index value corresponding to the arrangement of the intermediate coefficients of the point cloud group in the cache into the encoded bitstream.

[0229] In one embodiment, the encoding includes run length encoding; the processing unit 703 is configured to control the cache logic during the encoding process according to the cache limit parameter, specifically to:

[0230] Determine the cache limit point threshold of the points to be encoded that are allowed to be written into the cache according to the cache limit parameter. The cache limit point threshold is used to limit the number of points to be encoded that are allowed to be written into the cache during the encoding process.

[0231] In the run length encoding process, setting the run length used to count consecutive specific symbols;

[0232] Set the run length to the initial value and sequentially read the intermediate coefficients of the points to be encoded from the cache:

[0233] If an intermediate coefficient with a specific symbol is read, the run length count value is increased by one step; if the number of points to be encoded read is equal to the cache limit point threshold, the current run length is encoded, or the cache limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is set to the initial value for re-counting;

[0234] If the intermediate coefficient of the point to be encoded that is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0235] In one embodiment, the restriction parameter includes a delay restriction parameter; the processing unit 703 is configured to control the encoding process according to the restriction parameter, specifically to:

[0236] The delay logic in the encoding process is controlled according to the delay limit parameters.

[0237] In one embodiment, the encoding includes run length encoding; the processing unit 703 is configured to control the delay logic in the encoding process according to the delay limit parameter, specifically to:

[0238] Determine a delay limit point threshold according to the delay limit parameter, where the delay limit point threshold is used to limit the number of points to be encoded that are allowed to be delayed during the encoding process;

[0239] In the process of run length encoding, a run length is set for counting consecutive specific symbols;

[0240] Set the run length to the initial value and sequentially read the intermediate coefficients of the points to be encoded from the cache:

[0241] If an intermediate coefficient with a specific symbol is read, the run length count value is increased by one step; the run length count value is obtained, and if the run length count value reaches the delay limit point threshold, parameter encoding processing is performed, and the run length is reset to the initial value for re-counting;

[0242] If the intermediate coefficient of the point to be encoded that is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting;

[0243] The parameter coding process refers to: encoding the current run length; or encoding a restriction identifier, which is used to set the run length to a preset value; or encoding a delay limit point threshold and an intermediate coefficient of a point to be coded whose value is a specific symbol.

[0244] In one embodiment, the encoding includes run length encoding; the processing unit 703 is configured to control the delay logic in the encoding process according to the delay limit parameter, specifically to:

[0245] Determine a delay limit point threshold according to the delay limit parameter, where the delay limit point threshold is used to limit the number of points to be encoded that are allowed to be delayed during the encoding process;

[0246] In the process of run length encoding, a run length is set for counting consecutive specific symbols;

[0247] Set the run length to the initial value and sequentially read the intermediate coefficients of the points to be encoded from the cache:

[0248] If an intermediate coefficient with a specific symbol is read, the run length count value is increased by one step; if the number of points to be encoded reaches the delay limit point threshold, the current run length is encoded, or the delay limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is reset to the initial value for re-counting;

[0249] If the intermediate coefficient of the point to be encoded that is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0250] In one embodiment, the delay limit parameter is used to determine a delay limit point threshold; the encoding includes run length encoding, and the run length in the run length encoding is less than or equal to the delay limit point threshold; the processing unit 703 is further configured to:

[0251] Sets the run length in bits according to the delay limit threshold.

[0252] In one embodiment, the restriction parameters include a cache restriction parameter and a delay restriction parameter; the processing unit 703 is configured to control the encoding process according to the restriction parameters, specifically to:

[0253] The encoding logic in the encoding process is controlled according to the cache limit parameters and delay limit parameters.

[0254] In one embodiment, the limiting parameters are set in any one or more of the following ways:

[0255] Set the limit parameters to their default values;

[0256] Setting the restriction parameters to fixed values ​​and encapsulating the set restriction parameters into the encoded bitstream;

[0257] Setting the restriction parameters to variable values ​​and encapsulating the set restriction parameters into the encoded bitstream;

[0258] The values ​​of the restriction parameters are dynamically adjusted during the encoding process, and the dynamically adjusted restriction parameters are encapsulated into the encoded bitstream formed by the encoding.

[0259] In one embodiment, when the restriction parameter includes only one parameter, the unit of the restriction parameter includes any one of the following: points, groups;

[0260] When the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: point number, group number, and multiple relationship.

[0261] In one embodiment, the restriction parameter is used to determine the encoding restriction point threshold; if the restriction parameter is a cache restriction parameter, the encoding restriction point threshold is the cache restriction point threshold; if the restriction parameter is a delay restriction parameter, the encoding restriction point threshold is the delay restriction point threshold;

[0262] If the limit parameter is in points and the value of the limit parameter is X1, then the coding limit point threshold is the actual total number of intermediate coefficients of the X1 points to be coded;

[0263] If the restriction parameter is in groups and the value of the restriction parameter is X2, then the coding restriction point threshold is the actual total number of intermediate coefficients of the points to be coded in the X2 group of point cloud groups;

[0264] If the limit parameter is in groups, and the value of the limit parameter is X3, and the number of points in each point cloud group is limited to X4, then the encoding limit point threshold is the total number of intermediate coefficients of the points to be encoded in the X3 groups of point cloud groups, and the total number does not exceed the value of X3*X4.

[0265] In one embodiment, the restriction parameter includes a cache restriction parameter and a delay restriction parameter; the cache restriction parameter is used to determine the cache restriction point threshold; the delay restriction parameter is used to determine the delay restriction point threshold;

[0266] If the cache limit parameter is in multiples and the value of the cache limit parameter is Y1, then the cache limit point threshold is Y1 times the delay limit point threshold;

[0267] If the delay limit parameter is in a multiple relationship and the value of the delay limit parameter is Y2, then the delay limit point threshold is Y2 times the cache limit point threshold.

[0268] According to one embodiment of the present application, Figure 3 and Figure 4 Some of the steps involved in the point cloud processing method shown can be represented by Figure 7 The point cloud processing device shown is executed by each unit. For example, Figure 3 The step S301 shown in FIG. Figure 7 The setting unit 701 shown in FIG. 1 is executed, and step S302 can be performed by Figure 7 The acquisition unit 702 shown in FIG. 1 is executed, and step S303 can be performed by Figure 7 The processing unit 703 is shown to execute. Figure 4 The step S401 shown in FIG. Figure 7 The setting unit 701 shown in FIG. 7 is executed, and step S402 can be performed by Figure 7 The acquisition unit 702 shown in FIG. 1 is executed, and steps S403 to S405 can be performed by Figure 7 The processing unit 703 is shown to execute. Figure 7 The various units in the point cloud processing device shown can be individually or entirely combined into one or several other units to form a whole, or one (or more) of the units can be further divided into multiple functionally smaller units to form a whole, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the point cloud processing device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0269] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM) and other processing elements and storage elements. Figure 3 and Figure 4 A computer program (including program code) for each step involved in the corresponding method shown in Figure 7 The point cloud processing device shown in and the point cloud processing method of the embodiment of the present application are implemented. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0270] Based on the same inventive concept, the principles and beneficial effects of the point cloud processing device provided in the embodiment of the present application in solving the problem are similar to the principles and beneficial effects of the point cloud processing method in the method embodiment of the present application in solving the problem. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0271] See Figure 8 , Figure 8 This is a schematic diagram of the structure of another point cloud processing device provided in an embodiment of the present application; the point cloud processing device can be a computer program (including program code) running in a decoding device, for example, the point cloud processing device can be an application software in the decoding device. Figure 8 As shown, the point cloud processing device includes an acquisition unit 801 and a processing unit 802. Figure 8 , the detailed description of each unit is as follows:

[0272] An acquisition unit 801 is configured to acquire a coded stream of a point to be decoded in a point cloud, and acquire restriction parameters of the point cloud, where the restriction parameters include one or both of a buffer restriction parameter and a delay restriction parameter.

[0273] The processing unit 802 is configured to decode the coded code stream of the decoding point; and control the decoding process according to the restriction parameters.

[0274] In one embodiment, the acquisition unit 801 acquires the point cloud restriction parameters including any one or more of the following:

[0275] Set the limit parameters to their default values;

[0276] Parse the restricted parameters that have been set to fixed values ​​from the encoded bitstream;

[0277] Parse the restricted parameters that have been set as variable values ​​from the encoded bitstream;

[0278] Parse the set dynamically adjusted restriction parameters from the encoded bitstream.

[0279] In one embodiment, the decoding method includes a decoding method based on point numbers; the processing unit 802 is configured to decode the coded bit stream of the point to be decoded, specifically to:

[0280] Parse the coded bitstream of the point to be decoded to obtain the intermediate coefficients of the point to be decoded; the intermediate coefficients include residuals or transform coefficients;

[0281] Write the intermediate coefficients of the point to be decoded into the cache;

[0282] The intermediate coefficients of the point to be decoded are read from the cache for decoding.

[0283] In one embodiment, the decoding method includes a decoding method based on the number of groups; the processing unit 802 is configured to decode the coded bit stream of the decoding point, specifically to:

[0284] Parse the coded bitstream of the point to be decoded to obtain multiple point cloud groups, each of which contains intermediate coefficients of the point to be decoded; the intermediate coefficients include residuals or transform coefficients;

[0285] Write point cloud groups into the cache;

[0286] The intermediate coefficients of the points to be decoded in the point cloud group are read from the cache in sequence for decoding.

[0287] In one embodiment, the decoding includes run-length decoding; the processing unit 802 is configured to control the decoding process according to the restriction parameters, specifically to:

[0288] Determine the limit point threshold according to the limit parameter, and the limit point threshold is used to limit the number of points to be decoded;

[0289] During the run length decoding process, if the limit flag is read, the count value of the first run length is set to a preset value;

[0290] If the count value of the first run length is greater than zero and is not equal to the limit point threshold, then the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0291] The intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded.

[0292] In one embodiment, the decoding includes run-length decoding; the processing unit 802 is configured to control the decoding process according to the restriction parameters, specifically to:

[0293] Determine the limit point threshold according to the limit parameter, and the limit point threshold is used to limit the number of points to be decoded;

[0294] During run length decoding, a count value of a first run length is read from the encoded bit stream;

[0295] If the count value of the first run length is greater than zero and equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0296] Reading a count value of a second run length from the encoded bitstream, and if the count value of the second run length is greater than zero and not equal to the limit point number threshold, setting the intermediate coefficient of the current point to be decoded to a preset value, and reducing the count value of the second run length by one step size until the count value of the second run length is equal to zero;

[0297] The intermediate coefficient of the point to be decoded read when the count value of the second run length is equal to zero is obtained and decoded.

[0298] In one embodiment, the decoding includes run-length decoding; the processing unit 802 is configured to control the decoding process according to the constraint parameters, specifically to:

[0299] During run length decoding, a count value of a first run length is read from the encoded bit stream;

[0300] If the count value of the first run length is greater than zero, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0301] The intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded.

[0302] In one embodiment, the restriction parameter includes a cache restriction parameter; the processing unit 802 is configured to control the decoding process according to the restriction parameter, specifically to:

[0303] Control the caching logic during decoding according to the cache limit parameters.

[0304] In one embodiment, the processing unit 802 is configured to control the cache logic during the decoding process according to the cache limit parameter, specifically to:

[0305] Determine the cache limit point threshold according to the cache limit parameter, and the cache limit point threshold is used to limit the number of decoding points allowed to be written into the cache;

[0306] Whenever the number of points to be decoded written into the cache reaches a threshold value of the cache limit number, the intermediate coefficients of the points to be decoded are read from the cache for decoding.

[0307] In one embodiment, the decoding method includes a decoding method based on the number of groups; the processing unit 802 is configured to control reading the intermediate coefficients of the to-be-decoded points from the cache for decoding whenever the number of to-be-decoded points written into the cache reaches a cache limit point threshold, specifically for:

[0308] Count the total number of points to be decoded contained in each point cloud group in the cache. If the cache includes M point cloud groups and the total number of points to be decoded contained in the M point cloud groups reaches the cache limit point threshold, then control the reading of the intermediate coefficients of the points to be decoded contained in the M point cloud groups from the cache for decoding; or,

[0309] The number of point cloud groups M contained in the cache is counted, and the total number of points to be encoded contained in the cache is calculated based on M and the point limit threshold of the point cloud group. If the total number of points to be encoded contained in the cache reaches the cache limit threshold, the intermediate coefficients of the points to be decoded contained in the M groups of point cloud groups are read from the cache for decoding.

[0310] In one embodiment, the decoding method includes a decoding method based on the number of groups; the processing unit 802 is configured to control the cache logic in the decoding process according to the cache limit parameter, specifically to:

[0311] Determine the threshold number of point cloud groups allowed to be written into the cache according to the cache limit parameter;

[0312] Whenever the number of point cloud groups written into the cache reaches a group number threshold, the intermediate coefficients of the to-be-decoded points contained in the point cloud groups written into the cache are read from the cache for decoding.

[0313] In one embodiment, the processing unit 802 is further configured to:

[0314] Arrange the point cloud groups written into the cache according to the preset arrangement;

[0315] The preset arrangement mode includes any of the following: fixed arrangement mode and dynamic arrangement mode; the fixed arrangement mode means that the arrangement mode of the intermediate coefficients of the point cloud grouping in the cache is fixed, and the dynamic arrangement mode means that the arrangement mode of the intermediate coefficients of the point cloud grouping in the cache is variable.

[0316] In one embodiment, each arrangement corresponds to an index value; the processing unit 802 is further configured to:

[0317] The index value is parsed from the encoded code stream, and the arrangement of the point cloud groups in the cache is determined according to the index value.

[0318] In one embodiment, the restriction parameter includes a delay restriction parameter; the processing unit 802 is configured to control the decoding process according to the restriction parameter, specifically to:

[0319] The delay logic in the decoding process is controlled according to the delay limit parameters.

[0320] In one embodiment, the restriction parameters include a cache restriction parameter and a delay restriction parameter; the processing unit 802 is configured to control the decoding process according to the restriction parameters, specifically to:

[0321] The decoding logic in the decoding process is controlled according to the buffer limit parameters and the delay limit parameters.

[0322] In one embodiment, the limiting parameter includes one or both of a buffer limiting parameter and a delay limiting parameter;

[0323] When the restriction parameter includes only one parameter, the unit of the restriction parameter includes any one or more of the following: points, groups;

[0324] When the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: point number, group number, and multiple relationship.

[0325] According to one embodiment of the present application, Figure 5 and Figure 6 Some of the steps involved in the point cloud processing method shown can be represented by Figure 8 The point cloud processing device shown is executed by each unit. For example, Figure 5 Steps S501 and S502 shown in FIG can be replaced by Figure 8 The acquisition unit 801 shown in FIG. 8 is executed, and step S503 can be performed by Figure 8 Processing unit 802 is shown executing. Figure 6 Steps S601 and S602 shown in FIG can be replaced by Figure 8 The acquisition unit 801 shown in FIG. 8 is executed, and steps S603 to S605 can be performed by Figure 8 Processing unit 802 is shown executing. Figure 8 The various units in the point cloud processing device shown can be individually or entirely combined into one or several other units to form a whole, or one (or more) of the units can be further divided into multiple functionally smaller units to form a whole, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the point cloud processing device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0326] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM) and other processing elements and storage elements. Figure 5 and Figure 6 A computer program (including program code) for each step involved in the corresponding method shown in Figure 8 The point cloud processing device shown in and the point cloud processing method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer readable recording medium, and loaded into the above-mentioned computing device through the computer readable recording medium and run therein.

[0327] Based on the same inventive concept, the principles and beneficial effects of the point cloud processing device provided in the embodiment of the present application in solving the problem are similar to the principles and beneficial effects of the point cloud processing method in the method embodiment of the present application in solving the problem. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0328] Figure 9 This is a schematic diagram of the structure of an encoding device provided in an embodiment of the present application; the encoding device may refer to a computer device used by a provider of point cloud media, which may be a terminal (such as a PC, a smart mobile device (such as a smartphone), etc.) or a server. Figure 9 As shown, the encoding device includes a capture device 901, a processor 902, a memory 903 and a transmitter 904.

[0329] Capture device 901 is used to capture real-world audio-visual scenes to obtain raw data of point cloud media (including audio and video content synchronized in time and space). Capture device 901 may include, but is not limited to, audio equipment, video equipment, and sensor equipment. Audio equipment may include audio sensors and microphones. Video equipment may include standard cameras, stereo cameras, light field cameras, and other equipment. Sensor equipment may include laser equipment, radar equipment, and other equipment.

[0330] The processor 902 (also called CPU (Central Processing Unit)) is the processing core of the encoding device. The processor 902 is suitable for implementing one or more program instructions, and is specifically suitable for loading and executing one or more program instructions to achieve Figure 3 and Figure 4 The flow of the point cloud processing method shown.

[0331] The memory 903 is a memory device in the encoding device, which is used to store programs and media resources. It is understandable that the memory 903 here can include both the built-in storage medium in the encoding device and the extended storage medium supported by the encoding device. It should be noted that the memory can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located away from the aforementioned processor. The memory provides a storage space, which is used to store the operating system of the encoding device. In addition, the storage space is also used to store a computer program, which includes program instructions, and the program instructions are suitable for being called and executed by the processor to perform the various steps of the point cloud processing method. In addition, the memory 903 can also be used to store the point cloud media file formed after processing by the processor, and the point cloud media file includes media file resources and media presentation description information.

[0332] The transmitter 904 is used to implement transmission interaction between the encoding device and other devices, specifically to implement the transmission of point cloud media between the encoding device and the content playback device. That is, the encoding device transmits the relevant media resources of the point cloud media to the content playback device through the transmitter 904.

[0333] See also Figure 9 , the processor 902 may include a converter 921, an encoder 922, and a packager 923; wherein:

[0334] Converter 921 performs a series of conversions on the captured video content, making it suitable for point cloud media video encoding. This conversion process may include stitching and projection, and optionally, region packing. Converter 921 can convert the captured 3D video content into 2D images and provide them to the encoder for video encoding.

[0335] The encoder 922 is used to perform audio encoding on the captured audio content to form an audio code stream of the point cloud media. It is also used to perform video encoding on the 2D image converted by the converter 921 to obtain a video code stream.

[0336] The encapsulator 923 is used to encapsulate the audio and video streams into a file container according to the point cloud media file format (such as ISOBMFF) to form a media file resource for the point cloud media. The media file resource can be a media file or a media fragment to form a media file for the point cloud media. The encapsulator also uses media presentation description information to record metadata for the point cloud media file resource according to the requirements of the point cloud media file format. The encapsulated point cloud media file generated by the encapsulator is stored in storage and provided to content playback devices on demand for point cloud media presentation.

[0337] The processor 902 (specifically, each device included in the processor) executes by calling one or more instructions in the memory Figure 3 and Figure 4 Specifically, the memory 903 stores one or more first instructions, which are suitable for being loaded by the processor 902 and executing the following steps:

[0338] Set the limit parameters of the point cloud, which include one or both of cache limit parameters and delay limit parameters;

[0339] Get the attribute information of the points to be coded in the point cloud;

[0340] Encoding attribute information of the code point; and

[0341] The encoding process is controlled according to the constraint parameters.

[0342] In one embodiment, the encoding method includes an encoding method using points as units; a specific embodiment of the processor 902 encoding the attribute information of the coding point is as follows:

[0343] The attribute information of the to-be-coded point is converted to obtain the intermediate coefficients of the to-be-coded point; the intermediate coefficients include residuals or transform coefficients;

[0344] Write the intermediate coefficients of the point to be encoded into the cache;

[0345] The intermediate coefficients of the point to be encoded are read from the cache for encoding.

[0346] In one embodiment, the encoding method includes an encoding method using a group number as a unit; a specific embodiment of the processor 902 encoding the attribute information of the code point is as follows:

[0347] The attribute information of the coded points is grouped and processed to obtain multiple point cloud groups;

[0348] The attribute information of the points to be coded in the point cloud group is converted to obtain a converted point cloud group, wherein the converted point cloud group contains intermediate coefficients of the points to be coded; the intermediate coefficients include residuals or transformation coefficients;

[0349] Write the converted point cloud groups into the cache;

[0350] The intermediate coefficients of the points to be encoded in the converted point cloud group are read from the cache in sequence for encoding.

[0351] In one implementation, the restriction parameter includes a cache restriction parameter; and a specific embodiment in which the processor 902 controls the encoding process according to the restriction parameter is as follows:

[0352] Control the caching logic during encoding according to the cache limit parameters.

[0353] In one implementation, a specific example of the processor 902 controlling the cache logic during the encoding process according to the cache limit parameter is as follows:

[0354] Determine the cache limit point threshold according to the cache limit parameter. The cache limit point threshold is used to limit the number of points to be encoded that are allowed to be written into the cache.

[0355] Whenever the number of points to be encoded written into the cache reaches a cache limit point number threshold, the intermediate coefficients of the points to be encoded are read from the cache for encoding.

[0356] In one embodiment, the encoding method includes an encoding method based on the number of groups; whenever the number of points to be encoded written into the cache reaches a cache limit point threshold, the processor 902 controls the reading of intermediate coefficients of the points to be encoded from the cache for encoding. A specific embodiment is as follows:

[0357] Count the total number of points to be encoded contained in each point cloud group in the cache. If the cache includes M point cloud groups and the total number of points to be encoded contained in the M point cloud groups reaches the cache limit point threshold, then control the reading of the intermediate coefficients of the points to be encoded contained in the M point cloud groups from the cache for encoding; or,

[0358] The number of point cloud groups M contained in the cache is counted, and the total number of points to be encoded contained in the cache is calculated based on M and the point limit threshold of the point cloud group. If the total number of points to be encoded contained in the cache reaches the cache limit threshold, the intermediate coefficients of the points to be encoded contained in the M groups of point cloud groups are read from the cache for encoding.

[0359] In one embodiment, the encoding method includes an encoding method based on the number of groups; a specific embodiment in which the processor 902 controls the cache logic in the encoding process according to the cache limit parameter is as follows:

[0360] Determine the threshold number of point cloud groups allowed to be written into the cache according to the cache limit parameter;

[0361] Whenever the number of point cloud groups written in the cache reaches a group number threshold, the intermediate coefficients of the points to be encoded contained in the point cloud groups written in the cache are read from the cache for encoding.

[0362] In one embodiment, the computer program in the memory 903 is loaded by the processor 902 and further performs the following steps:

[0363] Arrange the point cloud groups written into the cache according to the preset arrangement;

[0364] The preset arrangement mode includes any of the following: fixed arrangement mode and dynamic arrangement mode; the fixed arrangement mode means that the arrangement mode of the intermediate coefficients of the point cloud grouping in the cache is fixed, and the dynamic arrangement mode means that the arrangement mode of the intermediate coefficients of the point cloud grouping in the cache is variable.

[0365] In one embodiment, each arrangement corresponds to an index value; the computer program in the memory 903 is loaded by the processor 902 and further performs the following steps:

[0366] Write the index value corresponding to the arrangement of the intermediate coefficients of the point cloud group in the cache into the encoded bitstream.

[0367] In one embodiment, the encoding includes run length encoding; a specific embodiment in which the processor 902 controls the cache logic during the encoding process according to the cache limit parameter is as follows:

[0368] Determine the cache limit point threshold of the points to be encoded that are allowed to be written into the cache according to the cache limit parameter. The cache limit point threshold is used to limit the number of points to be encoded that are allowed to be written into the cache during the encoding process.

[0369] In the run length encoding process, setting the run length used to count consecutive specific symbols;

[0370] Set the run length to the initial value and sequentially read the intermediate coefficients of the points to be encoded from the cache:

[0371] If an intermediate coefficient with a specific symbol is read, the run length count value is increased by one step; if the number of points to be encoded read is equal to the cache limit point threshold, the current run length is encoded, or the cache limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is set to the initial value for re-counting;

[0372] If the intermediate coefficient of the point to be encoded that is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0373] In one implementation, the restriction parameter includes a delay restriction parameter; and a specific embodiment in which the processor 902 controls the encoding process according to the restriction parameter is as follows:

[0374] The delay logic in the encoding process is controlled according to the delay limit parameters.

[0375] In one embodiment, the encoding includes run length encoding; a specific embodiment in which the processor 902 controls the delay logic in the encoding process according to the delay limit parameter is as follows:

[0376] Determine a delay limit point threshold according to the delay limit parameter, where the delay limit point threshold is used to limit the number of points to be encoded that are allowed to be delayed during the encoding process;

[0377] In the process of run length encoding, a run length is set for counting consecutive specific symbols;

[0378] Set the run length to the initial value and sequentially read the intermediate coefficients of the points to be encoded from the cache:

[0379] If an intermediate coefficient with a specific symbol is read, the run length count value is increased by one step; the run length count value is obtained, and if the run length count value reaches the delay limit point threshold, parameter encoding processing is performed, and the run length is reset to the initial value for re-counting;

[0380] If the intermediate coefficient of the point to be encoded that is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting;

[0381] The parameter coding process refers to: encoding the current run length; or encoding a restriction identifier, which is used to set the run length to a preset value; or encoding a delay limit point threshold and an intermediate coefficient of a point to be coded whose value is a specific symbol.

[0382] In one embodiment, the encoding includes run length encoding; a specific embodiment in which the processor 902 controls the delay logic in the encoding process according to the delay limit parameter is as follows:

[0383] Determine a delay limit point threshold according to the delay limit parameter, where the delay limit point threshold is used to limit the number of points to be encoded that are allowed to be delayed during the encoding process;

[0384] In the process of run length encoding, a run length is set for counting consecutive specific symbols;

[0385] Set the run length to the initial value and sequentially read the intermediate coefficients of the points to be encoded from the cache:

[0386] If an intermediate coefficient with a specific symbol is read, the run length count value is increased by one step; if the number of points to be encoded reaches the delay limit point threshold, the current run length is encoded, or the delay limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is reset to the initial value for re-counting;

[0387] If the intermediate coefficient of the point to be encoded that is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0388] In one embodiment, the delay limit parameter is used to determine a delay limit point threshold; the encoding includes run length encoding, and the run length in the run length encoding is less than or equal to the delay limit point threshold; the computer program in the memory 903 is loaded by the processor 902 and further performs the following steps:

[0389] Sets the run length in bits according to the delay limit threshold.

[0390] In one implementation, the restriction parameters include a cache restriction parameter and a delay restriction parameter; and a specific embodiment in which the processor 902 controls the encoding process according to the restriction parameters is as follows:

[0391] The encoding logic in the encoding process is controlled according to the cache limit parameters and delay limit parameters.

[0392] In one embodiment, the limiting parameters are set in any one or more of the following ways:

[0393] Set the limit parameters to their default values;

[0394] Setting the restriction parameters to fixed values ​​and encapsulating the set restriction parameters into the encoded bitstream;

[0395] Setting the restriction parameters to variable values ​​and encapsulating the set restriction parameters into the encoded bitstream;

[0396] The values ​​of the restriction parameters are dynamically adjusted during the encoding process, and the dynamically adjusted restriction parameters are encapsulated into the encoded bitstream formed by the encoding.

[0397] In one embodiment, when the restriction parameter includes only one parameter, the unit of the restriction parameter includes any of the following: points, groups;

[0398] When the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: point number, group number, and multiple relationship.

[0399] In one embodiment, the restriction parameter is used to determine the encoding restriction point threshold; if the restriction parameter is a cache restriction parameter, the encoding restriction point threshold is the cache restriction point threshold; if the restriction parameter is a delay restriction parameter, the encoding restriction point threshold is the delay restriction point threshold;

[0400] If the limit parameter is in points and the value of the limit parameter is X1, then the coding limit point threshold is the actual total number of intermediate coefficients of the X1 points to be coded;

[0401] If the restriction parameter is in groups and the value of the restriction parameter is X2, then the coding restriction point threshold is the actual total number of intermediate coefficients of the points to be coded in the X2 group of point cloud groups;

[0402] If the limit parameter is in groups, and the value of the limit parameter is X3, and the number of points in each point cloud group is limited to X4, then the encoding limit point threshold is the total number of intermediate coefficients of the points to be encoded in the X3 groups of point cloud groups, and the total number does not exceed the value of X3*X4.

[0403] In one embodiment, the restriction parameter includes a cache restriction parameter and a delay restriction parameter; the cache restriction parameter is used to determine the cache restriction point threshold; the delay restriction parameter is used to determine the delay restriction point threshold;

[0404] If the cache limit parameter is in multiples and the value of the cache limit parameter is Y1, then the cache limit point threshold is Y1 times the delay limit point threshold;

[0405] If the delay limit parameter is in a multiple relationship and the value of the delay limit parameter is Y2, then the delay limit point threshold is Y2 times the cache limit point threshold.

[0406] Based on the same inventive concept, the principles and beneficial effects of the problems solved by the encoding device provided in the embodiment of the present application are similar to the principles and beneficial effects of the problems solved by the point cloud processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0407] Figure 10 A schematic diagram of the structure of a decoding device provided in an embodiment of the present application; the decoding device may refer to a computer device used by a user of point cloud media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smartphone), a VR device (such as a VR helmet, VR glasses, etc.)). Figure 10 As shown, the decoding device includes a receiver 1001, a processor 1002, a memory 1003, and a display / playback device 1004.

[0408] The receiver 1001 is used to implement decoding and transmission interaction with other devices, specifically to implement the transmission of point cloud media between the encoding device and the decoding device. That is, the decoding device receives the relevant media resources of the point cloud media transmitted by the encoding device through the receiver 1001.

[0409] The processor 1002 (also called CPU (Central Processing Unit)) is the processing core of the encoding device. The processor 1002 is suitable for implementing one or more program instructions, and is specifically suitable for loading and executing one or more program instructions to achieve Figure 5and Figure 6 The flow of the point cloud processing method shown.

[0410] The memory 1003 is a memory device in the decoding device, which is used to store programs and media resources. It is understandable that the memory 1003 here can include both the built-in storage medium in the decoding device and the extended storage medium supported by the decoding device. It should be noted that the memory 1003 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located away from the aforementioned processor. The memory 1003 provides a storage space, which is used to store the operating system of the decoding device. In addition, the storage space is also used to store a computer program, which includes program instructions, and the program instructions are suitable for being called and executed by the processor to perform the various steps of the point cloud processing method. In addition, the memory 1003 can also be used to store the three-dimensional image of the point cloud media formed after processing by the processor, the audio content corresponding to the three-dimensional image, and the information required for rendering the three-dimensional image and audio content.

[0411] The display / playback device 1004 is used to output the rendered sound and three-dimensional image.

[0412] See also Figure 10 , the processor 1002 may include a parser 1021, a decoder 1022, a converter 1023 and a renderer 1024; wherein:

[0413] The parser 1021 is used to decapsulate the encapsulated files of the rendering media from the encoding device. Specifically, it decapsulates the media file resources according to the file format requirements of the point cloud media to obtain audio code streams and video code streams; and provides the audio code streams and video code streams to the decoder 1022.

[0414] Decoder 1022 decodes the audio stream to obtain audio content and provides it to the renderer for audio rendering. Decoder 1022 also decodes the video stream to obtain a 2D image. Based on the metadata provided by the media presentation description, if the metadata indicates that the point cloud media has undergone a region encapsulation process, the 2D image is a encapsulated image; if the metadata indicates that the point cloud media has not undergone a region encapsulation process, the planar image is a projected image.

[0415] Converter 1023 is used to convert a 2D image into a 3D image. If the point cloud media has undergone a region packing process, converter 1023 will first perform region unpacking on the packed image to obtain a projected image. This projected image is then reconstructed to obtain a 3D image. If the rendering media has not undergone a region packing process, converter 1023 will directly reconstruct the projected image to obtain a 3D image.

[0416] The renderer 1024 is used to render the audio content and 3D images of the point cloud media. Specifically, the renderer 1024 renders the audio content and 3D images based on the rendering and viewport-related metadata in the media presentation description information. After the rendering is completed, the rendering is output to the display / playback device.

[0417] In an exemplary embodiment, the processor 1002 (specifically, each device included in the processor) executes the instruction by calling one or more instructions in the memory. Figure 5 and Figure 6 Specifically, the memory stores one or more first instructions, which are suitable for being loaded by the processor 1002 and executing the following steps:

[0418] Get the encoded code stream of the point to be decoded in the point cloud;

[0419] Obtaining restriction parameters of the point cloud, where the restriction parameters include one or both of a cache restriction parameter and a delay restriction parameter;

[0420] Decoding the coded code stream of the decoding point; and

[0421] The decoding process is controlled according to the constraint parameters.

[0422] In one embodiment, the restriction parameters for obtaining the point cloud include any one or more of the following:

[0423] Set the limit parameters to their default values;

[0424] Parse the restricted parameters that have been set to fixed values ​​from the encoded bitstream;

[0425] Parse the restricted parameters that have been set as variable values ​​from the encoded bitstream;

[0426] Parse the set dynamically adjusted restriction parameters from the encoded bitstream.

[0427] In one embodiment, the decoding method includes a decoding method based on point numbers. The specific embodiment of the processor 1002 decoding the coded bit stream of the decoding point is as follows:

[0428] Parse the coded bitstream of the point to be decoded to obtain the intermediate coefficients of the point to be decoded; the intermediate coefficients include residuals or transform coefficients;

[0429] Write the intermediate coefficients of the point to be decoded into the cache;

[0430] The intermediate coefficients of the point to be decoded are read from the cache for decoding.

[0431] In one embodiment, the decoding method includes a decoding method based on the number of groups; a specific embodiment of the processor 1002 decoding the coded bit stream of the decoding point is:

[0432] Parse the coded bitstream of the point to be decoded to obtain multiple point cloud groups, each of which contains intermediate coefficients of the point to be decoded; the intermediate coefficients include residuals or transform coefficients;

[0433] Write point cloud groups into the cache;

[0434] The intermediate coefficients of the points to be decoded in the point cloud group are read from the cache in sequence for decoding.

[0435] In one embodiment, the decoding includes run-length decoding; a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameters is as follows:

[0436] Determine the limit point threshold according to the limit parameter, and the limit point threshold is used to limit the number of points to be decoded;

[0437] During the run length decoding process, if the limit flag is read, the count value of the first run length is set to a preset value;

[0438] If the count value of the first run length is greater than zero and is not equal to the limit point threshold, then the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0439] The intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded.

[0440] In one embodiment, the decoding includes run-length decoding; a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameters is as follows:

[0441] Determine the limit point threshold according to the limit parameter, and the limit point threshold is used to limit the number of points to be decoded;

[0442] During run length decoding, a count value of a first run length is read from the encoded bit stream;

[0443] If the count value of the first run length is greater than zero and equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0444] Reading a count value of a second run length from the encoded bitstream, and if the count value of the second run length is greater than zero and not equal to the limit point number threshold, setting the intermediate coefficient of the current point to be decoded to a preset value, and reducing the count value of the second run length by one step size until the count value of the second run length is equal to zero;

[0445] The intermediate coefficient of the point to be decoded read when the count value of the second run length is equal to zero is obtained and decoded.

[0446] In one embodiment, the decoding includes run-length decoding; a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameters is as follows:

[0447] During run length decoding, a count value of a first run length is read from the encoded bit stream;

[0448] If the count value of the first run length is greater than zero, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0449] The intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded.

[0450] In one implementation, the restriction parameter includes a cache restriction parameter; a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameter is as follows:

[0451] Control the caching logic during decoding according to the cache limit parameters.

[0452] In one implementation, a specific example of the processor 1002 controlling the cache logic during the decoding process according to the cache limit parameter is as follows:

[0453] Determine the cache limit point threshold according to the cache limit parameter, and the cache limit point threshold is used to limit the number of points to be decoded that are allowed to be written into the cache;

[0454] Whenever the number of points to be decoded written into the cache reaches a threshold value of the cache limit number, the intermediate coefficients of the points to be decoded are read from the cache for decoding.

[0455] In one embodiment, the decoding method includes a decoding method based on the number of groups. When the number of points to be decoded written into the cache reaches a cache point limit threshold, the processor 1002 controls the reading of intermediate coefficients of the points to be decoded from the cache for decoding. A specific embodiment is as follows:

[0456] Count the total number of points to be decoded contained in each point cloud group in the cache. If the cache includes M point cloud groups and the total number of points to be decoded contained in the M point cloud groups reaches the cache limit point threshold, then control the reading of the intermediate coefficients of the points to be decoded contained in the M point cloud groups from the cache for decoding; or,

[0457] The number of point cloud groups M contained in the cache is counted, and the total number of points to be encoded contained in the cache is calculated based on M and the point limit threshold of the point cloud group. If the total number of points to be encoded contained in the cache reaches the cache limit threshold, the intermediate coefficients of the points to be decoded contained in the M groups of point cloud groups are read from the cache for decoding.

[0458] In one embodiment, the decoding method includes a decoding method based on the number of groups; a specific embodiment in which the processor 1002 controls the cache logic in the decoding process according to the cache limit parameter is as follows:

[0459] Determine the threshold number of point cloud groups allowed to be written into the cache according to the cache limit parameter;

[0460] Whenever the number of point cloud groups written into the cache reaches a group number threshold, the intermediate coefficients of the to-be-decoded points contained in the point cloud groups written into the cache are read from the cache for decoding.

[0461] In one embodiment, the computer program in the memory 1003 is loaded by the processor 1002 and further performs the following steps:

[0462] Arrange the point cloud groups written into the cache according to the preset arrangement;

[0463] The preset arrangement mode includes any of the following: fixed arrangement mode and dynamic arrangement mode; the fixed arrangement mode means that the arrangement mode of the intermediate coefficients of the point cloud grouping in the cache is fixed, and the dynamic arrangement mode means that the arrangement mode of the intermediate coefficients of the point cloud grouping in the cache is variable.

[0464] In one embodiment, each arrangement corresponds to an index value; the computer program in the memory 1003 is loaded by the processor 1002 and further performs the following steps:

[0465] The index value is parsed from the encoded code stream, and the arrangement of the point cloud groups in the cache is determined according to the index value.

[0466] In one implementation, the restriction parameter includes a delay restriction parameter; and a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameter is as follows:

[0467] The delay logic in the decoding process is controlled according to the delay limit parameters.

[0468] In one implementation, the restriction parameters include a cache restriction parameter and a delay restriction parameter; a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameters is as follows:

[0469] The decoding logic in the decoding process is controlled according to the buffer limit parameters and the delay limit parameters.

[0470] In one embodiment, the restriction parameter includes one or both of a buffer restriction parameter and a delay restriction parameter;

[0471] When the restriction parameter includes only one parameter, the unit of the restriction parameter includes any one or more of the following: points, groups;

[0472] When the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: point number, group number, and multiple relationship.

[0473] Based on the same inventive concept, the principles and beneficial effects of the decoding device provided in the embodiment of the present application in solving the problem are similar to the principles and beneficial effects of the point cloud processing method in the method embodiment of the present application in solving the problem. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0474] An embodiment of the present application also provides a computer-readable storage medium, in which one or more instructions are stored, and the one or more instructions are suitable for being loaded by a processor and executing the point cloud processing method of the above method embodiment.

[0475] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the point cloud processing method of the above method embodiment.

[0476] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described point cloud processing method.

[0477] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0478] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0479] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0480] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made in accordance with the claims of this application are still within the scope covered by the application.

Claims

1. A point cloud processing method, characterized in that: include: Get the encoded code stream of the point cloud; If the coded code stream encapsulates the restriction parameters of the point cloud, the restriction parameters of the point cloud are parsed from the coded code stream; the restriction parameters of the point cloud parsed from the coded code stream include fixed values ​​or variable values; Decoding the encoded code stream and controlling the decoding process according to the constraint parameters of the point cloud; The decoding includes run-length decoding; and the controlling the decoding process according to the restriction parameters of the point cloud includes: Determine a limit point threshold according to the limit parameter, wherein the limit point threshold is used to limit the number of points to be decoded; During the run decoding process, if a limit flag is read, the first run length is parsed to obtain a count value of the first run length; If the count value of the first run length is greater than zero and is not equal to the limit point number threshold, setting the intermediate coefficient of the current to-be-decoded point to a specific value and reducing the count value of the first run length by one step until the count value of the first run length is equal to zero; The intermediate coefficient of the point to be decoded, which is read when the count value of the first run length is equal to zero, is obtained and decoded.

2. The method according to claim 1, wherein The restriction parameter is used to limit the number threshold of points to be decoded during the decoding process; the restriction parameter includes one or both of a buffer restriction parameter and a delay restriction parameter; The cache limit parameter is used to determine a cache limit point threshold, and the number of points to be decoded during the decoding process is less than or equal to the cache limit point threshold; the cache limit parameter is also used to determine a group number threshold, and the number of point cloud groups during the decoding process is less than or equal to the group number threshold; The delay limit parameter is used to determine a delay limit point threshold; the delay limit point threshold is used to limit a threshold value of the number of to-be-decoded points that are allowed to be delayed during the decoding process.

3. The method according to claim 1, wherein The decoding method includes a decoding method based on points; and the decoding of the encoded code stream includes: Parsing the encoded code stream to obtain intermediate coefficients of the points to be decoded in the point cloud; the intermediate coefficients include residuals or transform coefficients; Decode the intermediate coefficients of the points to be decoded.

4. The method according to claim 3, wherein The restriction parameters include cache restriction parameters; and controlling the decoding process according to the restriction parameters of the point cloud includes: Determine a cache limit point threshold according to the cache limit parameter; Whenever the number of points to be decoded in the decoding process reaches the buffer limit point number threshold, the intermediate coefficients of the points to be decoded are controlled to be decoded.

5. A point cloud processing method, characterized in that: include: Setting the restriction parameters of the point cloud; if the restriction parameters of the point cloud are set to fixed values ​​or variable values, encapsulating the restriction parameters of the point cloud into the encoded code stream; Obtaining attribute information of the points to be encoded in the point cloud; Encoding the attribute information of the point to be encoded; and controlling the encoding process according to the restriction parameters; Wherein, the encoding includes run length encoding; if the restriction parameter includes a delay restriction parameter, then controlling the encoding process according to the restriction parameter includes: Determining a delay limit point threshold according to the delay limit parameter; In the process of run length encoding, setting a run length for counting consecutive specific symbols; The run length is set to an initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded: If an intermediate coefficient having a specific symbol is read, the run length count value is increased by one step; the run length count value is obtained, and if the run length count value reaches the delay limit point threshold, parameter encoding processing is performed, and the run length is reset to the initial value for re-counting; If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting.

6. The method according to claim 5, wherein If the restriction parameter includes a cache restriction parameter and the encoding includes run length encoding, then controlling the encoding process according to the restriction parameter includes: Determine a cache limit point threshold according to the cache limit parameter; In the run length encoding process, setting the run length used to count consecutive specific symbols; The run length is set to an initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded: If an intermediate coefficient with a specific symbol is read, the count value of the run length is increased by one step; if the number of points to be encoded read is equal to the cache limit point threshold, the current run length is encoded, or the cache limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is set to the initial value for recounting; If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting.

7. The method according to claim 5, wherein The parameter encoding process refers to: encoding the current run length; or encoding a restriction identifier, which is used to set the run length to a preset value; or encoding the delay limit point threshold and the intermediate coefficients of the to-be-encoded points whose values ​​are specific symbols.

8. The method according to any one of claims 5 to 7, wherein: The point cloud restriction parameters may include any one or more of the following: Setting said limiting parameters to default values; Setting the limiting parameter to a fixed value; Setting the limit parameter to a variable value; The value of the restriction parameter is dynamically adjusted during the encoding process.

9. A point cloud processing device, characterized in that: The device comprises: An acquisition unit, used to obtain the encoded code stream of the point cloud; The acquisition unit is further configured to parse the point cloud restriction parameters from the encoded code stream if the point cloud restriction parameters are encapsulated in the encoded code stream; the point cloud restriction parameters parsed from the encoded code stream include fixed values ​​or variable values; a processing unit, configured to decode the encoded code stream and control the decoding process according to the restriction parameters of the point cloud; The decoding includes run-length decoding; and the processing unit is configured to control the decoding process according to the restriction parameters of the point cloud, specifically to: Determine a limit point threshold according to the limit parameter, wherein the limit point threshold is used to limit the number of points to be decoded; During the run decoding process, if a limit flag is read, the first run length is parsed to obtain a count value of the first run length; If the count value of the first run length is greater than zero and is not equal to the limit point number threshold, setting the intermediate coefficient of the current to-be-decoded point to a specific value and reducing the count value of the first run length by one step until the count value of the first run length is equal to zero; The intermediate coefficient of the point to be decoded, which is read when the count value of the first run length is equal to zero, is obtained and decoded.

10. A point cloud processing device, characterized in that: The device comprises: a setting unit, configured to set restriction parameters of the point cloud; if the restriction parameters of the point cloud are set to fixed values ​​or variable values, encapsulating the restriction parameters of the point cloud into an encoded code stream; An acquisition unit, configured to acquire attribute information of the points to be encoded in the point cloud; a processing unit, configured to encode the attribute information of the point to be encoded; and to control the encoding process according to the restriction parameter; Wherein, the encoding includes run length encoding; if the restriction parameter includes a delay restriction parameter, the processing unit is configured to control the encoding process according to the restriction parameter, specifically to: Determining a delay limit point threshold according to the delay limit parameter; In the process of run length encoding, setting a run length for counting consecutive specific symbols; The run length is set to an initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded: If an intermediate coefficient having a specific symbol is read, the run length count value is increased by one step; the run length count value is obtained, and if the run length count value reaches the delay limit point threshold, parameter encoding processing is performed, and the run length is reset to the initial value for re-counting; If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting.

11. A computer device, characterized in that: The computer device comprises: a processor suitable for implementing a computer program; A computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by the processor and executing the point cloud processing method according to any one of claims 1 to 4, or the point cloud processing method according to any one of claims 5 to 8.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the point cloud processing method according to any one of claims 1 to 4, or the point cloud processing method according to any one of claims 5 to 8.

13. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the point cloud processing method according to any one of claims 1 to 4, or implements the steps of the point cloud processing method according to any one of claims 5 to 8.

Images