Point cloud processing method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides a point cloud processing method and device, equipment, a storage medium and a product. The method comprises the following steps: when a to-be-coded point in a point cloud needs to be coded, setting a limit parameter of the point cloud, acquiring attribute information of the to-be-coded point in the point cloud, coding the attribute information of the to-be-coded point, and controlling a coding process according to the limit parameter; when the to-be-decoded point in the point cloud needs to be decoded, the encoding code stream of the to-be-decoded point in the point cloud and the limit parameter of the point cloud can be acquired, the encoding code stream of the to-be-decoded point is decoded, and the decoding process is controlled according to the limit parameter. According to the embodiment of the invention, the coding and decoding process of the attribute information of each point in the point cloud is controllable through the limiting parameters of the point cloud, and the coding and decoding performance of the attribute information of the point cloud is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to the field of coding and decoding technology, and in particular to a point cloud processing method, a point cloud processing device, a computer device, a computer-readable storage medium and a point cloud processing product. Background Art

[0002] With the continuous development of science and technology, it is now possible to obtain a large number of high-precision point clouds at a low cost and in a short period of time. The point cloud can include multiple points, and each point in the point cloud has geometric information and attribute information. In order to improve the transmission efficiency of the point cloud, it is usually necessary to encode the point cloud before transmitting it; specifically, after the encoding end encodes the geometric information and attribute information of each point in the point cloud, the encoded point cloud can be transmitted to the decoding end, and the decoding end can decode the encoded point cloud to reconstruct the geometric information and attribute information of each point in the point cloud. In practice, it is found that because the encoding process requires the attribute information of each point to be sorted, the waiting time for the attribute information of each point in the point cloud is long during encoding, and the encoding performance is low. Summary of the invention

[0003] The embodiments of the present application provide a point cloud processing method and related equipment, which can improve the encoding performance of point cloud attribute information.

[0004] On the one hand, an embodiment of the present application provides a point cloud processing method, the point cloud processing method comprising:

[0005] Set the limit parameters of the point cloud, which include one or both of cache limit parameters and delay limit parameters;

[0006] Get the attribute information of the points to be coded in the point cloud;

[0007] Encoding attribute information of the code point to be coded; and,

[0008] The encoding process is controlled according to the constraint parameters.

[0009] In the embodiment of the present application, when it is necessary to encode the points to be encoded in the point cloud, it is possible to set the restriction parameters of the point cloud, obtain the attribute information of the points to be encoded in the point cloud, encode the attribute information of the points to be encoded, and control the encoding process according to the restriction parameters. By setting the restriction parameters of the point cloud, the embodiment of the present application makes the encoding process of the attribute information of the points to be encoded controllable, thereby improving the encoding performance of the point cloud attribute information.

[0010] On the one hand, an embodiment of the present application provides another point cloud processing method, which includes:

[0011] Get the encoded code stream of the point to be decoded in the point cloud;

[0012] Obtaining restriction parameters of the point cloud, the restriction parameters including one or both of a cache restriction parameter and a delay restriction parameter;

[0013] Decoding the coded bitstream of the decoding point; and

[0014] The decoding process is controlled according to the constraint parameters.

[0015] In the embodiment of the present application, when it is necessary to decode the points to be decoded in the point cloud, the encoding stream of the points to be decoded in the point cloud and the restriction parameters of the point cloud can be obtained, the encoding stream of the points to be decoded can be decoded, and the decoding process can be controlled according to the restriction parameters. By obtaining the restriction parameters of the point cloud, the embodiment of the present application makes the attribute information decoding process of the points to be decoded controllable, thereby improving the decoding performance of the point cloud attribute information.

[0016] On the one hand, an embodiment of the present application provides a point cloud processing device, the point cloud processing device comprising:

[0017] A setting unit, used to set limit parameters of the point cloud, the limit parameters including one or both of a cache limit parameter and a delay limit parameter;

[0018] An acquisition unit, used to acquire attribute information of points to be encoded in the point cloud;

[0019] The processing unit is used to encode the attribute information of the coding point; and control the coding process according to the restriction parameters.

[0020] On the one hand, an embodiment of the present application provides another point cloud processing device, the point cloud processing device comprising:

[0021] An acquisition unit, used to acquire a coded code stream of a point to be decoded in the point cloud, and to acquire a restriction parameter of the point cloud, wherein the restriction parameter includes one or both of a cache restriction parameter and a delay restriction parameter;

[0022] The processing unit is used to decode the coded code stream of the decoding point; and control the decoding process according to the restriction parameters.

[0023] Accordingly, an embodiment of the present application provides a computer device, which includes a processor and a computer-readable storage medium; wherein the processor is suitable for implementing a computer program, the computer-readable storage medium stores the computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned point cloud processing method.

[0024] Accordingly, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned point cloud processing method.

[0025] Accordingly, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned point cloud processing method.

[0026] In an embodiment of the present application, when it is necessary to encode the points to be encoded in the point cloud, it is possible to set restriction parameters of the point cloud, obtain attribute information of the points to be encoded in the point cloud, encode the attribute information of the points to be encoded, and control the encoding process according to the restriction parameters. When it is necessary to decode the points to be decoded in the point cloud, it is possible to obtain the encoding code stream of the points to be decoded in the point cloud and the restriction parameters of the point cloud, decode the encoding code stream of the points to be decoded, and control the decoding process according to the restriction parameters. The embodiment of the present application makes the encoding and decoding process of the attribute information of each point in the point cloud controllable through the restriction parameters of the point cloud, thereby improving the encoding and decoding performance of the point cloud attribute information. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1a A schematic diagram of a coding framework provided in an embodiment of the present application;

[0029] Figure 1b A schematic diagram of the relationship between a block where a current point is located and its parent block neighbors provided in an embodiment of the present application;

[0030] Figure 2 An architecture diagram of a point cloud processing system provided in an embodiment of the present application;

[0031] Figure 3 A point cloud processing method provided in an embodiment of the present application;

[0032] Figure 4 Another point cloud processing method provided in an embodiment of the present application;

[0033] Figure 5 Another point cloud processing method provided in an embodiment of the present application;

[0034] Figure 6 Another point cloud processing method provided in the embodiment of the present application;

[0035] Figure 7 A schematic diagram of the structure of a point cloud processing device provided in an embodiment of the present application;

[0036] Figure 8 A schematic diagram of the structure of another point cloud processing device provided in an embodiment of the present application;

[0037] Fig. 9 A schematic diagram of the structure of an encoding device provided in an embodiment of the present application;

[0038] Fig.10 A schematic diagram of the structure of a decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] In order to more clearly understand the technical solution provided by the embodiments of the present application, the key terms involved in the embodiments of the present application are first introduced here:

[0041] (1) Point Cloud. A point cloud is a set of irregularly distributed discrete points in space that express the spatial structure and surface properties of a three-dimensional object or scene. Point clouds can be divided into different categories according to different classification standards. For example, according to the way point clouds are acquired, they can be divided into dense point clouds and sparse point clouds. For example, according to the time series type of point clouds, they can be divided into static point clouds and dynamic point clouds.

[0042] (2) Point Cloud Data. The geometric information and attribute information of each point in the point cloud together constitute the point cloud data. Geometric information can also be called three-dimensional position information. The geometric information of a point in the point cloud refers to the spatial coordinates (x, y, z) of the point, which can include the coordinate values ​​of the point in each coordinate axis direction of the three-dimensional coordinate system, for example, the coordinate value x in the X-axis direction, the coordinate value y in the Y-axis direction, and the coordinate value z in the Z-axis direction. The attribute information of a point in the point cloud may include at least one of the following: color information, material information, laser reflection intensity information (also called reflectivity); usually, each point in the point cloud has the same amount of attribute information. For example, each point in the point cloud can have two kinds of attribute information: color information and laser reflection intensity. For another example, each point in the point cloud can have three kinds of attribute information: color information, material information, and laser reflection intensity information.

[0043] (3) Point Cloud Compression (PCC). Point cloud coding refers to the process of encoding the geometric information and attribute information of each point in the point cloud to obtain a compressed code stream. Point cloud coding can include two main processes: geometric information encoding and attribute information encoding. The current mainstream point cloud coding technology can be divided into point cloud coding based on geometry structure and point cloud coding based on projection according to different types of point clouds. Here, G-PCC (Geometry-based Point Cloud Compression) in MPEG (Moving Picture Expert Group, an international audio and video coding standard) and AVS-PCC, a point cloud coding standard in AVS (Audio Video Coding Standard, China's national video coding standard) are used as examples for introduction.

[0044] The coding frameworks of G-PCC and AVS-PCC are roughly the same, such as Figure 1a As shown in FIG, it can be divided into a geometric information encoding process and an attribute information encoding process. The geometric information encoding process encodes the geometric information of each point in the point cloud data to obtain a geometric bit stream; the attribute information encoding process encodes the attribute information of each point in the point cloud data to obtain an attribute bit stream; the geometric bit stream and the attribute bit stream together constitute the compressed code stream of the point cloud data.

[0045] For the geometric information encoding process, the main operations and processing can be described as follows:

[0046] ① Pre-Processing: It can include coordinate transformation and voxelization. Through scaling and translation operations, the point cloud data in the three-dimensional space is converted into integer form, and its minimum geometric position is moved to the coordinate origin.

[0047] ② Geometric octree encoding: Octree is a tree data structure. In the three-dimensional space division, the pre-set bounding box is evenly divided, and each node has eight child nodes. By using "1" and "0" to indicate whether each child node of the octree is occupied or not, the occupancy code information (Occupancy Code) is obtained as the code stream of the point cloud geometry information. The bounding box is an algorithm for solving the optimal bounding space of a discrete point set. The basic idea is to use a geometric body with a slightly larger volume and simpler characteristics (called a bounding box) to approximately replace complex geometric objects.

[0048] ③ Geometry Entropy Encoding: Statistical compression encoding is performed on the occupancy code information of the octree, and finally a binary (0 or 1) compressed code stream is output. Statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. The commonly used statistical coding method is context-based binary arithmetic coding (Content Adaptive Binary Arithmetic Coding, CABAC).

[0049] For the attribute information encoding process, the main operations and processing can be seen in the following description:

[0050] ① Attribute recoloring: In the case of lossy coding, after the geometric information is encoded, the encoder needs to decode and reconstruct the geometric information, that is, restore the geometric information of each point in the point cloud. Find the attribute information corresponding to one or more neighboring points in the original point cloud as the attribute information of the reconstructed point.

[0051] ② Attribute transform (Transform attributes) encoding: Use attribute transform algorithm (such as DCT, Haar, etc.) to group and transform attribute information and quantize transform coefficients; obtain attribute reconstruction value through inverse quantization and inverse transformation; calculate the difference between the original attribute and the attribute reconstruction value to obtain the attribute residual and quantize it; encode the quantized transform coefficients and attribute residual.

[0052] ③Attribute Quantization: The degree of quantization is usually determined by the quantization parameter. In attribute prediction coding, entropy coding is performed on the quantized prediction residual information; in attribute transform coding and attribute prediction transform coding, entropy coding is performed on the quantized transform coefficients.

[0053] ④ Attribute Entropy Coding: The quantized prediction residual information or transform coefficients are generally compressed using Run Length Coding and Arithmetic Coding. The corresponding coding mode, quantization parameters and other information are also encoded using entropy encoders.

[0054] (4) Point cloud decoding. Point cloud decoding refers to the process of decoding the compressed bitstream obtained by point cloud encoding to reconstruct the point cloud; in detail, it refers to the process of reconstructing the geometric information and attribute information of each point in the point cloud based on the geometric bitstream and attribute bitstream in the compressed bitstream. After obtaining the compressed bitstream at the decoding end, the geometric bitstream is first entropy decoded to obtain the quantized geometric information of each point in the point cloud, and then inverse quantization is performed to reconstruct the geometric information of each point in the point cloud. As for the attribute bitstream, entropy decoding is first performed to obtain the quantized prediction residual information or quantized transform coefficient of each point in the point cloud; then the quantized prediction residual information is inversely quantized to obtain the reconstructed residual information, and the quantized transform coefficient is inversely quantized to obtain the reconstructed transform coefficient. The reconstructed transform coefficient is inversely transformed to obtain the reconstructed residual information. According to the reconstructed residual information of each point in the point cloud, the attribute information of each point in the point cloud can be reconstructed. The reconstructed attribute information of each point in the point cloud is matched one by one with the reconstructed geometric information in order to reconstruct the point cloud.

[0055] In addition, this application also involves the following technologies:

[0056] (1) Attribute prediction:

[0057] ① Point cloud reordering, the specific process is as follows:

[0058] Get the coordinates (x, y, z) of the point cloud, generate the Morton code or Hilbert code of each point based on the space filling curve, and sort them according to the encoding order of each point (such as sorting from small to large) to obtain the Morton order or Hilbert order. Taking the Morton code of geometric coordinates as an example, the geometric position of the point in the point cloud is represented by three-dimensional Cartesian coordinates (X, Y, Z). Use N bits to represent each coordinate value, and the coordinate of the k-th point (X k ,Y k ,Z k ) can be expressed as:

[0059]

[0060] The Morton code corresponding to the k-th point can be expressed as follows:

[0061]

[0062] Represent each three bits in octal. n=0,1,…,N-1, then the Morton code corresponding to the k-th point can be expressed as:

[0063]

[0064] ②Sequence-based neighbor point selection, the specific process is as follows:

[0065] The order-based neighbor point selection includes the distance-based selection method and the spatial relationship and distance-based selection method. The above methods are introduced below:

[0066] i) Distance-based method, the specific steps are as follows:

[0067] Among the first P neighbor candidate points in the Morton order or Hilbert order, P can be determined by the maximum number of neighbors field (maxNumOfNeighbours), and each neighbor candidate point (the i-th neighbor candidate point is represented by (x i ,y i ,z i )), the Manhattan distance d to the current decoding point (x, y, z) = | i |+|yy i |+|zz i |. After obtaining the Manhattan distance from each neighbor candidate point to the current point to be decoded (x, y, z), the k neighbor candidate points with the shortest Manhattan distance can be used as neighbors of the current point to be decoded; or, the maximum distance value among the k points with the shortest Manhattan distance is determined, and all neighbor candidate points with Manhattan distances less than or equal to the maximum distance value are used as neighbors of the current point, and finally the m points with the closest Manhattan distances are determined as the nearest neighbors of the current point to be decoded, where m and k are positive integers and m≤k.

[0068] ii) Selection method based on spatial relationship and distance. The specific steps are as follows:

[0069] First, determine the initial block size, that is, determine the initial right shift number N of the codeword corresponding to each point. On this basis, N+3 is the corresponding parent block range.

[0070] Then, the point cloud is traversed in a certain order, such as Figure 1bAs shown, the current point to be decoded (point P) searches for the nearest neighbors of the decoded points (limited to the first k points) within the parent block of the B block where it is located and the neighboring blocks that are coplanar, colinear, and co-point with it.

[0071] If the number of neighbor points of the current point to be decoded determined by the above method is insufficient, the neighbor point selection is performed according to the distance-based method in i).

[0072] ③ Calculation of predicted value, the specific process is as follows:

[0073] Let the attribute reconstruction value of each neighbor be j=0,1…k, then the attribute prediction value of the current point for:

[0074]

[0075] i) Weighted calculation based on distance:

[0076] In one implementation, the inverse of the Manhattan distance between the current point to be decoded and the neighboring point is used as the weight, and finally the weighted average of the attribute reconstruction values ​​of the k neighboring points is calculated to obtain the attribute prediction value. Assume that the geometric coordinates of the current point to be decoded are (x i ,y i , z i ), the geometric coordinates of the jth neighbor point are (x ij ,y ij , z ij ), then the weight of the j-th neighbor point is w ij for:

[0077]

[0078] In another embodiment, if the components in the x, y, and z directions use different weights, the weight w of the jth neighbor point is ij for:

[0079]

[0080] ii) Weighted calculation based on distance and other parameters:

[0081] The weight of each neighbor point of the current point to be decoded is w=1 / d, and the optimization weight of the neighbor candidate point whose distance is equal to the maximum distance value is wm=(1 / d)*dwm, where the size of dwm is the minimum value between Qstep (attribute quantization step) and the number of neighbor candidate points whose distance is equal to the maximum distance value.

[0082] (2) Attribute transformation:

[0083] The core of attribute transformation is attribute encoding that combines prediction and transformation. The specific process is as follows:

[0084] First, the point cloud to be encoded after color space conversion and recoloring is reordered from small to large according to the Hilbert code, and then adaptive grouping and corresponding K-ary transformation are used. The specific process is as follows:

[0085] i) Point cloud grouping:

[0086] The point cloud is grouped in sequence based on the order of the space filling curve, and the points with the same first L bits of the Hilbert code are grouped together. When the number of points in the same group exceeds the preset limit, the group is subdivided. In addition, the grouping can also be adjusted according to the number of grouping points in the previous group.

[0087] ii) In-group prediction:

[0088] The prediction point selection range is the P points before the first point of the current group, where P can be determined by the maximum number of neighbors field (maxNumOfNeighbours). These P points are predicted point by point, and a unified prediction value is selected or calculated for each group. For example, the prediction value of the first point of the current group is used as the prediction value of the current group.

[0089] iii) Transformation based on residual value:

[0090] The predicted attribute residual value of each group of points is subjected to K_i-element discrete cosine transform (DCT for Discrete Cosine Transform, DCT transform), where K_i=2…8, and the transform coefficients are quantized and entropy coded. When K_i=1, no transform calculation is required, and the attribute residual value is directly quantized and entropy coded. The obtained transform coefficients are quantized.

[0091] (3) Entropy coding:

[0092] Binarize and process the signed attribute prediction residual or transform coefficient after quantization (lossy case). Entropy coding can adopt variable length coding, context-based adaptive binary arithmetic coding, run length coding, etc. Specifically:

[0093] ①Variable length encoding:

[0094] Codewords of different lengths are used to represent the residual or coefficients that need to be encoded. The code length needs to be designed based on the probability of the symbol appearing. Commonly used variable-length codes include exponential Golomb coding (exp-Golomb) and arithmetic coding.

[0095] ②Context-based adaptive binary arithmetic coding (CABAC), mainly includes the following steps:

[0096] Binarization: CABAC uses binary arithmetic coding, which means that only two numbers (1 or 0) are encoded. A non-binary numerical symbol, such as a conversion coefficient or motion vector, is first binarized or converted into a binary codeword before arithmetic coding. This process is similar to converting a numerical value into a variable-length codeword. In the specific implementation process, this binary codeword is further encoded by an arithmetic encoder before transmission.

[0097] Context model selection: The context model is a probabilistic model that is selected based on the statistics of the most recent coded data symbols. This model stores the probability of each 'bin' being 1 or 0.

[0098] Arithmetic Coding: The arithmetic encoder encodes each 'bin' according to the chosen probability model.

[0099] Probability update: The selected context model is updated according to the actual encoded value. For example, if the value of 'bin' is 1, the frequency count of 1 is increased.

[0100] ③Run-length coding:

[0101] The number of points in the statistics that are consecutively a specific symbol is recorded as the run length run-length. If the current symbol is zero, the run-length value increases by 1; if it is non-zero, the run_length value is first encoded, then the non-zero attribute prediction residual is encoded, and finally the run-length value is set to 0 to restart counting. The residual can use common entropy coding methods, such as variable length coding or CABAC.

[0102] Specifically, run length coding is divided into fixed-length run length coding and variable-length run length coding. Fixed-length run length coding means that the number of binary bits used for the encoded run length is fixed. Variable-length run length coding means that different ranges of run lengths are encoded using different numbers of binary bits, and a flag bit needs to be added to indicate the number of binary bits used.

[0103] Based on the above descriptions of point cloud, point cloud data, point cloud encoding, point cloud decoding, attribute prediction, attribute transformation and entropy coding, the embodiment of the present application provides a point cloud processing solution. When it is necessary to encode the points to be encoded in the point cloud, the restriction parameters of the point cloud can be set, the attribute information of the points to be encoded in the point cloud can be obtained, the attribute information of the points to be encoded can be encoded, and the encoding process can be controlled according to the restriction parameters. When it is necessary to decode the points to be decoded in the point cloud, the encoding code stream of the points to be decoded in the point cloud and the restriction parameters of the point cloud can be obtained, the encoding code stream of the points to be decoded can be decoded, and the decoding process can be controlled according to the restriction parameters. The embodiment of the present application makes the encoding and decoding process of the attribute information of each point in the point cloud controllable through the restriction parameters of the point cloud, thereby improving the encoding and decoding performance of the point cloud attribute information.

[0104] The point cloud processing solution provided in the embodiment of the present application can also be combined with cloud computing, cloud storage and other technologies in cloud technology. Among them, cloud computing is a computing mode that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. Cloud computing can provide powerful computing support for the encoding stage and decoding stage of point cloud attributes, which can greatly improve the encoding efficiency of point cloud attributes and the decoding efficiency of point cloud attributes. Cloud storage is a new concept extended and developed from the concept of cloud computing. The distributed cloud storage system (hereinafter referred to as the storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide data storage and business access functions to the outside world. Cloud storage can provide powerful storage support for the encoding stage and decoding stage of point cloud attributes, which can further improve the encoding efficiency of point cloud attributes and the decoding efficiency of point cloud attributes.

[0105] In addition, the point cloud processing solution provided in the embodiment of the present application can also be combined with artificial intelligence. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0106] AI technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, large application processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0107] Machine learning is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of AI and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning / deep learning usually includes artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching.

[0108] Deep learning: The concept of deep learning originates from the study of artificial neural networks. A multilayer perceptron with multiple hidden layers is a deep learning structure. Deep learning combines low-level features to form more abstract high-level representation attribute categories or features to discover distributed feature representations of data. The embodiments of the present application mainly involve optimizing the training of the model used in the encoding and decoding process through a training data set. For example, the initial model can be trained with a training data set to obtain a context model.

[0109] Based on the above description, the following Figure 2 A point cloud processing system suitable for implementing the point cloud processing solution provided in the embodiment of the present application is introduced. Figure 2 As shown, the point cloud processing system 20 may include an encoding device 201 and a decoding device 202. The encoding device 201 may be a terminal or a server, and the decoding device 202 may be a terminal or a server. A communication connection may be established between the encoding device 201 and the decoding device 202. The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, a smart TV, etc., but is not limited thereto. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0110] (1) For the encoding device 201:

[0111] The encoding device 201 can obtain point cloud data (i.e., geometric information and attribute information of each point in the point cloud), and the point cloud data can be obtained by scene capture or device generation. Scene capture point cloud data refers to point cloud data obtained by collecting real-world visual scenes through the capture device associated with the encoding device 201; wherein the capture device is used to provide point cloud data acquisition services for the encoding device 201, and the capture device may include but is not limited to any of the following: camera equipment, sensor equipment, scanning equipment; wherein the camera equipment may include ordinary cameras, stereo cameras, and light field cameras, etc., the sensor equipment may include laser equipment, radar equipment, etc., and the scanning equipment may include three-dimensional laser scanning equipment, etc.; the capture device associated with the encoding device 201 may refer to a hardware component set in the encoding device 201, such as a camera, sensor, etc. of a terminal, and the capture device associated with the encoding device may also refer to a hardware device connected to the encoding device 201, such as a camera connected to a server, etc. Device generation of point cloud data refers to the point cloud data generated by the encoding device 201 according to virtual objects (such as virtual three-dimensional objects and virtual three-dimensional scenes obtained through three-dimensional modeling).

[0112] The encoding device 201 can encode the geometric information of each point in the point cloud to obtain a geometric bit stream (including the geometric encoding information of each point in the point cloud), and can encode the attribute information of each point in the point cloud to obtain an attribute bit stream (including the attribute encoding information of each point in the point cloud). In particular, when the encoding device 201 encodes the attribute information of the points to be encoded in the point cloud, it can set the restriction parameters of the point cloud, and encode the acquired attribute information of the points to be encoded based on the restriction parameters of the point cloud, thereby controlling the encoding process. After obtaining the compressed code stream of the point cloud, the encoding device 201 transmits the compressed code stream (i.e., the attribute bit stream and the geometric bit stream) to the decoding device 202.

[0113] (2) For the decoding device 202:

[0114] After receiving the compressed code stream (i.e., the attribute bit stream and the geometry bit stream) transmitted by the encoding device 201, the decoding device 202 can decode the geometry bit stream to reconstruct the geometry information of each point in the point cloud, and can decode the attribute bit stream to obtain the attribute information of each point in the point cloud. In particular, when the decoding device 202 decodes the points to be decoded in the point cloud, it can obtain the restriction parameters of the point cloud, and decode the obtained encoded code stream of the points to be decoded based on the restriction parameters of the point cloud, thereby controlling the decoding process. After obtaining the reconstructed geometry information and the reconstructed attribute information of each point in the point cloud, the decoding device 202 matches the reconstructed geometry information and the reconstructed attribute information of each point in the point cloud one by one to reconstruct the point cloud.

[0115] In the embodiment of the present application, when it is necessary to encode the points to be encoded in the point cloud, the restriction parameters of the point cloud can be set, the attribute information of the points to be encoded in the point cloud can be obtained, the attribute information of the points to be encoded can be encoded, and the encoding process can be controlled according to the restriction parameters. When it is necessary to decode the points to be decoded in the point cloud, the encoding code stream of the points to be decoded in the point cloud and the restriction parameters of the point cloud can be obtained, the encoding code stream of the points to be decoded can be decoded, and the decoding process can be controlled according to the restriction parameters. In the embodiment of the present application, the restriction parameters of the point cloud are used to make the encoding and decoding process of the attribute information of each point in the point cloud controllable, thereby improving the encoding and decoding performance of the point cloud attribute information. It can be understood that the point cloud processing system described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. It can be known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0116] Based on the above description of the point cloud processing system, the point cloud processing solution provided in the embodiment of the present application is introduced in more detail below in combination with the accompanying drawings.

[0117] See also Figure 3 , Figure 3 A point cloud processing method provided in an embodiment of the present application can be executed by a computer device, which can be specifically Figure 2 The encoding device 201 in the point cloud processing system 20 shown. Figure 3 As shown, the point cloud processing method may include the following steps S301-S303:

[0118] S301, setting restriction parameters of the point cloud.

[0119] The restriction parameter is used to set the trigger condition for encoding the attribute information of the points to be encoded in the point cloud. The restriction parameter may include at least one of a cache restriction parameter and a delay restriction parameter; wherein the cache restriction parameter is used to control the cache logic in the encoding process, for example, when the number of intermediate coefficients corresponding to the attribute information of the points to be encoded in the cache queue (such as the residual, transform coefficient, etc. corresponding to the attribute information of the points to be encoded) meets the trigger condition set based on the cache restriction parameter (such as the number of intermediate coefficients corresponding to the attribute information of the points to be encoded in the cache queue reaches the cache threshold indicated by the cache parameter), the computer device (encoding device) encodes the intermediate coefficients corresponding to the attribute information of the points to be encoded in the cache queue; similarly, the delay restriction parameter is used to control the encoding logic in the encoding process, for example, when the delayed number of intermediate coefficients corresponding to the attribute information of the points to be encoded meets the trigger condition set based on the delay restriction parameter (such as the delayed number of intermediate coefficients corresponding to the attribute information of the points to be encoded reaches the delay threshold indicated by the delay parameter), the computer device (encoding device) encodes the intermediate coefficients corresponding to the attribute information of the points to be encoded, and the delay number is used to indicate the number of intermediate coefficients corresponding to the attribute information of the current points to be encoded.

[0120] The setting methods of the restriction parameters include any one or more of the following: setting the restriction parameters to default values; setting the restriction parameters to fixed values, and encapsulating the set restriction parameters into the encoding code stream formed by encoding; setting the restriction parameters to variable values, and encapsulating the set restriction parameters into the encoding code stream formed by encoding, for example, setting the restriction parameters to a in encoding code stream 1, setting the restriction parameters to b in encoding code stream 2, and setting the restriction parameters to c in encoding code stream 3, and a, b, and c are different from each other; dynamically adjusting the value of the restriction parameters during the encoding process, and encapsulating the dynamically adjusted restriction parameters into the encoding code stream formed by encoding; taking the encoding as run encoding as an example, the computer device can count the lengths of the first Q runs, and dynamically adjust the value of the restriction parameters according to the lengths of the first Q runs, and encapsulate the dynamically adjusted restriction parameters into the encoding code stream formed by encoding. It can be understood that if the restriction parameters are not set, the restriction parameters are default values.

[0121] S302: Obtain attribute information of the points to be encoded in the point cloud.

[0122] The attribute information of the point to be coded may include at least one of the following: color information, material information, laser reflection intensity information (also known as reflectivity); usually, each point in the point cloud has the same amount of attribute information, for example, each point in the point cloud may have two kinds of attribute information, color information and laser reflection intensity, and for another example, each point in the point cloud may have three kinds of attribute information, color information, material information and laser reflection intensity information. The attribute information of the point to be coded may be generated based on data collected by a collection device (such as a camera, a 3D laser scanner), or may be directly generated by a computer based on virtual three-dimensional objects and scenes.

[0123] S303: Encode the attribute information of the point to be encoded, and control the encoding process according to the restriction parameters.

[0124] The method of encoding the attribute information of the coded point includes: encoding the attribute information of the coded point in units of point number, and encoding the attribute information of the coded point in units of group number. Encoding in units of point number means encoding a point in the point cloud as a basic encoding unit; encoding in units of group number means encoding a group in the point cloud (a group may include at least one point in the point cloud) as a basic encoding unit.

[0125] In one embodiment, a computer device encodes the attribute information of a to-be-encoded point in units of points as follows: converting the attribute information of the to-be-encoded point to obtain an intermediate coefficient of the to-be-encoded point; wherein, converting the attribute information of the to-be-encoded point may be: predicting the attribute information of the to-be-encoded point to obtain a residual between the predicted information of the to-be-encoded point and the attribute information of the to-be-encoded point; or may be: after obtaining the residual, transforming the residual to obtain a transformation coefficient; that is, converting the attribute information of the to-be-encoded point to obtain an intermediate coefficient of the to-be-encoded point may be a residual or a transformation coefficient.

[0126] After obtaining the intermediate coefficients of the points to be encoded, the computer device can write the intermediate coefficients of the points to be encoded into the cache, and read the intermediate coefficients of the points to be encoded from the cache for encoding; for example, the computer device can write the intermediate coefficients of the points to be encoded into a cache queue set based on the cache limit parameters. When the cache queue is full (such as the intermediate coefficients of the points to be encoded stored in the cache queue reach the cache threshold), the computer device reads the intermediate coefficients of the points to be encoded from the cache queue for encoding. The computer device can also obtain the delayed number of the intermediate coefficients of the current point to be encoded. If the delayed number of the intermediate coefficients of the current point to be encoded reaches the delay threshold, the computer device encodes the intermediate coefficients of the current point to be encoded. The specific implementation method of the computer device encoding the intermediate coefficients of the points to be encoded can refer to the above-mentioned implementation method of point cloud encoding, which will not be repeated here.

[0127] In another embodiment, the computer device encodes the attribute information of the points to be encoded in units of groups as follows: the attribute information of the points to be encoded is grouped to obtain multiple groups of point cloud groups. The basis for grouping can be set according to the actual encoding requirements. In one embodiment, the computer device can group the attribute information of the points to be encoded based on the order of the space filling curve; for example, the computer device can group the attribute information of the points to be encoded with the same first L bits of the Hilbert code into one group. After obtaining multiple groups of point cloud groups, the computer device can convert the attribute information of the points to be encoded in the point cloud groups to obtain converted point cloud groups; the specific implementation method of converting the attribute information of the points to be encoded in the point cloud groups can refer to the previous embodiment, which will not be repeated here. The converted point cloud groups contain intermediate coefficients of the points to be encoded, and the intermediate coefficients include residuals or transformation coefficients.

[0128] After obtaining the converted point cloud grouping, the computer device may write the converted point cloud grouping into the cache, and sequentially read the intermediate coefficients of the points to be encoded in the converted point cloud grouping from the cache for encoding; for example, the computer device may write the converted point cloud grouping into a cache queue set based on the cache limit parameter, and when the cache queue is full (such as the converted point cloud groupings stored in the cache queue reaches the cache threshold), the computer device reads the converted point cloud grouping from the cache queue for encoding. The computer device may also obtain the delay number of the current converted point cloud grouping. If the delay number of the current converted point cloud grouping reaches the delay threshold, the computer device encodes the current converted point cloud grouping. The specific implementation method of the computer device encoding the converted point cloud grouping may refer to the above-mentioned implementation method of point cloud encoding, which will not be repeated here.

[0129] In the embodiment of the present application, when it is necessary to encode the points to be encoded in the point cloud, it is possible to set the restriction parameters of the point cloud, obtain the attribute information of the points to be encoded in the point cloud, encode the attribute information of the points to be encoded, and control the encoding process according to the restriction parameters. By setting the restriction parameters of the point cloud, the embodiment of the present application makes the encoding process of the attribute information of the points to be encoded controllable, thereby improving the encoding performance of the point cloud attribute information.

[0130] See also Figure 4 , Figure 4 Another point cloud processing method provided in an embodiment of the present application can be executed by a computer device, which can be Figure 2 The encoding device 201 in the point cloud processing system 20 shown. Figure 4 As shown, the point cloud processing method may include the following steps S401-S405:

[0131] S401, setting restriction parameters of the point cloud.

[0132] The restriction parameters include one or both of a buffer restriction parameter (coeffLengthControl) and a delay restriction parameter (maxNumofCoeff). The restriction parameters are used to determine the encoding restriction point number threshold.

[0133] In one embodiment, when the restriction parameter includes only one parameter, the unit of the restriction parameter includes any of the following: in points, in groups. Limiting in points means limiting the number of points to be encoded when encoding is performed in points; limiting in groups means limiting the number of groups to be encoded when encoding is performed in groups. When the restriction parameter is a cache restriction parameter, the encoding restriction point threshold is the cache restriction point threshold; the unit of the cache restriction parameter may be points or groups. Similarly, when the restriction parameter is a delay restriction parameter, the encoding restriction point threshold is the delay restriction point threshold; the unit of the delay restriction parameter may be points or groups. Specifically:

[0134] If the restriction parameter is in points and the value of the restriction parameter is X1, then the coding restriction point threshold is the actual total number of intermediate coefficients of X1 points to be encoded; for example, assuming that the restriction parameter is a cache restriction parameter, in points, and the value of the cache restriction parameter is 1024, then the coding restriction point threshold is the actual total number of intermediate coefficients of 1024 points to be encoded; for another example, assuming that the restriction parameter is a delay restriction parameter, in points, and the value of the delay restriction parameter is 2048, then the coding restriction point threshold is the actual total number of intermediate coefficients of 2048 points to be encoded.

[0135] If the cache limit parameter is in units of groups and the value of the limit parameter is X2, then the encoding limit point threshold is the actual total number of intermediate coefficients of the points to be encoded in the X2 groups of point cloud groups; for example, assuming that the limit parameter is a cache limit parameter, in units of groups, and the value of the cache limit parameter is 64, then the encoding limit point threshold is the actual total number of intermediate coefficients of the points to be encoded in the 64 groups of point cloud groups; assuming that the limit parameter is a delay limit parameter, in units of groups, and the value of the delay limit parameter is 128, then the encoding limit point threshold is the actual total number of intermediate coefficients of the points to be encoded in the 128 groups of point cloud groups.

[0136] If the restriction parameter is in units of groups, and the value of the restriction parameter is X3, and the number of points in each point cloud group is limited to X4, then the encoding restriction point threshold is the total number of intermediate coefficients of the points to be encoded in the X3 point cloud groups, and the total number does not exceed the value of X3*X4. For example, assuming that the restriction parameter is a cache restriction parameter, in units of groups, and the value of the cache restriction parameter is 64, and the number of points in each point cloud group is limited to 4, then the encoding restriction point threshold is the actual total number of intermediate coefficients of the points to be encoded in the 64 point cloud groups, and the total number does not exceed 64*4=256; assuming that the restriction parameter is a delay restriction parameter, in units of groups, and the value of the delay restriction parameter is 128, and the number of points in each point cloud group is limited to 4, then the encoding restriction point threshold is the actual total number of intermediate coefficients of the points to be encoded in the 128 point cloud groups, and the total number does not exceed 128*4=512.

[0137] In another embodiment, when the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: in units of points, in units of groups, and in units of multiples; wherein, restriction parameter 1 in units of multiples means that: when restriction parameter 2 limits the number of points to be encoded in units of points, restriction parameter 1 limits the number of points to be encoded in units of points, and restriction parameter 1 limits the number of points to be encoded by a multiple of the number of points to be encoded by restriction parameter 2; specifically, if the cache restriction parameter is in units of multiples, And the value of the cache limit parameter is Y1, then the cache limit point threshold is Y1 times the delay limit point threshold; if the delay limit parameter is in units of multiples and the value of the delay limit parameter is Y2, then the delay limit point threshold is Y2 times the cache limit point threshold; for example, assuming that the delay limit parameter (limit parameter 1) is in units of multiples and the value is 2; when the cache limit parameter (limit parameter 2) is in units of points and the value of the cache limit parameter is 1024, the delay limit parameter is in units of points, and the value of the delay limit parameter is 1024*2=2048.

[0138] When restriction parameter 2 limits the number of packets to be encoded in units of the number of groups, restriction parameter 1 limits the number of packets to be encoded in units of the number of groups, and restriction parameter 1 limits the number of packets to be encoded by a multiple of the number of packets to be encoded limited by restriction parameter 2; specifically, if the cache restriction parameter is in units of a multiple relationship and the value of the cache restriction parameter is Y1, the cache restriction point threshold is Y1 times the delay restriction point threshold; if the delay restriction parameter is in units of a multiple relationship and the value of the delay restriction parameter is Y2, the delay restriction point threshold is Y2 times the cache restriction point threshold; for example, assuming that the delay restriction parameter (restriction parameter 1) is in units of a multiple and the value is 2; when the cache restriction parameter (restriction parameter 2) is in units of the number of groups and the value of the cache restriction parameter is 64, the delay restriction parameter is in units of the number of groups and the value of the delay restriction parameter is 64*2=128.

[0139] As can be seen from the above, the combination methods include: both the restriction parameter 1 and the restriction parameter 2 are in units of points, or in units of groups; the restriction parameter 1 is in units of points, the restriction parameter 2 is in units of groups, or the restriction parameter 2 is in units of multiples; the restriction parameter 1 is in units of groups, the restriction parameter 2 is in units of points, or the restriction parameter 2 is in units of multiples. It can be understood that the restriction parameter 1 can be any one of the cache restriction parameter and the delay restriction parameter, and the restriction parameter 2 is the other of the cache restriction parameter and the delay restriction parameter.

[0140] S402: Obtain attribute information of the points to be encoded in the point cloud.

[0141] The specific implementation of step S402 can be found in Figure 3 The implementation of step S302 will not be repeated here.

[0142] When the restriction parameter is a cache restriction parameter, continue to execute step S403; when the restriction parameter is a delay restriction parameter, continue to execute step S404; when the restriction parameter includes a cache restriction parameter and a delay restriction parameter, continue to execute step S405.

[0143] S403: Encode the attribute information of the coding point, and control the cache logic in the encoding process according to the cache limit parameter.

[0144] In one embodiment, the cache limit parameter is used to limit the number of points to be encoded that are allowed to be written into the cache, and each point to be encoded may include one or more intermediate coefficients. The computer device determines a cache limit point number threshold according to the cache limit parameter, and whenever the number of points to be encoded written into the cache reaches the cache limit point number threshold, the computer device reads the intermediate coefficients of the points to be encoded from the cache for encoding.

[0145] In one embodiment, the encoding method is an encoding method based on the number of groups, that is, a group in the point cloud is used as a basic encoding unit for encoding. The computer device counts the total number of points to be encoded contained in each point cloud group in the cache. When the number of groups stored in the cache is M, and the total number of points to be encoded contained in the M point cloud groups reaches the cache limit point threshold, the computer device reads the intermediate coefficients of the points to be encoded contained in the M point cloud groups from the cache for encoding. Further, the computer device clears the cache and continues to write the intermediate coefficients of the points to be encoded into the cache.

[0146] In another embodiment, the encoding method is an encoding method based on the number of groups, that is, a group in the point cloud is encoded as the basic encoding unit. The computer device counts the number of point cloud groups M contained in the cache, and calculates the total number of points to be encoded contained in the cache based on M and the point limit threshold of the point cloud group; for example, assuming that M=64, the point limit threshold is 8, then the total number of points to be encoded contained in the cache=64*8=512. If the total number of points to be encoded contained in the cache reaches the cache limit point threshold, the computer device controls the reading of the intermediate coefficients of the points to be encoded contained in the M groups of point cloud groups from the cache for encoding. Further, the computer device clears the cache and continues to write the intermediate coefficients of the points to be encoded into the cache.

[0147] In another embodiment, the encoding method is an encoding method based on the number of groups, that is, a group in the point cloud (a group may include at least one point in the point cloud) is encoded as the basic encoding unit. The computer device determines the group number threshold of the point cloud groups allowed to be written in the cache according to the cache limit parameter; whenever the number of point cloud groups written in the cache reaches the group number threshold, the computer device reads the intermediate coefficients of the points to be encoded contained in the point cloud groups written in the cache from the cache for encoding. For example, if the value of the cache limit parameter is 64, the computer device determines that the number of point cloud groups allowed to be written in the cache is 64 according to the cache limit parameter, and whenever the number of point cloud groups written in the cache is 64, the computer device reads the intermediate coefficients of the points to be encoded contained in these 64 groups of point cloud groups from the cache for encoding.

[0148] Furthermore, the computer device can arrange the point cloud groups written in the cache according to a preset arrangement, and encapsulate the preset arrangement into the coded bitstream formed by the encoding; a specific implementation method for the computer device to encapsulate the intermediate coefficient arrangement of the point cloud groups in the cache into the coded bitstream formed by the encoding is: each arrangement corresponds to an index value, and the computer device writes the index value corresponding to the intermediate coefficient arrangement of the point cloud groups in the cache into the coded bitstream. The computer device can also arrange the point cloud groups written in the cache according to a preset arrangement, and send the preset arrangement directly to the decoding device; the computer device can also arrange the point cloud groups written in the cache according to a default arrangement. It can be understood that when arranging the point cloud groups written in the cache according to the default arrangement, the computer device does not need to inform the decoding device of the arrangement. Among them, the preset arrangement method includes any one of the following: a fixed arrangement method and a dynamic arrangement method; the fixed arrangement method refers to the fact that the arrangement method of the intermediate coefficients of the point cloud grouping cached each time (such as according to similarity, according to the number of points to be encoded contained in the group, etc.) is the same, and the dynamic arrangement method refers to the fact that there is a change in the arrangement method of the intermediate coefficients of the cached point cloud grouping; for example, the arrangement method of the intermediate coefficients of the point cloud grouping cached for the first time is the same as the arrangement method of the intermediate coefficients of the point cloud grouping cached for the second time, and the arrangement method of the intermediate coefficients of the point cloud grouping cached for the third time is different.

[0149] In another embodiment, the encoding includes run encoding, and the computer device determines the cache limit point threshold of the points to be encoded that are allowed to be written in the cache according to the cache limit parameter, and the cache limit point threshold is used to limit the number of points to be encoded that are allowed to be written in the cache during the encoding process; the determination method of the cache limit point threshold can refer to the determination method of the encoding limit point threshold in step S401, which will not be repeated here. In the run encoding process, the computer device sets the run length for counting continuous specific symbols. At the beginning, the run length is set to the initial value. In the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache, if the intermediate coefficients with a value of a specific symbol are read, the count value of the run length increases by one step; if the number of points to be encoded read is equal to the cache limit point threshold, the current run length is encoded, or the cache limit point threshold and the intermediate coefficients of the points to be encoded with a value of a specific symbol are encoded, and the run length is set to the initial value for recounting. If the intermediate coefficients of the points to be encoded with a value of a non-specific symbol are read, the current run length and the intermediate coefficients of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0150] For example, assume that the cache limit parameter is in points and the value is 1024 (i.e., the cache limit point number is 1024). If the computer device uses run length coding to encode the residual of the attribute information of the point to be coded, the computer device counts the number of points that are consecutive specific symbols (such as 0) in the residual of the attribute information of the point to be coded, recorded as run-length. If the residual of the attribute information of the current point to be coded is a specific symbol, the run-length value increases by 1. Whenever the number of points of the point to be coded in the cache reaches 1024, the computer device encodes the value of run-length and the residual of the attribute information of the current coded point, sets the run-length to the initial value (such as 0), and re-counts the run-length. If the residual of the attribute information of the current point to be coded is a non-specific symbol (such as non-0), the computer device encodes the current run-length value and the residual of the attribute information of the current point to be coded, sets the run-length to the initial value (such as 0), and re-counts the run-length.

[0151] S404: Encode the attribute information of the point to be encoded, and control the delay logic in the encoding process according to the delay limit parameter.

[0152] The specific encoding method used by the computer device to encode the attribute information of the code point is an encoding method that requires statistics on the preceding sequence; for example, the specific encoding method used by the computer device to encode the attribute information of the code point may be run length encoding. The following is an example of the encoding method being run length encoding:

[0153] In one embodiment, the computer device determines the delay limit point threshold according to the delay limit parameter, and the delay limit point threshold is used to limit the number of points to be encoded that are allowed to be delayed during the encoding process; the method for determining the delay limit point threshold can refer to the method for determining the encoding limit point threshold in step S401, which will not be repeated here. In the process of run encoding, a run length for counting continuous specific symbols is set, and the run length is set to an initial value (such as 0). In the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache, if the computer device reads the intermediate coefficients of the specific symbol, the count value of the run length increases by one step, and the computer device obtains the count value of the run length. If the count value of the run length reaches the delay limit point threshold, the computer device performs parameter encoding processing and resets the run length to the initial value to re-count. Among them, the computer device performs parameter encoding processing, which means: the computer device encodes the current run length; or the computer device encodes the restriction identifier, and the restriction identifier is used to set the run length to a preset value (such as the preset value can be the delay limit point threshold); or the computer device encodes the delay limit point threshold and the intermediate coefficients of the points to be encoded whose values ​​are specific symbols. If the computer device reads an intermediate coefficient of a point to be encoded whose value is a non-specific symbol, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to an initial value for recounting.

[0154] For example, assume that the current run length is 63, the specific symbol is 0, and the delay limit point threshold is 64; if the computer device reads that the value of the intermediate coefficient of the point to be encoded is 0, the current run length is 63+1=64, and the computer device encodes the current run length (64), or the computer device encodes the restriction identifier, or the computer device encodes the delay limit point threshold (64) and the intermediate coefficient of the point to be encoded whose value is the specific symbol, and resets the run length to the initial value for recounting; if the computer device reads that the value of the intermediate coefficient of the point to be encoded is 1 (non-0), the computer device encodes the current run length (63) and the intermediate coefficient of the point to be encoded whose value is 1, and resets the run length to the initial value for recounting.

[0155] In another embodiment, the computer device determines the delay limit point threshold according to the delay limit parameter, and the delay limit point threshold is used to limit the number of points to be encoded that are allowed to be delayed during the encoding process. The method for determining the delay limit point threshold can refer to the method for determining the encoding limit point threshold in step S401, which will not be repeated here. In the process of run encoding, a run length for counting continuous specific symbols is set, and the run length is set to an initial value (such as 0). In the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache, if the computer device reads the intermediate coefficients of the specific symbol, the count value of the run length increases by one step; if the number of points to be encoded reaches the delay limit point threshold, the computer device encodes the current run length, or the computer device encodes the delay limit point threshold and the intermediate coefficients of the points to be encoded with the specific symbol, and resets the run length to the initial value for recounting. If the computer device reads the intermediate coefficients of the points to be encoded with the non-specific symbol, the computer device encodes the current run length and the intermediate coefficients of the non-specific symbol, and resets the run length to the initial value for recounting.

[0156] For example, assuming that the number of intermediate coefficients currently delayed is 58, the specific symbol is 0, and the threshold value of the number of points to be encoded that are allowed to be delayed is 64; if the computer device reads that the value of the intermediate coefficient of the point to be encoded is 0, the current run length is 58+1=59, and the computer device continues to read the next intermediate coefficient; if the computer device reads that the value of the intermediate coefficient of the point to be encoded is 1 (non-0), the computer device encodes the current run length (59) and the intermediate coefficient of the point to be encoded with a value of 1, and resets the run length to the initial value to recount.

[0157] It can be understood that in the above embodiment, the run length in the run encoding is less than or equal to the delay limit point threshold; based on this, the computer device can also set the number of bits of the run length according to the delay limit point threshold.

[0158] For example, assuming that the delay limit parameter is in points and the value is 1024 (i.e., the delay limit point number is 1024), if the computer device uses run length coding to encode the residual of the attribute information of the point to be coded, the computer device counts the number of points that are consecutive specific symbols (such as 0) in the residual of the attribute information of the point to be coded, recorded as run-length, if the residual of the attribute information of the current point to be coded is a specific symbol, the run-length value increases by 1, and whenever the run-length value reaches 1024, the computer device encodes the value of run-length and the residual of the attribute information of the current coded point, sets the run-length to the initial value (such as 0), and re-counts the run-length. If the residual of the attribute information of the current point to be coded is a non-specific symbol (such as non-0), the computer device encodes the current run-length value and the residual of the attribute information of the current point to be coded, sets the run-length to the initial value (such as 0), and re-counts the run-length.

[0159] S405: Encode the attribute information of the coding point, and control the coding logic in the coding process according to the buffer limit parameter and the delay limit parameter.

[0160] In one embodiment, the cache limit parameter and the delay limit parameter are independent of each other. Taking the intermediate coefficients of the attribute information of the point to be encoded as the transformation coefficients as an example, the computer device first controls the cache logic in the encoding process according to the cache limit parameters; the specific implementation method can refer to the implementation method in step S403, which will not be repeated here. If encoding is performed in groups, after caching the intermediate coefficients of the point cloud grouping, the computer device can arrange the intermediate coefficients of the point cloud grouping, and can write the arrangement method into the encoded bit stream; or, directly specify the arrangement method to the decoding device. The computer device clears the cache and repeats the above steps until the intermediate coefficients of all point cloud groups are encoded. During the encoding process, the computer device determines the run length during run encoding according to the delay limit parameter, and controls the encoding logic in the encoding process based on the run length; the specific implementation method can refer to the implementation method in step S404, which will not be repeated here.

[0161] For example, assuming that the cache limit parameter and the delay limit parameter are both set in units of points, and the cache limit parameter is 1024, the delay limit parameter is 2048, and encoding is performed in units of groups; the computer device counts the number of points to be encoded contained in each point cloud group to be encoded in the cache, and determines the number of groups M based on the cache limit parameter and the number of points to be encoded contained in each point cloud group to be encoded in the cache; wherein M is a positive integer, the total number of points to be encoded contained in the M groups of point cloud groups to be encoded is less than or equal to 1024, and the total number of points to be encoded contained in the M+1 groups of point cloud groups to be encoded is greater than 1024. The computer device arranges the transform coefficients in the M groups of point cloud groups to be encoded one by one (such as a fixed arrangement, a random arrangement, or a specified arrangement). After obtaining the transform coefficients in the M groups of point cloud groups to be encoded, the computer device clears the current cache and restarts caching the transform coefficients of the groups to be encoded until the transform coefficients in all point cloud groups are encoded. In the specific implementation process, the transform coefficient of each point cloud group includes a first transform coefficient DC and at least one second transform coefficient {AC}. The arrangement of the transform coefficients is limited by the computer device (encoding device). For example, the transform coefficients within the cache limit parameter points of an even number of times, or the number of groups of point cloud groups in the cache is an even number, are arranged as [DC1,...,DCM,{AC1},...,{ACM}]; the transform coefficients within the cache limit parameter points of an odd number of times, or the number of groups of point cloud groups in the cache is an even number, are arranged as [{AC1},...,{ACM},DC1,...,DCM]. During the encoding process, the computer device can use run-length encoding to encode the residual of the attribute information of the point to be encoded, and count the number of points that are consecutive specific symbols in the residual of the attribute information of the point to be encoded as the run length run-length. If the residual symbol of the attribute information of the current point to be encoded is a specific value (such as 0), the run-length value increases by 1. Whenever the run-length reaches the delay limit point number indicated by the delay limit parameter, the value of the run-length and the residual of the attribute information of the current point to be encoded are encoded, and the run-length is set to 0 and the counting starts again. If the residual symbol of the attribute information of the current point to be encoded is a non-specific value (such as non-0), the computer device first encodes the run-length value, then encodes the residual of the attribute information of the current point to be encoded, sets the run-length to 0, and restarts counting.

[0162] In another embodiment, the cache limit parameter and the delay limit parameter are related to each other; that is, the cache limit parameter is determined based on the delay limit parameter, or the delay limit parameter is determined based on the cache limit parameter. Taking the intermediate coefficient of the attribute information of the point to be encoded as the transformation coefficient as an example, after the computer device determines the cache limit parameter and the delay limit parameter based on the correlation, it controls the cache logic in the encoding process according to the cache limit parameter; the specific implementation method can refer to the implementation method in step S403, which will not be repeated here. If encoding is performed in groups, after caching the intermediate coefficients of the point cloud grouping, the computer device can arrange the intermediate coefficients of the point cloud grouping, and can write the arrangement method into the encoding bit stream; or, directly specify the arrangement method to the decoding device. The computer device clears the cache and repeats the above steps until the intermediate coefficients of all point cloud groups are encoded. During the encoding process, the computer device determines the run length during run encoding according to the delay limit parameter, and controls the encoding logic in the encoding process based on the run length; the specific implementation method can refer to the implementation method in step S404, which will not be repeated here.

[0163] For example, let the delay limit parameter be in multiples and its value be 2; let the cache limit parameter be in groups and its value be 64, then the cache limit points determined based on the cache limit parameter are the total number of points to be encoded in the 64 point cloud groups, and the delay limit points determined based on the delay limit parameter are 64*2=the total number of points to be encoded in 128 point cloud groups. For another example, let the cache limit parameter be in groups and its value be 64, and the second transform coefficients of the first point cloud group in the 64 point cloud groups are k, then the delay limit parameter is in points and its value is 64+k, to ensure real-time decoding of the first point cloud group. If encoding is performed in groups; the computer device counts the number of points to be encoded contained in the 64 point cloud groups, and arranges the transform coefficients in the 64 point cloud groups to be encoded one by one (such as a fixed arrangement, a random arrangement, or a specified arrangement). After obtaining the transform coefficients in 64 groups of point cloud groups to be encoded, the computer device clears the current cache and restarts caching the transform coefficients of the groups to be encoded until the transform coefficients in all point cloud groups are encoded. In the specific implementation process, the transform coefficients of each point cloud group include a first transform coefficient DC and at least one second transform coefficient {AC}. The arrangement of the transform coefficients is limited by the computer device (encoding device). For example, for the transform coefficients in the cache limit parameter points of even times, or the number of groups of point cloud groups in the cache is an even number, they are arranged as [DC1,...,DCM,{AC1},...,{ACM}]; for the transform coefficients in the cache limit parameter points of odd times, or the number of groups of point cloud groups in the cache is an even number, they are arranged as [{AC1},...,{ACM},DC1,...,DCM]. In the encoding process, the computer device can use run-length encoding to encode the residual of the attribute information of the point to be encoded, and the number of points that are consecutively specific symbols in the residual of the attribute information of the point to be encoded is counted as the run-length. If the residual symbol of the attribute information of the current point to be encoded is a specific value (such as 0), the run-length value is increased by 1. Whenever the run-length reaches the delay threshold indicated by the delay limit parameter, the run-length value and the residual of the attribute information of the current point to be encoded are encoded, and the run-length is set to 0, and the counting starts again. If the residual symbol of the attribute information of the current point to be encoded is a non-specific value (such as non-0), the computer device first encodes the run-length value, then encodes the residual of the attribute information of the current point to be encoded, and sets the run-length to 0, and restarts the counting.

[0164] The embodiments of this application are Figure 3On the basis of the implementation example, on the one hand, the encoding process of point cloud attribute information is controlled by one or more of the cache limit parameters and the delay limit parameters; on the other hand, the attribute information of multiple points to be encoded or multiple point cloud groups to be encoded are arranged in a preset arrangement, thereby improving the encoding performance of the point cloud attribute information.

[0165] See also Figure 5 , Figure 5 Another point cloud processing method provided in an embodiment of the present application can be executed by a computer device, which can be specifically Figure 2 The decoding device 202 in the point cloud processing system 20 is shown. Figure 5 As shown, the point cloud processing method may include the following steps S501-S503:

[0166] S501, obtaining the encoded code stream of the point to be decoded in the point cloud.

[0167] The coded bitstream of the point to be decoded is obtained by encoding the point to be encoded by the encoding device. For details, see Figure 3 and Figure 4 The encoding code stream of the point to be decoded in the point cloud can be obtained by the computer device from the encoding device in real time, or it can be obtained by the computer device from the downloaded encoding code stream.

[0168] S502: Obtain restriction parameters of the point cloud.

[0169] The computer device obtains the restriction parameters of the point cloud including any one or more of the following: parsing the restriction parameters set as fixed values ​​from the coded code stream, parsing the restriction parameters set as variable values ​​from the coded code stream, and parsing the restriction parameters set after dynamic adjustment from the coded code stream. It is understandable that if the restriction parameters of the point cloud are not parsed from the coded code stream, the computer device determines that the restriction parameters of the point cloud are default values.

[0170] In one embodiment, when the restriction parameter includes only one parameter, the unit of the restriction parameter includes any one of the following: in points, in groups. Limiting in points means limiting the number of points to be decoded when decoding is performed in points; limiting in groups means limiting the number of groups to be decoded when decoding is performed in groups. When the restriction parameter is a cache restriction parameter, the decoding restriction point threshold is the cache restriction point threshold; the unit of the cache restriction parameter may be points or groups. Similarly, when the restriction parameter is a delay restriction parameter, the decoding restriction point threshold is the delay restriction point threshold; the unit of the delay restriction parameter may be points or groups.

[0171] In another embodiment, when the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: in units of points, in units of groups, and in units of multiples; wherein, restriction parameter 1 in units of multiples means that: when restriction parameter 2 limits the number of points to be decoded in units of points, restriction parameter 1 limits the number of points to be decoded in units of points, and restriction parameter 1 limits the number of points to be decoded to be a multiple of the number of points to be decoded limited by restriction parameter 2. When restriction parameter 2 limits the number of packets to be decoded in units of groups, restriction parameter 1 limits the number of packets to be decoded in units of groups, and restriction parameter 1 limits the number of packets to be decoded to be a multiple of the number of packets to be decoded limited by restriction parameter 2.

[0172] As can be seen from the above, the combination methods include: both the restriction parameter 1 and the restriction parameter 2 are in units of points, or in units of groups; the restriction parameter 1 is in units of points, the restriction parameter 2 is in units of groups, or the restriction parameter 2 is in units of multiples; the restriction parameter 1 is in units of groups, the restriction parameter 2 is in units of points, or the restriction parameter 2 is in units of multiples. It can be understood that the restriction parameter 1 can be any one of the cache restriction parameter and the delay restriction parameter, and the restriction parameter 2 is the other of the cache restriction parameter and the delay restriction parameter.

[0173] S503: Decode the coded bitstream of the point to be decoded, and control the decoding process according to the restriction parameters.

[0174] The decoding method includes run decoding. If the encoding device encodes in units of points, the computer device decodes in units of points; if the encoding device encodes in units of groups, the computer device decodes in units of groups. The restriction parameter may include at least one of a cache restriction parameter and a delay restriction parameter.

[0175] In the first implementation, the computer device controls the decoding process according to the restriction parameter in the following manner: determining the restriction point threshold according to the restriction parameter, the restriction point threshold is used to limit the number threshold of the points to be decoded; for example, if the restriction parameter is a cache restriction parameter, the computer device determines the number threshold of the points to be decoded that are allowed to be cached in the cache according to the cache restriction parameter; if the restriction parameter is a delay restriction parameter, the computer device determines the number threshold of the points to be decoded that are allowed to be delayed in the decoding process according to the delay restriction parameter. In one implementation, during the process of run decoding, if the computer device reads the restriction flag, the count value of the first run length can be set to a preset value (such as the restriction point threshold); if the count value of the first run length is greater than zero and is not equal to the restriction point threshold, the intermediate coefficient of the current point to be decoded is set to a specific value (such as 0), and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero; the computer device obtains the intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero and performs decoding. In another implementation, during the process of run decoding, if the computer device reads the restriction flag, the first run length can be parsed to obtain the count value of the first run length. If the count value of the first run length is greater than zero and is not equal to the limit point threshold, the intermediate coefficient of the current point to be decoded is set to a specific value (such as 0), and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero; the computer device obtains the intermediate coefficient of the point to be decoded when the count value of the first run length is equal to zero and decodes it. The restriction flag is used to indicate whether the limit point threshold is used. If the restriction flag is read, it means that the limit point threshold is used; if the restriction flag is not read, it means that the limit point threshold is not used. The computer device may read the restriction flag in any of the following ways: reading from the encoded code stream, or setting by default, or obtaining by conditional determination, the condition is, for example, that the first run length is equal to a preset value, that is, when the first run length is equal to the preset value, the restriction flag is set.

[0176] In the second implementation mode, the computer device controls the decoding process according to the restriction parameter in the following manner: determine the restriction point number threshold according to the restriction parameter, and the restriction point number threshold is used to limit the number threshold of the points to be decoded. In the process of run decoding, read the count value of the first run length from the coded code stream; if the count value of the first run length is greater than zero and equal to the restriction point number threshold, the computer device sets the intermediate coefficient of the current point to be decoded to a specific value, and reduces the count value of the first run length by one step until the count value of the first run length is equal to zero; read the count value of the second run length from the coded code stream, if the count value of the second run length is greater than zero and not equal to the restriction point number threshold, set the intermediate coefficient of the current point to be decoded to a preset value, and reduce the count value of the second run length by one step until the count value of the second run length is equal to zero; obtain the intermediate coefficient of the point to be decoded read when the count value of the second run length is equal to zero and perform decoding. It should be noted that the second run length is the next run length of the first run length. If the count value of the second run length is greater than zero and equal to the limit point threshold, the computer device sets the intermediate coefficient of the current point to be decoded to a specific value, and reduces the count value of the second run length by one step until the count value of the second run length is equal to zero; read the count value of the third run length from the encoded code stream, and continue to judge. In other words, if the count value of the read run length is greater than zero and equal to the limit point threshold, then when the count value of the run length is 0, the computer device will continue to read the count value of the next run length; if the count value of the read run length is greater than zero and not equal to the limit point threshold, then when the count value of the run length is 0, the computer device will obtain the intermediate coefficient of the point to be decoded read when the count value of the run length is equal to zero and perform decoding.

[0177] In implementation method three, during the process of run decoding, the computer device reads the count value of the first run length from the encoded code stream; if the count value of the first run length is greater than zero, the computer device sets the intermediate coefficient of the current point to be decoded to a specific value, and reduces the count value of the first run length by one step until the count value of the first run length is equal to zero; the intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded. It should be noted that in this implementation method, the intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero may be a specific value, and in a possible implementation method, the intermediate coefficient with a specific value can be accumulated for multiple run lengths.

[0178] It is understandable that when encoding is performed in groups, the decoding process is similar to the above-mentioned implementation methods 1 to 3, and will not be repeated here. The difference is that when the restriction parameter is a cache restriction parameter, after the computer device caches the intermediate parameters of the read point cloud grouping, it can also arrange the intermediate coefficients of each point cloud grouping in the cache according to the obtained arrangement method, and then decode according to the above-mentioned implementation methods 1 to 3.

[0179] In one embodiment, the decoding method is a decoding method based on the number of points, that is, a point in the point cloud is decoded as the basic decoding unit. The specific implementation method of the computer device decoding the coded code stream of the point to be decoded is: the coded code stream of the point to be decoded is parsed to obtain the intermediate coefficients of the point to be decoded, the intermediate coefficients include residuals or transform coefficients, the computer device writes the intermediate coefficients of the point to be decoded into the cache, and reads the intermediate coefficients of the point to be decoded from the cache for decoding. During the decoding process, the computer device can control the decoding process based on restriction parameters; for example, the computer device can control the decoding process based on delay restriction parameters; for another example, the computer device can control the decoding process based on cache restriction parameters; for another example, the computer device can control the decoding process based on cache restriction parameters and delay restriction parameters. The specific implementation method of the computer device decoding the coded code stream can refer to the above-mentioned implementation method of point cloud decoding, which will not be repeated here.

[0180] In another embodiment, the decoding method is a decoding method based on the number of groups, that is, a group in the point cloud is decoded as the basic decoding unit. The specific implementation method of the computer device decoding the coded code stream of the point to be decoded is: the coded code stream of the point to be decoded is parsed to obtain multiple groups of point cloud groups, each group of point cloud groups contains the intermediate coefficients of the point to be decoded, and the intermediate coefficients include residuals or transformation coefficients. The computer device writes the point cloud group into the cache, and reads the intermediate coefficients of the point to be decoded in the point cloud group from the cache in sequence for decoding. During the decoding process, the computer device can control the decoding process based on the restriction parameters; for example, the computer device can control the decoding process based on the delay restriction parameters; for another example, the computer device can control the decoding process based on the cache restriction parameters; for another example, the computer device can control the decoding process based on the cache restriction parameters and the delay restriction parameters. The specific implementation method of the computer device decoding the coded code stream can refer to the above-mentioned implementation method of point cloud decoding, which will not be repeated here.

[0181] In the embodiment of the present application, when it is necessary to decode the points to be decoded in the point cloud, the encoding stream of the points to be decoded in the point cloud and the restriction parameters of the point cloud can be obtained, the encoding stream of the points to be decoded can be decoded, and the decoding process can be controlled according to the restriction parameters. By obtaining the restriction parameters of the point cloud, the embodiment of the present application makes the attribute information decoding process of the points to be decoded controllable, thereby improving the decoding performance of the point cloud attribute information.

[0182] See also Figure 6 , Figure 6 Another point cloud processing method provided in the embodiment of the present application can be executed by a computer device, which can be Figure 2 The decoding device 202 in the point cloud processing system 20 is shown. Figure 5 As shown, the point cloud processing method may include the following steps S601-S605:

[0183] S601, obtaining the encoded code stream of the point to be decoded in the point cloud.

[0184] S602: Obtain restriction parameters of the point cloud.

[0185] The specific implementation of step S601 and step S602 can be referred to Figure 5 The implementation of step S501 and step S502 will not be repeated here.

[0186] When the restriction parameter is a cache restriction parameter, continue to execute step S603; when the restriction parameter is a delay restriction parameter, continue to execute step S604; when the restriction parameter includes a cache restriction parameter and a delay restriction parameter, continue to execute step S605.

[0187] S603: Decode the attribute information of the point to be decoded, and control the cache logic in the decoding process according to the cache limit parameter.

[0188] In one embodiment, the cache limit parameter is used to limit the number of points to be decoded that are allowed to be written into the cache, and each point to be decoded may include one or more intermediate coefficients. The computer device determines a cache limit point number threshold according to the cache limit parameter, and whenever the number of points to be decoded written into the cache reaches the cache limit point number threshold, the computer device reads the intermediate coefficients of the points to be decoded from the cache for decoding.

[0189] In one embodiment, the decoding method is a decoding method based on the number of groups, that is, a group in the point cloud is used as the basic decoding unit for decoding. The computer device counts the total number of points to be decoded contained in each point cloud group in the cache. When the number of groups stored in the cache is M, and the total number of points to be decoded contained in the M point cloud groups reaches the cache limit point threshold, the computer device reads the intermediate coefficients of the points to be decoded contained in the M point cloud groups from the cache for decoding. Furthermore, the computer device clears the cache and continues to write the intermediate coefficients of the points to be decoded into the cache.

[0190] In another embodiment, the decoding method is a decoding method based on the number of groups, that is, a group in the point cloud is used as the basic decoding unit for decoding. The computer device counts the total number of target points to be decoded contained in each point cloud group in the cache, and the number of target points to be decoded in each point cloud group is less than or equal to the point limit threshold of each point cloud group. If the cache includes M groups of point cloud groups, and the total number of target points to be decoded contained in the M groups of point cloud groups reaches the cache limit point threshold, then the intermediate coefficients of the points to be decoded contained in the M groups of point cloud groups are read from the cache for decoding. Further, the computer device clears the cache and continues to write the intermediate coefficients of the points to be decoded into the cache.

[0191] In another embodiment, the decoding method is a decoding method based on the number of groups, that is, a group in the point cloud (a group may include at least one point in the point cloud) is used as the basic decoding unit for decoding. The computer device counts the number of point cloud groups M contained in the cache, and calculates the total number of points to be decoded contained in the cache based on M and the point limit threshold of the point cloud group; for example, assuming that M=64, the point limit threshold is 8, then the total number of points to be decoded contained in the cache=64*8=512. If the total number of points to be decoded contained in the cache reaches the cache limit point threshold, the computer device controls the reading of the intermediate coefficients of the points to be decoded contained in the M groups of point cloud groups from the cache for decoding. Further, the computer device clears the cache and continues to write the intermediate coefficients of the points to be decoded into the cache.

[0192] In one embodiment, during the decoding process, the computer device can also obtain the preset arrangement of the point cloud groups in the cache. The preset arrangement includes any one of the following: a fixed arrangement and a dynamic arrangement; the fixed arrangement means that the arrangement of the intermediate coefficients of the point cloud groups cached each time (such as according to the similarity, according to the number of points to be encoded contained in the group, etc.) is the same, and the dynamic arrangement means that the arrangement of the intermediate coefficients of the cached point cloud groups changes; for example, the arrangement of the intermediate coefficients of the point cloud groups cached for the first time is the same as the arrangement of the intermediate coefficients of the point cloud groups cached for the second time, and the arrangement of the intermediate coefficients of the point cloud groups cached for the third time is different. The specific implementation method of the computer device obtaining the preset arrangement of the point cloud groups in the cache is: each arrangement corresponds to an index value, the computer device can parse the index value from the encoded code stream, and determine the arrangement of the point cloud groups according to the index value.

[0193] S604: Decode the attribute information of the point to be decoded, and control the delay logic in the decoding process according to the delay limit parameter.

[0194] The specific implementation of step S604 can be found in Figure 5 The implementation of step S503 will not be repeated here.

[0195] S605: Decode the attribute information of the point to be decoded, and control the decoding logic in the decoding process according to the buffer limit parameter and the delay limit parameter.

[0196] In one embodiment, the cache limit parameter and the delay limit parameter are independent of each other. The computer device first reads the intermediate parameters of the point to be decoded, or the intermediate parameters of the point cloud grouping, into the cache according to the delay limit parameter; the specific implementation method can refer to the implementation method in step S604, which will not be repeated here. If the encoding device encodes in groups, after caching the read intermediate parameters of the point cloud grouping, the computer device can also arrange the intermediate coefficients of each point cloud grouping in the cache according to the acquired arrangement. The computer device can then control the decoding logic in the decoding process according to the delay limit parameter; the specific implementation method can refer to the implementation method in step S503, which will not be repeated here.

[0197] In another embodiment, the cache limit parameter and the delay limit parameter are interrelated; that is, the cache limit parameter is determined based on the delay limit parameter, or the delay limit parameter is determined based on the cache limit parameter. The computer device first reads the intermediate parameters of the point to be decoded, or the intermediate parameters of the point cloud grouping, into the cache according to the delay limit parameter; the specific implementation method can refer to the implementation method in step S604, which will not be repeated here. If the encoding device encodes in groups, after caching the intermediate parameters of the read point cloud grouping, the computer device can also arrange the intermediate coefficients of each point cloud grouping in the cache according to the acquired arrangement. The computer device can then control the decoding logic in the decoding process according to the delay limit parameter; the specific implementation method can refer to the implementation method in step S503, which will not be repeated here.

[0198] The embodiments of this application are Figure 5 On the basis of the implementation example, on the one hand, the decoding process of the point cloud attribute information is controlled by one or more of the cache limit parameters and the delay limit parameters; on the other hand, the attribute information of multiple points to be decoded, or multiple point cloud groups to be decoded, are arranged according to the arrangement indicated by the encoding code stream, thereby improving the decoding performance of the point cloud attribute information.

[0199] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0200] See also Figure 7 , Figure 7 A schematic diagram of the structure of a point cloud processing device provided in an embodiment of the present application; the point cloud processing device may be a computer program (including program code) running in an encoding device, for example, the point cloud processing device may be an application software in the encoding device. Figure 7 As shown, the point cloud processing device includes a setting unit 701, an acquisition unit 702 and a processing unit 703. Figure 7 , the detailed description of each unit is as follows:

[0201] A setting unit 701, used to set restriction parameters of the point cloud, where the restriction parameters include one or both of a cache restriction parameter and a delay restriction parameter;

[0202] An acquisition unit 702 is used to acquire attribute information of a point to be encoded in a point cloud;

[0203] The processing unit 703 is used to encode the attribute information of the coding point; and control the coding process according to the restriction parameter.

[0204] In one implementation, the encoding method includes an encoding method using points as units; the processing unit 703 is used to encode the attribute information of the coded point, specifically to:

[0205] The attribute information of the to-be-coded point is converted to obtain the intermediate coefficient of the to-be-coded point; the intermediate coefficient includes a residual or a transformation coefficient;

[0206] Write the intermediate coefficients of the points to be encoded into the cache;

[0207] The intermediate coefficients of the point to be encoded are read from the buffer for encoding.

[0208] In one implementation, the encoding method includes an encoding method based on the number of groups; the processing unit 703 is used to encode the attribute information of the code point, specifically to:

[0209] The attribute information of the coded points is grouped to obtain multiple point cloud groups;

[0210] The attribute information of the points to be encoded in the point cloud group is converted to obtain a converted point cloud group, wherein the converted point cloud group contains intermediate coefficients of the points to be encoded; the intermediate coefficients include residuals or transformation coefficients;

[0211] Write the converted point cloud groups into the cache;

[0212] The intermediate coefficients of the points to be encoded in the converted point cloud group are read from the cache in sequence for encoding.

[0213] In one implementation, the restriction parameter includes a cache restriction parameter; the processing unit 703 is used to control the encoding process according to the restriction parameter, specifically to:

[0214] Control the cache logic during encoding according to the cache limit parameters.

[0215] In one implementation, the processing unit 703 is used to control the cache logic in the encoding process according to the cache limit parameter, specifically to:

[0216] Determine the cache limit point number threshold according to the cache limit parameter, and the cache limit point number threshold is used to limit the number threshold of the points to be encoded that are allowed to be written into the cache;

[0217] Whenever the number of points to be encoded written into the cache reaches the cache limit point number threshold, the control reads the intermediate coefficients of the points to be encoded from the cache for encoding.

[0218] In one embodiment, the encoding method includes an encoding method based on the number of groups; the processing unit 703 is used to control the reading of the intermediate coefficients of the to-be-encoded points from the cache for encoding whenever the number of to-be-encoded points written in the cache reaches the cache limit point number threshold, specifically for:

[0219] Count the total number of points to be encoded contained in each point cloud group in the cache. If the cache includes M point cloud groups and the total number of points to be encoded contained in the M point cloud groups reaches the cache limit point number threshold, then control the reading of the intermediate coefficients of the points to be encoded contained in the M point cloud groups from the cache for encoding; or,

[0220] The number of point cloud groups M contained in the cache is counted, and the total number of points to be encoded contained in the cache is calculated based on M and the point limit threshold of the point cloud group. If the total number of points to be encoded contained in the cache reaches the cache limit point threshold, the intermediate coefficients of the points to be encoded contained in the M groups of point cloud groups are read from the cache for encoding.

[0221] In one implementation, the encoding method includes an encoding method based on the number of groups; the processing unit 703 is used to control the cache logic in the encoding process according to the cache limit parameter, specifically for:

[0222] Determine the threshold of the number of point cloud groups allowed to be written into the cache according to the cache limit parameter;

[0223] Whenever the number of point cloud groups written in the cache reaches a group number threshold, the intermediate coefficients of the points to be encoded contained in the point cloud groups written in the cache are read from the cache for encoding.

[0224] In one implementation, the processing unit 703 is further configured to:

[0225] Arrange the point cloud groups written in the cache according to the preset arrangement;

[0226] The preset arrangement methods include any of the following: a fixed arrangement method and a dynamic arrangement method; a fixed arrangement method means that the arrangement method of the intermediate coefficients of the point cloud groups in the cache is fixed, and a dynamic arrangement method means that the arrangement method of the intermediate coefficients of the point cloud groups in the cache changes.

[0227] In one implementation, each arrangement corresponds to an index value; the processing unit 703 is further configured to:

[0228] Write the index value corresponding to the arrangement of the intermediate coefficients of the point cloud grouping in the cache into the encoded bitstream.

[0229] In one implementation, the encoding includes run length encoding; the processing unit 703 is used to control the cache logic in the encoding process according to the cache limit parameter, specifically to:

[0230] Determine the cache limit point number threshold of the to-be-encoded points allowed to be written into the cache according to the cache limit parameter, and the cache limit point number threshold is used to limit the number threshold of the to-be-encoded points allowed to be written into the cache during the encoding process;

[0231] In the run length encoding process, setting the run length used to count consecutive specific symbols;

[0232] Set the run length to the initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache:

[0233] If an intermediate coefficient with a specific symbol is read, the count value of the run length is increased by one step; if the number of points to be encoded read is equal to the cache limit point threshold, the current run length is encoded, or the cache limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is set to the initial value for recounting;

[0234] If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0235] In one implementation, the restriction parameter includes a delay restriction parameter; the processing unit 703 is used to control the encoding process according to the restriction parameter, specifically to:

[0236] The delay logic in the encoding process is controlled according to the delay limit parameters.

[0237] In one implementation, the encoding includes run length encoding; the processing unit 703 is used to control the delay logic in the encoding process according to the delay limit parameter, specifically for:

[0238] Determine a delay limit point number threshold according to the delay limit parameter, where the delay limit point number threshold is used to limit the number threshold of the to-be-encoded points that are allowed to be delayed during the encoding process;

[0239] In the process of run length encoding, a run length for counting consecutive specific symbols is set;

[0240] Set the run length to the initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache:

[0241] If the intermediate coefficient with a specific symbol is read, the count value of the run length is increased by one step; the count value of the run length is obtained, and if the count value of the run length reaches the delay limit point threshold, parameter encoding processing is performed, and the run length is reset to the initial value for re-counting;

[0242] If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting;

[0243] Among them, parameter coding processing refers to: encoding the current run length; or, encoding a restriction identifier, which is used to set the run length to a preset value; or, encoding the delay limit point threshold and the intermediate coefficients of the to-be-coded point whose value is a specific symbol.

[0244] In one implementation, the encoding includes run length encoding; the processing unit 703 is used to control the delay logic in the encoding process according to the delay limit parameter, specifically for:

[0245] Determine a delay limit point number threshold according to the delay limit parameter, where the delay limit point number threshold is used to limit the number threshold of the to-be-encoded points that are allowed to be delayed during the encoding process;

[0246] In the process of run length encoding, a run length for counting consecutive specific symbols is set;

[0247] Set the run length to the initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache:

[0248] If an intermediate coefficient with a specific symbol is read, the count value of the run length is increased by one step; if the number of points to be encoded reaches the delay limit point threshold, the current run length is encoded, or the delay limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is reset to the initial value for recounting;

[0249] If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0250] In one implementation, the delay limit parameter is used to determine the delay limit point threshold; the encoding includes run length encoding, and the run length in the run length encoding is less than or equal to the delay limit point threshold; the processing unit 703 is further used to:

[0251] Sets the run length in bits according to the delay limit point threshold.

[0252] In one implementation, the restriction parameters include a cache restriction parameter and a delay restriction parameter; the processing unit 703 is used to control the encoding process according to the restriction parameters, specifically to:

[0253] The encoding logic in the encoding process is controlled according to the cache limit parameters and the delay limit parameters.

[0254] In one embodiment, the limiting parameters are set in any one or more of the following ways:

[0255] Set limit parameters to default values;

[0256] The restriction parameters are set to fixed values, and the set restriction parameters are encapsulated into the encoded bit stream formed by encoding;

[0257] The restriction parameter is set to a variable value, and the set restriction parameter is encapsulated into the encoding bit stream formed by encoding;

[0258] The value of the restriction parameter is dynamically adjusted during the encoding process, and the dynamically adjusted restriction parameter is encapsulated into the encoding bitstream formed by the encoding.

[0259] In one embodiment, when the restriction parameter includes only one parameter, the unit of the restriction parameter includes any one of the following: in points, in groups;

[0260] When the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: point number, group number and multiple relationship.

[0261] In one embodiment, the restriction parameter is used to determine the encoding restriction point threshold; if the restriction parameter is a cache restriction parameter, the encoding restriction point threshold is the cache restriction point threshold; if the restriction parameter is a delay restriction parameter, the encoding restriction point threshold is the delay restriction point threshold;

[0262] If the restriction parameter is in points and the value of the restriction parameter is X1, the coding restriction point threshold is the actual total number of intermediate coefficients of the X1 points to be coded;

[0263] If the restriction parameter is in groups, and the value of the restriction parameter is X2, then the coding restriction point number threshold is the actual total number of intermediate coefficients of the points to be coded in the X2 group of point cloud groups;

[0264] If the restriction parameter is in groups, and the value of the restriction parameter is X3, and the point limit of each point cloud group is X4, then the encoding restriction point threshold is the total number of intermediate coefficients of the points to be encoded in the X3 point cloud groups, and the total number does not exceed the value of X3*X4.

[0265] In one embodiment, the restriction parameter includes two parameters: a cache restriction parameter and a delay restriction parameter; the cache restriction parameter is used to determine the cache restriction point threshold; the delay restriction parameter is used to determine the delay restriction point threshold;

[0266] If the cache limit parameter is in multiples, and the value of the cache limit parameter is Y1, then the cache limit point threshold is Y1 times the delay limit point threshold;

[0267] If the delay limit parameter is in multiple units, and the value of the delay limit parameter is Y2, then the delay limit point threshold is Y2 times the cache limit point threshold.

[0268] According to one embodiment of the present application, Figure 3 and Figure 4 Some of the steps involved in the point cloud processing method shown can be represented by Figure 7 The point cloud processing device shown in the figure is executed by each unit. For example, Figure 3 The step S301 shown in FIG. Figure 7 The setting unit 701 shown in FIG. 1 is executed, and step S302 can be performed by Figure 7 The acquisition unit 702 shown in FIG. 1 is executed, and step S303 can be performed by Figure 7 The processing unit 703 is shown to execute. Figure 4 The step S401 shown in FIG. 4 can be performed by Figure 7 The setting unit 701 shown in FIG. 1 is executed, and step S402 can be performed by Figure 7 The acquisition unit 702 shown in FIG. 1 is executed, and steps S403 to S405 can be performed by Figure 7 The processing unit 703 is shown to execute. Figure 7 The various units in the point cloud processing device shown can be individually or completely combined into one or several other units, or one (some) of the units can be further divided into multiple functionally smaller units, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the point cloud processing device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0269] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM) and other processing elements and storage elements. Figure 3 and Figure 4 A computer program (including program code) for each step involved in the corresponding method shown in Figure 7 The point cloud processing device shown in and the point cloud processing method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer readable recording medium, and loaded into the above-mentioned computing device through the computer readable recording medium and run therein.

[0270] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the point cloud processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the point cloud processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0271] See also Figure 8 , Figure 8 A schematic diagram of the structure of another point cloud processing device provided in an embodiment of the present application; the point cloud processing device may be a computer program (including program code) running in a decoding device, for example, the point cloud processing device may be an application software in the decoding device. Figure 8 As shown, the point cloud processing device includes an acquisition unit 801 and a processing unit 802. Figure 8 , the detailed description of each unit is as follows:

[0272] An acquisition unit 801 is used to acquire a coded bitstream of a point to be decoded in a point cloud, and to acquire a restriction parameter of the point cloud, where the restriction parameter includes one or both of a cache restriction parameter and a delay restriction parameter;

[0273] The processing unit 802 is configured to decode the coded bitstream of the point to be decoded; and control the decoding process according to the restriction parameter.

[0274] In one implementation, the acquisition unit 801 acquires the restriction parameters of the point cloud including any one or more of the following:

[0275] Set limit parameters to default values;

[0276] Parse the restriction parameters that have been set to fixed values ​​from the encoded bitstream;

[0277] Parse the restriction parameters that have been set as variable values ​​from the encoded bitstream;

[0278] Parse the set dynamically adjusted restriction parameters from the encoded bitstream.

[0279] In one implementation, the decoding method includes a decoding method based on point numbers; the processing unit 802 is used to decode the coded bit stream of the point to be decoded, specifically to:

[0280] Parse the coded bit stream of the point to be decoded to obtain the intermediate coefficient of the point to be decoded; the intermediate coefficient includes a residual or a transform coefficient;

[0281] Write the intermediate coefficients of the points to be decoded into the cache;

[0282] The intermediate coefficients of the point to be decoded are read from the cache for decoding.

[0283] In one implementation, the decoding method includes a decoding method based on the number of groups; the processing unit 802 is used to decode the coded bit stream of the decoding point, specifically to:

[0284] Parse the coded bit stream of the point to be decoded to obtain multiple groups of point cloud groups, each of which contains intermediate coefficients of the point to be decoded; the intermediate coefficients include residuals or transform coefficients;

[0285] Write point cloud groups into the cache;

[0286] The intermediate coefficients of the points to be decoded in the point cloud group are read from the cache in sequence for decoding.

[0287] In one implementation, the decoding includes run length decoding; the processing unit 802 is used to control the decoding process according to the restriction parameters, specifically to:

[0288] Determine a limit point number threshold according to the limit parameter, and the limit point number threshold is used to limit the number threshold of points to be decoded;

[0289] During the process of run decoding, if the limit flag is read, the count value of the first run length is set to a preset value;

[0290] If the count value of the first run length is greater than zero and is not equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0291] The intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded.

[0292] In one implementation, the decoding includes run length decoding; the processing unit 802 is used to control the decoding process according to the restriction parameters, specifically to:

[0293] Determine a limit point number threshold according to the limit parameter, and the limit point number threshold is used to limit the number threshold of points to be decoded;

[0294] During the run length decoding process, a count value of a first run length is read from the encoded code stream;

[0295] If the count value of the first run length is greater than zero and equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0296] Reading a count value of a second run length from the encoded bitstream, if the count value of the second run length is greater than zero and is not equal to the limit point number threshold, setting the intermediate coefficient of the current point to be decoded to a preset value, and reducing the count value of the second run length by one step until the count value of the second run length is equal to zero;

[0297] The intermediate coefficient of the point to be decoded read when the count value of the second run length is equal to zero is obtained and decoded.

[0298] In one implementation, the decoding includes run length decoding; the processing unit 802 is used to control the decoding process according to the restriction parameters, specifically to:

[0299] During the run length decoding process, a count value of a first run length is read from the encoded code stream;

[0300] If the count value of the first run length is greater than zero, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0301] The intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded.

[0302] In one implementation, the restriction parameter includes a cache restriction parameter; the processing unit 802 is used to control the decoding process according to the restriction parameter, specifically to:

[0303] The cache logic during decoding is controlled according to the cache limit parameters.

[0304] In one implementation, the processing unit 802 is configured to control the cache logic in the decoding process according to the cache limit parameter, specifically to:

[0305] Determine a cache limit point number threshold according to a cache limit parameter, where the cache limit point number threshold is used to limit the number of points to be decoded that are allowed to be written into the cache;

[0306] Whenever the number of points to be decoded written into the cache reaches the cache limit point number threshold, the control reads the intermediate coefficients of the points to be decoded from the cache for decoding.

[0307] In one embodiment, the decoding method includes a decoding method based on the number of groups; the processing unit 802 is used to control the reading of the intermediate coefficients of the points to be decoded from the cache for decoding whenever the number of points to be decoded written in the cache reaches the cache limit point number threshold, specifically for:

[0308] Count the total number of points to be decoded contained in each point cloud group in the cache. If the cache includes M point cloud groups and the total number of points to be decoded contained in the M point cloud groups reaches the cache limit point number threshold, then control the reading of the intermediate coefficients of the points to be decoded contained in the M point cloud groups from the cache for decoding; or,

[0309] The number of point cloud groups M contained in the cache is counted, and the total number of points to be encoded contained in the cache is calculated based on M and the point limit threshold of the point cloud group. If the total number of points to be encoded contained in the cache reaches the cache limit point threshold, the intermediate coefficients of the points to be decoded contained in the M groups of point cloud groups are read from the cache for decoding.

[0310] In one implementation, the decoding method includes a decoding method based on the number of groups; the processing unit 802 is used to control the cache logic in the decoding process according to the cache limit parameter, specifically used to:

[0311] Determine the threshold of the number of point cloud groups allowed to be written into the cache according to the cache limit parameter;

[0312] Whenever the number of point cloud groups written in the cache reaches a group number threshold, the control reads the intermediate coefficients of the points to be decoded contained in the point cloud groups written in the cache from the cache for decoding.

[0313] In one implementation, the processing unit 802 is further configured to:

[0314] Arrange the point cloud groups written in the cache according to the preset arrangement;

[0315] The preset arrangement methods include any of the following: a fixed arrangement method and a dynamic arrangement method; a fixed arrangement method means that the arrangement method of the intermediate coefficients of the point cloud groups in the cache is fixed, and a dynamic arrangement method means that the arrangement method of the intermediate coefficients of the point cloud groups in the cache changes.

[0316] In one implementation, each arrangement corresponds to an index value; the processing unit 802 is further configured to:

[0317] The index value is parsed from the encoded bitstream, and the arrangement of the point cloud groups in the cache is determined according to the index value.

[0318] In one implementation, the restriction parameter includes a delay restriction parameter; the processing unit 802 is configured to control the decoding process according to the restriction parameter, specifically to:

[0319] The delay logic in the decoding process is controlled according to the delay limit parameters.

[0320] In one implementation, the restriction parameters include a cache restriction parameter and a delay restriction parameter; the processing unit 802 is used to control the decoding process according to the restriction parameters, specifically to:

[0321] The decoding logic in the decoding process is controlled according to the buffer limit parameters and the delay limit parameters.

[0322] In one embodiment, the restriction parameter includes one or both of a buffer restriction parameter and a delay restriction parameter;

[0323] When the restriction parameter includes only one parameter, the unit of the restriction parameter includes any one or more of the following: the unit of points, the unit of groups;

[0324] When the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: point number, group number and multiple relationship.

[0325] According to one embodiment of the present application, Figure 5 and Figure 6 Some of the steps involved in the point cloud processing method shown can be represented by Figure 8 The point cloud processing device shown in the figure is executed by each unit. For example, Figure 5 Steps S501 and S502 shown in FIG. 5 can be represented by Figure 8 The acquisition unit 801 shown in FIG. 8 is executed, and step S503 can be performed by Figure 8 Processing unit 802 is shown executing. Figure 6 Steps S601 and S602 shown in FIG. 1 can be represented by Figure 8 The acquisition unit 801 shown in FIG. 8 is executed, and steps S603 to S605 can be performed by Figure 8 Processing unit 802 is shown executing. Figure 8 The various units in the point cloud processing device shown can be individually or completely combined into one or several other units, or one (some) of the units can be further divided into multiple functionally smaller units, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the point cloud processing device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0326] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM) and other processing elements and storage elements. Figure 5 and Figure 6 A computer program (including program code) for each step involved in the corresponding method shown in Figure 8 The point cloud processing device shown in and the point cloud processing method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer readable recording medium, and loaded into the above-mentioned computing device through the computer readable recording medium and run therein.

[0327] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the point cloud processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the point cloud processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0328] Fig. 9 A schematic diagram of the structure of an encoding device provided in an embodiment of the present application; the encoding device may refer to a computer device used by a provider of point cloud media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smart phone), etc.) or a server. Fig. 9 As shown, the encoding device includes a capture device 901, a processor 902, a memory 903 and a transmitter 904. Among them:

[0329] The capture device 901 is used to collect the sound-visual scene of the real world to obtain the original data of the point cloud media (including audio content and video content synchronized in time and space). The capture device 901 may include but is not limited to: audio equipment, camera equipment and sensor equipment. Among them, the audio equipment may include an audio sensor, a microphone, etc. The camera equipment may include an ordinary camera, a stereo camera, a light field camera, etc. The sensor equipment may include a laser device, a radar device, etc.

[0330] The processor 902 (or CPU (Central Processing Unit)) is the processing core of the encoding device. The processor 902 is suitable for implementing one or more program instructions, and is specifically suitable for loading and executing one or more program instructions to achieve Figure 3 and Figure 4 The flow of the point cloud processing method shown.

[0331] The memory 903 is a memory device in the encoding device, which is used to store programs and media resources. It is understandable that the memory 903 here can include both the built-in storage medium in the encoding device and the extended storage medium supported by the encoding device. It should be noted that the memory can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located away from the aforementioned processor. The memory provides a storage space, which is used to store the operating system of the encoding device. In addition, the storage space is also used to store a computer program, which includes program instructions, and the program instructions are suitable for being called and executed by the processor to perform the steps of the point cloud processing method. In addition, the memory 903 can also be used to store a point cloud media file formed after processing by the processor, and the point cloud media file includes media file resources and media presentation description information.

[0332] The transmitter 904 is used to implement the transmission interaction between the encoding device and other devices, and is specifically used to implement the transmission of point cloud media between the encoding device and the content playback device. That is, the encoding device transmits the relevant media resources of the point cloud media to the content playback device through the transmitter 904.

[0333] See also Fig. 9 , the processor 902 may include a converter 921, an encoder 922 and a packager 923; wherein:

[0334] The converter 921 is used to perform a series of conversion processes on the captured video content, so that the video content becomes content suitable for executing point cloud media video encoding. The conversion process may include: splicing and projection, and optionally, the conversion process also includes area packaging. The converter 921 can convert the captured 3D video content into a 2D image and provide it to the encoder for video encoding.

[0335] The encoder 922 is used to perform audio encoding on the captured audio content to form an audio code stream of the point cloud media, and is also used to perform video encoding on the 2D image converted by the converter 921 to obtain a video code stream.

[0336] The encapsulator 923 is used to encapsulate the audio code stream and the video code stream in a file container according to the file format of the point cloud media (such as ISOBMFF) to form a media file resource of the point cloud media. The media file resource can be a media file or a media segment to form a media file of the point cloud media; and according to the file format requirements of the point cloud media, the media presentation description information is used to record the metadata of the media file resource of the point cloud media. The encapsulated file of the point cloud media processed by the encapsulator will be stored in the storage and provided to the content playback device as needed for the presentation of the point cloud media.

[0337] The processor 902 (specifically, each device included in the processor) executes by calling one or more instructions in the memory. Figure 3 and Figure 4 Specifically, the memory 903 stores one or more first instructions, which are suitable for being loaded by the processor 902 and executing the following steps:

[0338] Set the limit parameters of the point cloud, which include one or both of cache limit parameters and delay limit parameters;

[0339] Get the attribute information of the points to be coded in the point cloud;

[0340] Encoding attribute information of the code point to be coded; and,

[0341] The encoding process is controlled according to the constraint parameters.

[0342] In one implementation, the encoding method includes an encoding method using points as units; a specific embodiment of the processor 902 encoding the attribute information of the coded point is:

[0343] The attribute information of the to-be-coded point is converted to obtain the intermediate coefficient of the to-be-coded point; the intermediate coefficient includes a residual or a transformation coefficient;

[0344] Write the intermediate coefficients of the points to be encoded into the cache;

[0345] The intermediate coefficients of the point to be encoded are read from the buffer for encoding.

[0346] In one implementation, the encoding method includes an encoding method using a group number as a unit; a specific embodiment of the processor 902 encoding the attribute information of the code point to be encoded is:

[0347] The attribute information of the coded points is grouped to obtain multiple point cloud groups;

[0348] The attribute information of the points to be encoded in the point cloud group is converted to obtain a converted point cloud group, wherein the converted point cloud group contains intermediate coefficients of the points to be encoded; the intermediate coefficients include residuals or transformation coefficients;

[0349] Write the converted point cloud groups into the cache;

[0350] The intermediate coefficients of the points to be encoded in the converted point cloud group are read from the cache in sequence for encoding.

[0351] In one implementation manner, the restriction parameter includes a cache restriction parameter; a specific embodiment in which the processor 902 controls the encoding process according to the restriction parameter is:

[0352] Control the cache logic during encoding according to the cache limit parameters.

[0353] In one implementation manner, a specific example in which the processor 902 controls the cache logic in the encoding process according to the cache limit parameter is:

[0354] Determine the cache limit point number threshold according to the cache limit parameter, and the cache limit point number threshold is used to limit the number threshold of the points to be encoded that are allowed to be written into the cache;

[0355] Whenever the number of points to be encoded written into the cache reaches the cache limit point number threshold, the control reads the intermediate coefficients of the points to be encoded from the cache for encoding.

[0356] In one implementation manner, the encoding method includes an encoding method based on the number of groups; when the number of points to be encoded written in the cache reaches the cache limit point number threshold, the processor 902 controls the intermediate coefficients of the points to be encoded to be read from the cache for encoding. A specific embodiment is as follows:

[0357] Count the total number of points to be encoded contained in each point cloud group in the cache. If the cache includes M point cloud groups and the total number of points to be encoded contained in the M point cloud groups reaches the cache limit point number threshold, then control the reading of the intermediate coefficients of the points to be encoded contained in the M point cloud groups from the cache for encoding; or,

[0358] The number of point cloud groups M contained in the cache is counted, and the total number of points to be encoded contained in the cache is calculated based on M and the point limit threshold of the point cloud group. If the total number of points to be encoded contained in the cache reaches the cache limit point threshold, the intermediate coefficients of the points to be encoded contained in the M groups of point cloud groups are read from the cache for encoding.

[0359] In one implementation, the encoding method includes an encoding method with a number of groups as a unit; a specific embodiment in which the processor 902 controls the cache logic in the encoding process according to the cache limit parameter is:

[0360] Determine the threshold of the number of point cloud groups allowed to be written into the cache according to the cache limit parameter;

[0361] Whenever the number of point cloud groups written in the cache reaches a group number threshold, the intermediate coefficients of the points to be encoded contained in the point cloud groups written in the cache are read from the cache for encoding.

[0362] In one implementation, the computer program in the memory 903 is loaded by the processor 902 and further performs the following steps:

[0363] Arrange the point cloud groups written in the cache according to the preset arrangement;

[0364] The preset arrangement methods include any of the following: a fixed arrangement method and a dynamic arrangement method; a fixed arrangement method means that the arrangement method of the intermediate coefficients of the point cloud groups in the cache is fixed, and a dynamic arrangement method means that the arrangement method of the intermediate coefficients of the point cloud groups in the cache changes.

[0365] In one implementation, each arrangement corresponds to an index value; the computer program in the memory 903 is loaded by the processor 902 and further performs the following steps:

[0366] Write the index value corresponding to the arrangement of the intermediate coefficients of the point cloud grouping in the cache into the encoded bitstream.

[0367] In one implementation, the encoding includes run length encoding; a specific embodiment in which the processor 902 controls the cache logic in the encoding process according to the cache limit parameter is:

[0368] Determine the cache limit point number threshold of the to-be-encoded points allowed to be written into the cache according to the cache limit parameter, and the cache limit point number threshold is used to limit the number threshold of the to-be-encoded points allowed to be written into the cache during the encoding process;

[0369] In the run length encoding process, setting the run length used to count consecutive specific symbols;

[0370] Set the run length to the initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache:

[0371] If an intermediate coefficient with a specific symbol is read, the count value of the run length is increased by one step; if the number of points to be encoded read is equal to the cache limit point threshold, the current run length is encoded, or the cache limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is set to the initial value for recounting;

[0372] If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0373] In one implementation manner, the restriction parameter includes a delay restriction parameter; and a specific embodiment in which the processor 902 controls the encoding process according to the restriction parameter is:

[0374] The delay logic in the encoding process is controlled according to the delay limit parameters.

[0375] In one implementation, the encoding includes run length encoding; a specific embodiment in which the processor 902 controls the delay logic in the encoding process according to the delay limit parameter is:

[0376] Determine a delay limit point number threshold according to the delay limit parameter, where the delay limit point number threshold is used to limit the number threshold of the to-be-encoded points that are allowed to be delayed during the encoding process;

[0377] In the process of run length encoding, a run length for counting consecutive specific symbols is set;

[0378] Set the run length to the initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache:

[0379] If the intermediate coefficient with a specific symbol is read, the count value of the run length is increased by one step; the count value of the run length is obtained, and if the count value of the run length reaches the delay limit point threshold, parameter encoding processing is performed, and the run length is reset to the initial value for re-counting;

[0380] If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting;

[0381] Among them, parameter coding processing refers to: encoding the current run length; or, encoding a restriction identifier, which is used to set the run length to a preset value; or, encoding the delay limit point threshold and the intermediate coefficients of the to-be-coded point whose value is a specific symbol.

[0382] In one implementation, the encoding includes run length encoding; a specific embodiment in which the processor 902 controls the delay logic in the encoding process according to the delay limit parameter is:

[0383] Determine a delay limit point number threshold according to the delay limit parameter, where the delay limit point number threshold is used to limit the number threshold of the to-be-encoded points that are allowed to be delayed during the encoding process;

[0384] In the process of run length encoding, a run length for counting consecutive specific symbols is set;

[0385] Set the run length to the initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded from the cache:

[0386] If an intermediate coefficient with a specific symbol is read, the count value of the run length is increased by one step; if the number of points to be encoded reaches the delay limit point threshold, the current run length is encoded, or the delay limit point threshold and the intermediate coefficient of the point to be encoded with a specific symbol are encoded, and the run length is reset to the initial value for recounting;

[0387] If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

[0388] In one implementation, the delay limit parameter is used to determine a delay limit point threshold; the encoding includes run length encoding, and the run length in the run length encoding is less than or equal to the delay limit point threshold; the computer program in the memory 903 is loaded by the processor 902 and further performs the following steps:

[0389] Sets the run length in bits according to the delay limit point threshold.

[0390] In one implementation manner, the restriction parameters include a cache restriction parameter and a delay restriction parameter; a specific embodiment in which the processor 902 controls the encoding process according to the restriction parameters is:

[0391] The encoding logic in the encoding process is controlled according to the cache limit parameters and the delay limit parameters.

[0392] In one implementation, the limiting parameters are set in any one or more of the following ways:

[0393] Set limit parameters to default values;

[0394] The restriction parameters are set to fixed values, and the set restriction parameters are encapsulated into the encoded bit stream formed by encoding;

[0395] The restriction parameter is set to a variable value, and the set restriction parameter is encapsulated into the encoding bit stream formed by encoding;

[0396] The value of the restriction parameter is dynamically adjusted during the encoding process, and the dynamically adjusted restriction parameter is encapsulated into the encoding bitstream formed by the encoding.

[0397] In one implementation, when the restriction parameter includes only one parameter, the unit of the restriction parameter includes any one of the following: in points, in groups;

[0398] When the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: point number, group number and multiple relationship.

[0399] In one implementation, the restriction parameter is used to determine the encoding restriction point threshold; if the restriction parameter is a cache restriction parameter, the encoding restriction point threshold is the cache restriction point threshold; if the restriction parameter is a delay restriction parameter, the encoding restriction point threshold is the delay restriction point threshold;

[0400] If the restriction parameter is in points and the value of the restriction parameter is X1, the coding restriction point threshold is the actual total number of intermediate coefficients of the X1 points to be coded;

[0401] If the restriction parameter is in groups, and the value of the restriction parameter is X2, then the coding restriction point number threshold is the actual total number of intermediate coefficients of the points to be coded in the X2 group of point cloud groups;

[0402] If the restriction parameter is in groups, and the value of the restriction parameter is X3, and the point limit of each point cloud group is X4, then the encoding restriction point threshold is the total number of intermediate coefficients of the points to be encoded in the X3 point cloud groups, and the total number does not exceed the value of X3*X4.

[0403] In one implementation, the restriction parameter includes two parameters: a cache restriction parameter and a delay restriction parameter; the cache restriction parameter is used to determine the cache restriction point threshold; the delay restriction parameter is used to determine the delay restriction point threshold;

[0404] If the cache limit parameter is in multiples, and the value of the cache limit parameter is Y1, then the cache limit point threshold is Y1 times the delay limit point threshold;

[0405] If the delay limit parameter is in multiple units, and the value of the delay limit parameter is Y2, then the delay limit point threshold is Y2 times the cache limit point threshold.

[0406] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the encoding device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the point cloud processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0407] Fig.10 A schematic diagram of a decoding device provided in an embodiment of the present application; the decoding device may refer to a computer device used by a user of point cloud media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smart phone), a VR device (such as a VR helmet, VR glasses, etc.)). Fig.10 As shown, the decoding device includes a receiver 1001, a processor 1002, a memory 1003, and a display / playing device 1004. Among them:

[0408] The receiver 1001 is used to implement the transmission interaction between decoding and other devices, and is specifically used to implement the transmission of point cloud media between the encoding device and the decoding device. That is, the decoding device receives the relevant media resources of the point cloud media transmitted by the encoding device through the receiver 1001.

[0409] The processor 1002 (or CPU (Central Processing Unit)) is the processing core of the encoding device. The processor 1002 is suitable for implementing one or more program instructions, and is specifically suitable for loading and executing one or more program instructions to achieve Figure 5and Figure 6 The flow of the point cloud processing method shown.

[0410] The memory 1003 is a memory device in the decoding device, which is used to store programs and media resources. It is understandable that the memory 1003 here can include both the built-in storage medium in the decoding device and the extended storage medium supported by the decoding device. It should be noted that the memory 1003 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located away from the aforementioned processor. The memory 1003 provides a storage space, which is used to store the operating system of the decoding device. In addition, the storage space is also used to store a computer program, which includes program instructions, and the program instructions are suitable for being called and executed by the processor to perform each step of the point cloud processing method. In addition, the memory 1003 can also be used to store the three-dimensional image of the point cloud media formed after processing by the processor, the audio content corresponding to the three-dimensional image, and the information required for rendering the three-dimensional image and audio content.

[0411] The display / playback device 1004 is used to output the rendered sound and three-dimensional image.

[0412] See also Fig.10 , the processor 1002 may include a parser 1021, a decoder 1022, a converter 1023 and a renderer 1024; wherein:

[0413] The parser 1021 is used to decapsulate the encapsulated files of the rendered media from the encoding device, specifically to decapsulate the media file resources according to the file format requirements of the point cloud media to obtain audio code streams and video code streams; and provide the audio code streams and video code streams to the decoder 1022.

[0414] The decoder 1022 performs audio decoding on the audio code stream to obtain audio content and provides it to the renderer for audio rendering. In addition, the decoder 1022 decodes the video code stream to obtain a 2D image. According to the metadata provided by the media presentation description information, if the metadata indicates that the point cloud media has performed a region encapsulation process, the 2D image refers to a packaged image; if the metadata indicates that the point cloud media has not performed a region encapsulation process, the plane image refers to a projected image.

[0415] The converter 1023 is used to convert a 2D image into a 3D image. If the point cloud media has performed a regional encapsulation process, the converter 1023 will first perform regional decapsulation on the encapsulated image to obtain a projection image. The projection image is then reconstructed to obtain a 3D image. If the rendering media has not performed a regional encapsulation process, the converter 1023 will directly reconstruct the projection image to obtain a 3D image.

[0416] The renderer 1024 is used to render the audio content and 3D images of the point cloud media. Specifically, the audio content and 3D images are rendered according to the metadata related to the rendering and the window in the media presentation description information, and the rendering is output to the display / playback device after completion.

[0417] In an exemplary embodiment, the processor 1002 (specifically, each device included in the processor) executes by calling one or more instructions in the memory. Figure 5 and Figure 6 Specifically, the memory stores one or more first instructions, which are suitable for being loaded by the processor 1002 and executing the following steps:

[0418] Get the encoded code stream of the point to be decoded in the point cloud;

[0419] Obtaining restriction parameters of the point cloud, the restriction parameters including one or both of a cache restriction parameter and a delay restriction parameter;

[0420] Decoding the coded bitstream of the decoding point; and

[0421] The decoding process is controlled according to the constraint parameters.

[0422] In one embodiment, the restriction parameters for obtaining the point cloud include any one or more of the following:

[0423] Set limit parameters to default values;

[0424] Parse the restriction parameters that have been set to fixed values ​​from the encoded bitstream;

[0425] Parse the restriction parameters that have been set as variable values ​​from the encoded bitstream;

[0426] Parse the set dynamically adjusted restriction parameters from the encoded bitstream.

[0427] In one implementation manner, the decoding method includes a decoding method based on points; a specific embodiment of the processor 1002 decoding the coded bit stream of the decoding point is:

[0428] Parse the coded bit stream of the point to be decoded to obtain the intermediate coefficient of the point to be decoded; the intermediate coefficient includes a residual or a transform coefficient;

[0429] Write the intermediate coefficients of the points to be decoded into the cache;

[0430] The intermediate coefficients of the point to be decoded are read from the cache for decoding.

[0431] In one implementation manner, the decoding method includes a decoding method based on the number of groups; a specific embodiment of the processor 1002 decoding the coded bit stream of the decoding point is:

[0432] Parse the coded bit stream of the point to be decoded to obtain multiple groups of point cloud groups, each of which contains intermediate coefficients of the point to be decoded; the intermediate coefficients include residuals or transform coefficients;

[0433] Write point cloud groups into the cache;

[0434] The intermediate coefficients of the points to be decoded in the point cloud group are read from the cache in sequence for decoding.

[0435] In one implementation, decoding includes run length decoding; a specific example in which the processor 1002 controls the decoding process according to the restriction parameters is:

[0436] Determine a limit point number threshold according to the limit parameter, and the limit point number threshold is used to limit the number threshold of points to be decoded;

[0437] During the process of run decoding, if the limit flag is read, the count value of the first run length is set to a preset value;

[0438] If the count value of the first run length is greater than zero and is not equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0439] The intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded.

[0440] In one implementation, decoding includes run length decoding; a specific example in which the processor 1002 controls the decoding process according to the restriction parameters is:

[0441] Determine a limit point number threshold according to the limit parameter, and the limit point number threshold is used to limit the number threshold of points to be decoded;

[0442] During the run length decoding process, a count value of a first run length is read from the encoded code stream;

[0443] If the count value of the first run length is greater than zero and equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0444] Reading a count value of a second run length from the encoded bitstream, if the count value of the second run length is greater than zero and is not equal to the limit point number threshold, setting the intermediate coefficient of the current point to be decoded to a preset value, and reducing the count value of the second run length by one step until the count value of the second run length is equal to zero;

[0445] The intermediate coefficient of the point to be decoded read when the count value of the second run length is equal to zero is obtained and decoded.

[0446] In one implementation, decoding includes run length decoding; a specific example in which the processor 1002 controls the decoding process according to the restriction parameters is:

[0447] During the run length decoding process, a count value of a first run length is read from the encoded code stream;

[0448] If the count value of the first run length is greater than zero, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero;

[0449] The intermediate coefficient of the point to be decoded read when the count value of the first run length is equal to zero is obtained and decoded.

[0450] In one implementation manner, the restriction parameter includes a cache restriction parameter; a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameter is:

[0451] The cache logic during decoding is controlled according to the cache limit parameters.

[0452] In one implementation manner, a specific example in which the processor 1002 controls the cache logic in the decoding process according to the cache limit parameter is:

[0453] Determine a cache limit point number threshold according to a cache limit parameter, where the cache limit point number threshold is used to limit the number of points to be decoded that are allowed to be written into the cache;

[0454] Whenever the number of points to be decoded written into the cache reaches the cache limit point number threshold, the control reads the intermediate coefficients of the points to be decoded from the cache for decoding.

[0455] In one implementation manner, the decoding method includes a decoding method based on the number of groups; when the number of points to be decoded written in the cache reaches the cache limit point number threshold, the processor 1002 controls the reading of the intermediate coefficients of the points to be decoded from the cache for decoding. The specific implementation is:

[0456] Count the total number of points to be decoded contained in each point cloud group in the cache. If the cache includes M point cloud groups and the total number of points to be decoded contained in the M point cloud groups reaches the cache limit point number threshold, then control the reading of the intermediate coefficients of the points to be decoded contained in the M point cloud groups from the cache for decoding; or,

[0457] The number of point cloud groups M contained in the cache is counted, and the total number of points to be encoded contained in the cache is calculated based on M and the point limit threshold of the point cloud group. If the total number of points to be encoded contained in the cache reaches the cache limit point threshold, the intermediate coefficients of the points to be decoded contained in the M groups of point cloud groups are read from the cache for decoding.

[0458] In one implementation manner, the decoding method includes a decoding method based on the number of groups; a specific embodiment in which the processor 1002 controls the cache logic in the decoding process according to the cache limit parameter is:

[0459] Determine the threshold of the number of point cloud groups allowed to be written into the cache according to the cache limit parameter;

[0460] Whenever the number of point cloud groups written in the cache reaches a group number threshold, the control reads the intermediate coefficients of the points to be decoded contained in the point cloud groups written in the cache from the cache for decoding.

[0461] In one implementation, the computer program in the memory 1003 is loaded by the processor 1002 and further performs the following steps:

[0462] Arrange the point cloud groups written in the cache according to the preset arrangement;

[0463] The preset arrangement methods include any of the following: a fixed arrangement method and a dynamic arrangement method; a fixed arrangement method means that the arrangement method of the intermediate coefficients of the point cloud groups in the cache is fixed, and a dynamic arrangement method means that the arrangement method of the intermediate coefficients of the point cloud groups in the cache changes.

[0464] In one implementation, each arrangement corresponds to an index value; the computer program in the memory 1003 is loaded by the processor 1002 and further performs the following steps:

[0465] The index value is parsed from the encoded bitstream, and the arrangement of the point cloud groups in the cache is determined according to the index value.

[0466] In one implementation manner, the restriction parameter includes a delay restriction parameter; and a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameter is:

[0467] The delay logic in the decoding process is controlled according to the delay limit parameters.

[0468] In one implementation manner, the restriction parameters include a cache restriction parameter and a delay restriction parameter; a specific embodiment in which the processor 1002 controls the decoding process according to the restriction parameters is:

[0469] The decoding logic in the decoding process is controlled according to the buffer limit parameters and the delay limit parameters.

[0470] In one implementation, the restriction parameter includes one or both of a cache restriction parameter and a delay restriction parameter;

[0471] When the restriction parameter includes only one parameter, the unit of the restriction parameter includes any one or more of the following: the unit of points, the unit of groups;

[0472] When the restriction parameter includes two parameters, the unit of the restriction parameter includes any one or more of the following: point number, group number and multiple relationship.

[0473] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the decoding device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the point cloud processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0474] An embodiment of the present application also provides a computer-readable storage medium, in which one or more instructions are stored, and the one or more instructions are suitable for being loaded by a processor and executing the point cloud processing method of the above method embodiment.

[0475] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the point cloud processing method of the above method embodiment.

[0476] The embodiment of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned point cloud processing method.

[0477] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0478] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0479] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the readable storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0480] What is disclosed above is only a preferred embodiment of the present application, and it certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiment and equivalent changes made according to the claims of the present application are still within the scope covered by the application.

Claims

1. A point cloud processing method, characterized in that: include: Get the encoded code stream of the point cloud; If the encoding code stream encapsulates the restriction parameters of the point cloud, parsing the restriction parameters of the point cloud from the encoding code stream; Decoding the encoded code stream and controlling the decoding process according to the restriction parameters of the point cloud; The restriction parameters of the point cloud parsed from the encoded code stream include fixed values ​​or variable values.

2. The method according to claim 1, characterized in that The restriction parameter is used to limit the number threshold of points to be decoded during the decoding process; the restriction parameter includes one or both of a cache restriction parameter and a delay restriction parameter; The cache limit parameter is used to determine the cache limit point threshold, and the number of points to be decoded in the decoding process is less than or equal to the cache limit point threshold; the cache limit parameter is also used to determine the group number threshold, and the number of point cloud groups in the decoding process is less than or equal to the group number threshold; The delay limit parameter is used to determine a delay limit point number threshold; the delay limit point number threshold is used to limit a threshold number of points to be decoded that are allowed to be delayed during the decoding process.

3. The method according to claim 1 or 2, characterized in that The decoding includes run-length decoding; and the controlling the decoding process according to the restriction parameters of the point cloud includes: Determine a limit point number threshold according to the limit parameter, wherein the limit point number threshold is used to limit the number threshold of points to be decoded; During the process of the run decoding, if a limit flag is read, the count value of the first run length is set to a preset value; If the count value of the first run length is greater than zero and is not equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero; The intermediate coefficient of the point to be decoded that is read when the count value of the first run length is equal to zero is obtained and decoded.

4. The method according to claim 1 or 2, characterized in that: The decoding includes run-length decoding; and the controlling the decoding process according to the restriction parameters of the point cloud includes: Determine a limit point number threshold according to the limit parameter, wherein the limit point number threshold is used to limit the number threshold of points to be decoded; During the process of the run decoding, if a limit flag is read, the first run length is parsed to obtain a count value of the first run length; If the count value of the first run length is greater than zero and is not equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero; The intermediate coefficient of the point to be decoded that is read when the count value of the first run length is equal to zero is obtained and decoded.

5. The method according to claim 1 or 2, characterized in that: The decoding includes run-length decoding; and the controlling the decoding process according to the restriction parameters of the point cloud includes: Determine a limit point number threshold according to the limit parameter, wherein the limit point number threshold is used to limit the number threshold of points to be decoded; During the run length decoding, a count value of a first run length is read from the encoded code stream; If the count value of the first run length is greater than zero and equal to the limit point number threshold, the intermediate coefficient of the current point to be decoded is set to a specific value, and the count value of the first run length is reduced by one step until the count value of the first run length is equal to zero; Reading a count value of a second run length from the encoded bitstream, and if the count value of the second run length is greater than zero and not equal to the limit point number threshold, setting the intermediate coefficient of the current point to be decoded to a specific value, and reducing the count value of the second run length by one step until the count value of the second run length is equal to zero; The intermediate coefficient of the point to be decoded read when the count value of the second run length is equal to zero is obtained and decoded.

6. The method according to claim 1, characterized in that The decoding method of the decoding includes a decoding method using points as units; the decoding of the encoded code stream includes: Parsing the encoded bitstream to obtain intermediate coefficients of the points to be decoded in the point cloud; the intermediate coefficients include residuals or transform coefficients; The intermediate coefficients of the points to be decoded are decoded.

7. The method according to claim 6, characterized in that The restriction parameters include cache restriction parameters; and controlling the decoding process according to the restriction parameters of the point cloud includes: Determine a cache limit point threshold according to the cache limit parameter; Whenever the number of points to be decoded in the decoding process reaches the cache limit point number threshold, the intermediate coefficients of the points to be decoded are controlled to be decoded.

8. A point cloud processing method, characterized in that: include: Set the limit parameters of the point cloud; Obtaining attribute information of the points to be encoded in the point cloud; Encoding the attribute information of the point to be encoded; as well as, Controlling the encoding process according to the restriction parameters; If the restriction parameters of the point cloud are set to fixed values ​​or variable values, the restriction parameters of the point cloud are encapsulated into the encoding code stream formed by the encoding.

9. The method according to claim 8, characterized in that The restriction parameters include cache restriction parameters; the encoding includes run length encoding; and the controlling the encoding process according to the restriction parameters includes: Determine a cache limit point threshold according to the cache limit parameter; In the run length encoding process, setting the run length used to count consecutive specific symbols; The run length is set to an initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded: If an intermediate coefficient with a value of a specific symbol is read, the count value of the run length is increased by one step; if the number of points to be encoded read is equal to the cache limit point threshold, the current run length is encoded, or the cache limit point threshold and the intermediate coefficient of the point to be encoded with a value of a specific symbol are encoded, and the run length is set to the initial value for recounting; If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for recounting.

10. The method according to claim 8, characterized in that The restriction parameters include delay restriction parameters; the encoding includes run length encoding; and the controlling the encoding process according to the restriction parameters includes: Determine a delay limit point threshold according to the delay limit parameter; In the process of run length encoding, setting a run length for counting consecutive specific symbols; The run length is set to an initial value, and in the process of sequentially reading the intermediate coefficients of the points to be encoded: If an intermediate coefficient with a value of a specific symbol is read, the count value of the run length is increased by one step; the count value of the run length is obtained, and if the count value of the run length reaches the delay limit point threshold, parameter encoding processing is performed, and the run length is reset to the initial value for recounting; If the intermediate coefficient of the to-be-encoded point whose value is a non-specific symbol is read, the current run length and the intermediate coefficient of the non-specific symbol are encoded, and the run length is reset to the initial value for re-counting; Among them, the parameter encoding processing refers to: encoding the current run length; or, encoding a restriction identifier, which is used to set the run length to a preset value; or, encoding the delay limit point threshold and the intermediate coefficients of the points to be encoded whose values ​​are specific symbols.

11. The method according to any one of claims 8 to 10, characterized in that: The point cloud restriction parameters may include any one or more of the following: Setting the restriction parameters to default values; Setting the limiting parameter to a fixed value; Setting the limit parameter to a variable value; The value of the restriction parameter is dynamically adjusted during the encoding process.

12. A point cloud processing device, characterized in that: The device comprises: An acquisition unit, used for acquiring a coded code stream of a point cloud; The acquisition unit is further configured to parse the restriction parameters of the point cloud from the encoded code stream if the restriction parameters of the point cloud are encapsulated in the encoded code stream; A processing unit, configured to decode the encoded code stream and control the decoding process according to the restriction parameters of the point cloud; The restriction parameters of the point cloud parsed from the encoded code stream include fixed values ​​or variable values.

13. A point cloud processing device, characterized in that: The device comprises: Setting unit, used to set the limit parameters of the point cloud; An acquisition unit, used to acquire attribute information of the points to be encoded in the point cloud; A processing unit, configured to encode the attribute information of the point to be encoded; and to control the encoding process according to the restriction parameter; If the restriction parameters of the point cloud are set to fixed values ​​or variable values, the restriction parameters of the point cloud are encapsulated into the encoding code stream formed by the encoding.

14. A computer device, characterized in that: The computer device comprises: a processor suitable for implementing a computer program; A computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by the processor and executing the point cloud processing method as described in any one of claims 1 to 7, or the point cloud processing method as described in any one of claims 8 to 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the point cloud processing method according to any one of claims 1 to 7, or the point cloud processing method according to any one of claims 8 to 11.

16. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the point cloud processing method as described in any one of claims 1 to 7, or implements the steps of the point cloud processing method as described in any one of claims 8 to 11.

Images