Coding method, apparatus and device of point cloud, storage medium and program product

Abstract

Embodiments of the present application disclose a point cloud coding and decoding method, device, equipment, storage medium and program product, and belong to the technical field of data processing. In the embodiments of the present application, an N-ary tree is used to describe a space in which a point cloud is located. In the coding and decoding process of the point cloud, instead of inputting an occupancy information represented by eight-bit binary into a context prediction network after converting the occupancy information into a decimal number, occupancy rates of each peripheral node of a current node are input into the context prediction network to predict a probability distribution of the current node. The occupancy rate represents a probability that a corresponding peripheral node corresponds to a space in which a point exists. It can be seen that the present scheme inputs the occupancy rates of each peripheral node into the context prediction network, so that the context prediction network does not mistakenly think that the spaces corresponding to each peripheral node have different weights, thereby accurately understanding the occupancy rates of each peripheral node. In this way, the prediction accuracy of the context prediction network can be improved, and the performance of coding and decoding is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a point cloud encoding / decoding method, apparatus, device, storage medium, and program product. Background Technology

[0002] A point cloud is a dataset of points in a specific space, containing the three-dimensional coordinates of those points. It allows for the reconstruction of the three-dimensional real world. Because the scale of point clouds being collected is increasingly large, encoding and decoding of point clouds is necessary.

[0003] An octree is a tree-like data structure for describing three-dimensional space. Therefore, related technologies construct octrees to describe the space containing point clouds. An octree consists of multiple nodes, each describing a space; that is, each node corresponds to one space, and the sum of the spaces corresponding to these nodes is the space containing the point cloud. Furthermore, each parent node has eight child nodes, and the sum of the eight spaces corresponding to these eight child nodes is the space corresponding to the parent node. Additionally, each node in the octree is assigned a value of 0 or 1 depending on whether it contains points within its corresponding space. Moreover, the values ​​of the eight child nodes of the same parent node form an 8-bit binary number, which is called the occupancy information of the parent node, used to represent the distribution of points within the eight spaces corresponding to the parent node's eight child nodes.

[0004] In the point cloud encoding process, the encoder traverses each node in the octree. Upon reaching a node, it determines the surrounding nodes from the already encoded nodes and converts the occupancy information of these surrounding nodes (such as the parent node) into a decimal number between 0 and 255. This decimal number is then input into the context prediction network to predict the probability distribution of the currently traversed node. Finally, based on the probability distribution of the currently traversed node, its occupancy information is encoded into the bitstream. The point cloud decoding process corresponds to the encoding process.

[0005] However, converting an eight-bit binary number to a decimal number is equivalent to assigning a weight to each bit of that eight bits. The weights for these eight bits are 128, 64, 32, 16, 8, 4, 2, and 1, respectively. These eight bits correspond to eight spatial representations, which can mislead people into believing that these eight spatial representations have different weights. In reality, these eight spatial representations have no inherent hierarchy of weights. Therefore, related technologies misunderstand the distribution of points within these eight spatial representations using eight-bit binary numbers, leading to lower prediction accuracy in context prediction networks and consequently, poor encoding and decoding performance. Summary of the Invention

[0006] This application provides a point cloud encoding / decoding method, apparatus, device, storage medium, and program product, which can, to a certain extent, solve the problem of context prediction networks misunderstanding the distribution of points in space, thereby improving prediction accuracy and encoding / decoding performance. The technical solution is as follows:

[0007] Firstly, a point cloud encoding method is provided, which includes:

[0008] An N-ary tree is determined, which describes the space where the point cloud resides, where N is an integer greater than 1. The input data corresponding to the current node is fed into the context prediction network to obtain the probability distribution of the current node. This input data includes the occupancy rate of each of the current node's one or more neighboring nodes. The occupancy rate represents the probability of a point existing in the space corresponding to the corresponding neighboring node. The current node is a node to be encoded in the N-ary tree, and the probability distribution of the current node represents the probability of points existing in multiple spaces corresponding to the multiple child nodes of the current node. Based on the probability distribution of the current node, the occupancy information of the current node is encoded into the bitstream. This occupancy information represents the distribution of points in multiple spaces corresponding to the multiple child nodes of the current node. In other words, an N-ary tree is first constructed based on the point cloud, and then the point cloud is compressed by encoding the occupancy information of the nodes in the N-ary tree.

[0009] In this embodiment, an N-ary tree is used to describe the space of the point cloud. During the point cloud encoding process, instead of converting the occupancy information (represented by eight bits) into decimal numbers and inputting it into the context prediction network, the occupancy rates of each neighboring node are input into the context prediction network to predict the probability distribution of the current node. This scheme, by independently inputting each occupancy rate into the context prediction network, prevents the network from mistakenly believing that the spaces corresponding to each neighboring node have different weights, thus accurately understanding the occupancy rate of each neighboring node. This improves the prediction accuracy of the context prediction network and enhances the encoding and decoding performance.

[0010] It should be noted that the current node is a non-leaf node in the N-ary tree to be encoded; that is, the occupancy information of the non-leaf nodes in the N-ary tree is encoded into the bitstream. For leaf nodes, the occupancy information of the non-leaf nodes includes the occupancy status of the leaf nodes, which is the actual situation of whether a point exists in the space corresponding to the leaf node. When the occupancy information of the last non-leaf node in the N-ary tree is encoded into the bitstream, the point cloud encoding based on the N-ary tree is completed.

[0011] Furthermore, in this embodiment, an N-ary tree is constructed by recursively partitioning the space containing the point cloud. Each node in the N-ary tree corresponds to a space. The space corresponding to the root node of the N-ary tree is the overall space (i.e., the large space) containing the point cloud. The spaces corresponding to nodes other than the root node are local spaces (i.e., small spaces or subspaces), which are parts of the overall space containing the point cloud. Each parent node (i.e., a node with child nodes) in the N-ary tree contains points from the point cloud within its corresponding space. Leaf nodes with a depth less than the maximum depth of the N-ary tree do not contain points within their corresponding spaces. Leaf nodes with a depth equal to the maximum depth of the N-ary tree may or may not contain points from the point cloud within their corresponding spaces. In other words, the space is partitioned based on the presence or absence of points. Before reaching the smallest granularity of space, spaces without points are not further subdivided into subspaces, while spaces with points are further subdivided.

[0012] Optionally, N can be a constant or a variable. When N is a constant, the number of child nodes of each parent node in an N-ary tree is N. For example, a binary tree, quadtree, or octree can all have N child nodes. When N is a variable, some parent nodes in an N-ary tree have N child nodes, while the number of child nodes of other parent nodes is not equal to N; it can be greater than or less than N. For example, when N is a variable, one or more parts of an N-ary tree may be binary trees, quadtrees, or octrees, while the rest may be octrees. Another example is an N-ary tree where one or more parts are octrees, and the rest are quadtrees.

[0013] In this embodiment, the current node's one or more surrounding nodes include one or more first surrounding nodes, whose parent node is an encoded node. The occupancy rate of the first surrounding nodes is a first value or a second value, where the first value indicates that no point exists in the corresponding space, and the second value indicates that a point exists in the corresponding space. The one or more surrounding nodes also include one or more second surrounding nodes, whose parent node is an uncoded node. The occupancy rate of the second surrounding nodes is a predetermined value, which is greater than the first value and less than the second value. That is, in this embodiment, the probability distribution prediction can be based not only on the relevant information of some encoded surrounding nodes but also on the relevant information of some uncoded surrounding nodes. This provides rich features for the probability distribution prediction and improves the prediction accuracy of the context prediction network to a certain extent.

[0014] Optionally, the one or more surrounding nodes include the sibling nodes of the current node and the child nodes of the sibling nodes of the current node. It is evident that the spatial relationships of the surrounding nodes are close to the spatial relationships of the current node, which improves the prediction accuracy of the probability distribution.

[0015] Optionally, the context prediction network includes a feature fusion network and a probability distribution prediction network. Inputting the input data corresponding to the current node into the context prediction network to obtain the probability distribution of the current node includes: inputting the input data corresponding to the current node into the feature fusion network to obtain fused features; and inputting the fused features into the probability distribution prediction network to obtain the probability distribution of the current node. That is, the various input features are fused first, and then probability distribution prediction is performed.

[0016] Optionally, the occupancy of the one or more peripheral nodes is represented by a floating-point number of one or more channels, with each peripheral node corresponding to one or more channels. In other words, the context prediction network has multiple input data channels, the occupancy of each peripheral node is represented by a floating-point number, and each occupancy is input to the context prediction network through one channel.

[0017] Secondly, a point cloud decoding method is provided, which includes:

[0018] The process involves identifying one or more neighboring nodes of the current node in an N-ary tree to be reconstructed. This N-ary tree describes the space in which the point cloud resides. The current node is a node to be decoded within the N-ary tree, where N is an integer greater than 1. The input data of the current node is fed into a context prediction network to obtain its probability distribution. This input data includes the occupancy rate of each of the one or more neighboring nodes, representing the probability of a point existing in the space corresponding to that neighboring node. The probability distribution of the current node represents the probability of points existing in multiple spaces corresponding to its multiple child nodes. Based on the probability distribution of the current node, its occupancy information is parsed from the bitstream. This occupancy information represents the distribution of points in multiple spaces corresponding to the multiple child nodes of the current node. The N-ary tree is then reconstructed based on the occupancy information of the multiple nodes, including the current node, to reconstruct the point cloud. In other words, the occupancy information of each node in the N-ary tree to be reconstructed is sequentially parsed from the bitstream to reconstruct the N-ary tree and thus the point cloud.

[0019] In this embodiment, an N-ary tree is used to describe the space of the point cloud. During the decoding process of the point cloud, instead of converting the occupancy information represented by eight bits into decimal numbers and inputting it into the context prediction network, the occupancy rates of each neighboring node of the current node are input into the context prediction network to predict the probability distribution of the current node. It is evident that this scheme, by independently inputting each occupancy rate into the context prediction network, prevents the context prediction network from mistakenly believing that the spaces corresponding to each neighboring node have different weights, thus accurately understanding the occupancy rate of each neighboring node. This improves the prediction accuracy of the context prediction network, thereby enhancing the encoding and decoding performance.

[0020] It should be noted that the current node is a non-leaf node to be decoded in the N-ary tree to be reconstructed. Parsing the occupancy information of non-leaf nodes in the N-ary tree from the bitstream allows for the reconstruction of the N-ary tree. For leaf nodes, the occupancy information of non-leaf nodes includes the occupancy status of the leaf nodes, which indicates whether a point actually exists within the space of the leaf node pair. When the occupancy information of the last non-leaf node in the N-ary tree is parsed from the bitstream, the N-ary tree is reconstructed.

[0021] Optionally, N can be a constant or a variable. It should be noted that the structure of the reconstructed N-ary tree during the decoding process is the same as the structure of the N-ary tree constructed during the encoding process.

[0022] Optionally, the current node's one or more surrounding nodes include one or more first surrounding nodes, whose parent node is a decoded node. The occupancy rate of the first surrounding nodes is a first value or a second value, where the first value indicates that no point exists in the corresponding space, and the second value indicates that a point exists in the corresponding space. The one or more surrounding nodes also include one or more second surrounding nodes, whose parent node is an undecoded node. The occupancy rate of the second surrounding nodes is a predetermined value, which is greater than the first value and less than the second value. That is, in this embodiment, the probability distribution prediction can be based not only on the relevant information of some decoded surrounding nodes but also on the relevant information of some undecoded surrounding nodes. This provides rich features for the probability distribution prediction and improves the prediction accuracy of the context prediction network to a certain extent.

[0023] It should be noted that the neighboring nodes of the current node during decoding are the same as those during encoding. Optionally, one or more neighboring nodes of the current node include the sibling nodes of the current node and the child nodes of the sibling nodes. It is evident that the spatial relationships of the neighboring nodes are close to the spatial relationships of the current node, which improves the prediction accuracy of the probability distribution.

[0024] Optionally, the context prediction network includes a feature fusion network and a probability distribution prediction network. Inputting the input data of the current node into the context prediction network to obtain the probability distribution of the current node includes: inputting the input data corresponding to the current node into the feature fusion network to obtain fused features; and inputting the fused features into the probability distribution prediction network to obtain the probability distribution of the current node. That is, the various input features are fused first, and then probability distribution prediction is performed.

[0025] Optionally, the occupancy of the one or more peripheral nodes is represented by a floating-point number of one or more channels, with each peripheral node corresponding to one or more channels. In other words, the context prediction network has multiple input data channels, the occupancy of each peripheral node is represented by a floating-point number, and each occupancy is input to the context prediction network through one channel.

[0026] Thirdly, a point cloud encoding apparatus is provided, which has the function of implementing the point cloud encoding method described in the first aspect above. The encoding apparatus includes one or more modules for implementing the point cloud encoding method provided in the first aspect above.

[0027] Fourthly, a point cloud decoding apparatus is provided, which has the function of implementing the point cloud decoding method described in the second aspect above. The decoding apparatus includes one or more modules for implementing the point cloud decoding method provided in the second aspect above.

[0028] Fifthly, a point cloud encoding device is provided, the device comprising:

[0029] One or more processors;

[0030] A non-transitory computer-readable storage medium is coupled to the processor and stores program instructions that are executed by the processor, wherein, when executed by the processor, the program instructions cause the encoding device to perform the point cloud encoding method provided in the first aspect above.

[0031] Sixthly, a point cloud decoding device is provided, the device comprising:

[0032] One or more processors;

[0033] A non-transitory computer-readable storage medium is coupled to the processor and stores program instructions that are executed by the processor, wherein, when executed by the processor, the program instructions cause the encoding device to perform the point cloud decoding method provided in the second aspect above.

[0034] In a seventh aspect, an encoder is provided, including processing circuitry for performing the point cloud encoding method provided in the first aspect above.

[0035] Eighthly, a decoder is provided, including processing circuitry for performing the point cloud decoding method provided in the second aspect above.

[0036] A ninth aspect provides an electronic device comprising a processor and a memory, the memory being used to store a program for executing the point cloud encoding method provided in the first aspect and / or the point cloud decoding method provided in the second aspect, and to store data related to implementing the point cloud encoding method provided in the first aspect and / or the point cloud decoding method provided in the second aspect. The processor is configured to execute the program stored in the memory. The operating means of the storage device may further include a communication bus for establishing a connection between the processor and the memory.

[0037] In a tenth aspect, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer or processor, implement the point cloud encoding method described in the first aspect or the point cloud decoding method described in the second aspect.

[0038] In the eleventh aspect, a computer program product containing instructions is provided, including a computer program that, when run on a computer or processor, implements the point cloud encoding method described in the first aspect or the point cloud decoding method described in the second aspect.

[0039] The technical effects achieved by the third to eleventh aspects mentioned above are similar to those achieved by the corresponding technical means in the first or second aspects, and will not be repeated here.

[0040] The technical solutions provided in this application can bring at least the following beneficial effects:

[0041] In this embodiment, an N-ary tree is used to describe the space where the point cloud is located. During the encoding and decoding process of the point cloud, instead of converting the occupancy information represented by eight bits into decimal numbers and inputting it into the context prediction network, the occupancy rates of each neighboring node of the current node are input into the context prediction network to predict the probability distribution of the current node. Here, the occupancy rate represents the probability of a point existing in the space corresponding to the corresponding neighboring node. It is evident that this scheme, by inputting the occupancy rates of each neighboring node into the context prediction network, prevents the context prediction network from mistakenly believing that the spaces corresponding to each neighboring node have different weights, thus accurately understanding the occupancy rates of each neighboring node. This improves the prediction accuracy of the context prediction network, thereby enhancing the encoding and decoding performance. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of a point cloud applicable to embodiments of this application;

[0043] Figure 2 This is a schematic block diagram of a point cloud decoding system provided in an embodiment of this application;

[0044] Figure 3 This is an illustrative diagram illustrating an example of a point cloud decoding system provided in an embodiment of this application;

[0045] Figure 4 This is a schematic diagram of a point cloud encoding / decoding algorithm framework provided in an embodiment of this application;

[0046] Figure 5 This is a flowchart of a point cloud encoding method provided in an embodiment of this application;

[0047] Figure 6 This is a schematic diagram of an N-ary tree structure provided in an embodiment of this application;

[0048] Figure 7 This is a schematic diagram of another N-ary tree structure provided in an embodiment of this application;

[0049] Figure 8 This is a schematic diagram of the structure of a context prediction network provided in an embodiment of this application;

[0050] Figure 9 This is a schematic diagram of the framework of an LSTM network provided in an embodiment of this application;

[0051] Figure 10 This is a schematic diagram of another context prediction network structure provided in an embodiment of this application;

[0052] Figure 11 This is a flowchart of a point cloud decoding method provided in an embodiment of this application;

[0053] Figure 12 This is a schematic diagram of the structure of a point cloud encoding device provided in an embodiment of this application;

[0054] Figure 13 This is a schematic diagram of the structure of a point cloud decoding device provided in an embodiment of this application;

[0055] Figure 14 This is a schematic block diagram illustrating one implementation of the decoding apparatus used in embodiments of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0057] First, it should be noted that the system architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0058] To facilitate understanding, some terms involved in the embodiments of this application will be explained first.

[0059] Point cloud: A dataset of points in a specific space, containing the three-dimensional coordinates of those points. Point clouds can be used to reconstruct the three-dimensional real world. In some embodiments, point clouds also contain one or more pieces of information such as the color, classification value, intensity value, and time of the points in space. Figure 1 This is a schematic diagram of a point cloud applicable to embodiments of this application. Figure 1 The point cloud shown is a collected point cloud of a human body. This point cloud is a dataset of points included in the human body in space. Based on this point cloud, a three-dimensional human body can be reconstructed.

[0060] N-ary trees: Taking octrees as an example, an octree is a tree-like data structure for describing three-dimensional space. By constructing an octree, the space containing the point cloud is described. Each node in an octree describes a space; that is, each node corresponds to one space. The root node corresponds to the overall space containing the point cloud, and each node other than the root node corresponds to a local space. Each node in an octree, except for the leaf nodes, has eight child nodes. The sum of the eight spaces corresponding to the eight child nodes is the space corresponding to the parent node. In this embodiment, N is an integer greater than 1. N can be a fixed value; for example, an N-ary tree can be a binary tree, a quadtree, an octree, etc. Alternatively, in some embodiments, N is a variable. For example, one or more parts of an N-ary tree may be binary trees, quadtrees, or octrees, while other parts are octrees. Or, for example, one or more parts of an N-ary tree may be octrees, while other parts are quadtrees. For example, in one example of constructing an N-ary tree based on point clouds, the space is first partitioned according to N=4 to construct the shallow part of the N-ary tree. When a certain depth is reached, the space is then partitioned according to N=8 to construct the deep part of the N-ary tree. In another example, the space is first partitioned according to N=8 to construct the shallow part of the N-ary tree. When the spatial granularity of the partitioning reaches a certain threshold, the space is then partitioned according to N=4 to construct the deeper part of the N-ary tree. When the smallest granularity of the spatial partitioning is reached, the space is then partitioned according to N=8 to construct the deepest part of the N-ary tree, i.e., the deepest leaf node.

[0061] Occupancy: Based on whether a point exists in the space corresponding to each node in the N-ary tree, assign a value of 0 or 1 to the corresponding node to obtain the occupancy status of the node. Occupancy status indicates whether there is a point in the point cloud within the corresponding node.

[0062] Occupancy: In the point cloud encoding process based on N-ary trees, context prediction is performed based on the occupancy of one or more neighboring nodes of the current node to obtain the probability distribution of the current node. Occupancy represents the probability of a point existing in the space corresponding to a neighboring node. If a neighboring node is already encoded, its occupancy is its occupied status. If a neighboring node is not encoded, its occupancy is a predetermined value.

[0063] Occupancy byte: Represents the distribution of points in space corresponding to multiple child nodes of a parent node. When using an octree to represent a point cloud, each of the eight child nodes of a parent node is assigned a value of 0 or 1 based on whether a point exists in the space corresponding to that child node. The values ​​of the eight child nodes are then combined to obtain the occupancy byte of the parent node, which is an 8-bit binary number. Since the occupancy byte is represented by 8 bits, there are 256 possible values ​​for the occupancy byte of a parent node, from 0 (00000000 in binary) to 255 (11111111 in binary).

[0064] Occupancy information: This represents the distribution of multiple points in space corresponding to the multiple child nodes of the current node. In this embodiment of the application, taking an octree as an example, the occupancy information of the current node to be encoded is represented by one occupancy byte, or by eight occupancy bytes.

[0065] Child node number: Also known as the octant of a child node, it represents the relative position of the child node in the space corresponding to the parent node, and also indicates which of the parent node's multiple children the child node is. Taking an octree as an example, the possible values ​​for the child node number are 1 to 8, or 0 to 7, representing eight different child nodes.

[0066] Hybrid compression algorithm: refers to a general term for algorithms that follow the architecture of traditional compression algorithms as a whole, but use deep learning methods to replace individual modules.

[0067] The following section introduces some application scenarios and related technical backgrounds involved in the embodiments of this application.

[0068] Point cloud data, as a form of data, is closely related to current industries such as virtual reality (VR) / augmented reality (AR) and autonomous driving. Point cloud is the foundation of most current Level 4 autonomous driving solutions and is indispensable in the positioning module. Simultaneously, in the VR / AR field, point cloud has important applications in digital humans and virtual object representation. With the development of VR / AR and autonomous driving industries, the application of point cloud data will become increasingly widespread.

[0069] Meanwhile, point clouds are also a form of data with a large volume. Taking the point clouds collected by LiDAR, commonly used in autonomous driving, as an example, a mainstream 64-line LiDAR can collect over 100GB of data per hour. Such a large data volume poses a significant challenge to data storage and transmission. The practical applications of point clouds in autonomous driving or VR / AR often rely heavily on data storage and transmission. During autonomous driving, the point clouds obtained by radar sensors need to be transmitted to the processor; in vehicle-to-everything (V2X) scenarios, there is a need to upload point cloud data to the cloud. In VR / AR scene reconstruction and virtual human scenarios, large amounts of reconstructed or generated point cloud data also need to be stored. To meet the transmission and storage requirements of point clouds, efficient compression algorithms are indispensable.

[0070] Point cloud compression algorithms fall into two main categories: traditional point cloud compression algorithms and artificial intelligence (AI)-based point cloud compression algorithms. Traditional point cloud compression algorithms are further divided into two types: one converts the point cloud into a two-dimensional signal and then compresses it using existing image compression algorithms, such as MPEG V-PCC; the other converts the point cloud into a tree structure and then performs entropy encoding, such as the Draco algorithm based on kd-trees and the MPEG G-PCC algorithm based on octrees. AI-based point cloud compression algorithms also include two types: one is based on a traditional algorithm framework but replaces some modules with AI algorithms, such as the OctSqueeze algorithm, which is based on the traditional octree compression framework but uses an AI algorithm to replace the context coding module. The other type is end-to-end AI compression algorithms, such as self-encoder / decoder algorithms based on PointNet++ features or three-dimensional convolutional neural networks (3DCNN). Overall, tree-based point cloud compression is a commonly used point cloud encoding and decoding method.

[0071] Before providing a detailed explanation of the embodiments of this application, the implementation environment of the embodiments of this application will be introduced first:

[0072] An implementation environment provided in this application embodiment includes a point cloud decoding system. Figure 2 This is a schematic block diagram of a point cloud decoding system provided for an embodiment of this application. The term "point cloud decoding" or "decoding" generally refers to point cloud encoding or point cloud decoding. See also... Figure 2 The point cloud decoding system includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 generates encoded point cloud data; therefore, it can also be called a point cloud encoding device. The destination device 20 decodes the encoded point cloud data generated by the source device 10; therefore, it can also be called a point cloud decoding device. The link 30 receives the encoded point cloud data generated by the source device 10 and transmits it to the destination device 20. The storage device 40 receives and stores the encoded point cloud data generated by the source device 10, allowing the destination device 20 to directly retrieve the encoded point cloud data from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded point cloud data generated by the source device 10, allowing the destination device 20 to stream or download the encoded point cloud data stored in the storage device 40.

[0073] Both source device 10 and destination device 20 may include one or more processors and memory coupled to the one or more processors. This memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other media that can be used to store desired program code in the form of computer-accessible instructions or data structures. For example, both source device 10 and destination device 20 may include desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handsets such as so-called "smartphones," televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0074] Link 30 may include one or more media or devices capable of transmitting encoded point cloud data from source device 10 to destination device 20. In one possible implementation, link 30 may include one or more communication media enabling source device 10 to directly transmit encoded point cloud data to destination device 20 in real time. In this embodiment, source device 10 may modulate the encoded point cloud data according to a communication standard, such as a wireless communication protocol, and transmit the modulated point cloud data to destination device 20. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, wide area network, or global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices facilitating communication from source device 10 to destination device 20, etc., which are not specifically limited in this embodiment.

[0075] In one possible implementation, storage device 40 can store the received encoded point cloud data sent by source device 10, and destination device 20 can directly retrieve the encoded point cloud data from storage device 40. Thus, storage device 40 can include any of a variety of distributed or locally accessible data storage media, such as hard disk drives, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded point cloud data.

[0076] In one possible implementation, storage device 40 may correspond to a file server or another intermediate storage device capable of storing the encoded point cloud data generated by source device 10. Destination device 20 may stream or download the point cloud data stored in storage device 40. The file server may be any type of server capable of storing and sending the encoded point cloud data to destination device 20. In one possible implementation, the file server may include a web server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, etc. Destination device 20 may acquire the encoded point cloud data via any standard data connection (including an Internet connection). Any standard data connection may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both suitable for acquiring the encoded point cloud data stored on a file server. The transmission of the encoded point cloud data from storage device 40 may be streaming, downloading, or a combination of both.

[0077] Figure 2 The point cloud decoding system shown is only one possible implementation, and the technology of this application can be applied to more than just... Figure 2 The source device 10 shown can encode point clouds, and the destination device 20 can decode the encoded point cloud data. It can also be applied to other devices that can encode point clouds and decode encoded point cloud data. This application embodiment does not specifically limit them.

[0078] exist Figure 2 In the point cloud decoding system shown, source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, output interface 140 may include a modem / demodulator and / or a transmitter, wherein the transmitter may also be referred to as a transmitter. Data source 120 may include a point cloud capture device (e.g., a camera, etc.), a point cloud archive containing previously captured point cloud data, a point cloud feed interface for receiving point cloud data from a point cloud content provider, and / or a computer graphics system for generating point cloud data, or a combination of these sources of point cloud data.

[0079] A data source can send point clouds to encoder 100, which can encode the received point clouds sent by data source 120 to obtain encoded point cloud data. The encoder can then send the encoded point cloud data to an output interface. In some embodiments, source device 10 directly sends the encoded point cloud data to destination device 20 via output interface 140. In other embodiments, the encoded point cloud data can also be stored on storage device 40 for later retrieval by destination device 20 for decoding and / or playback.

[0080] exist Figure 2 In some embodiments, the destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive encoded point cloud data via link 30 and / or from storage device 40, and then send it to the decoder 200, which may decode the received encoded point cloud data to obtain decoded point cloud data. The decoder may send the decoded point cloud data to the display device 220. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded point cloud data. The display device 220 may be any type of display device, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0081] although Figure 2 Not shown, but in some respects, encoder 100 and decoder 200 may be integrated with audio encoder and decoder respectively, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP), if applicable.

[0082] Encoder 100 and decoder 200 may each be any of the following circuits: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If this application is implemented in part in software, the device may store instructions for software in a suitable non-volatile computer-readable storage medium, and the instructions may be executed in hardware using one or more processors to implement the technology of this application. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors. Each of encoder 100 and decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (encoder-decoder) in the respective device.

[0083] In this application embodiment, encoder 100 may be generally referred to as an apparatus that “signals” or “sends” certain information to, for example, decoder 200. The terms “signals” or “sends” may generally refer to the transmission of syntax elements and / or other data for decoding compressed point cloud data. This transmission may occur in real-time or nearly real-time. Alternatively, this communication may occur after a period of time, for example, during encoding when syntax elements are stored in a computer-readable storage medium in a encoded bitstream, and the decoding apparatus may then retrieve the syntax elements at any time after they have been stored in this medium.

[0084] Figure 3 This is an illustrative diagram illustrating an example of a point cloud decoding system 40 comprising a point cloud encoder 20 and / or a point cloud decoder 30, based on an exemplary embodiment. The point cloud decoding system 40 may include an imaging device 41, a point cloud encoder 20, a point cloud decoder 30 (and / or a point cloud encoder / decoder implemented by processing circuitry 46), an antenna 42, one or more processors 43, one or more memory storage devices 44, and / or a display device 45.

[0085] like Figure 3 As shown, the imaging device 41, antenna 42, processing circuit 46, point cloud encoder 20, point cloud decoder 30, processor 43, memory storage 44, and / or display device 45 are capable of communicating with each other. In different instances, the point cloud decoding system 40 may contain only the point cloud encoder 20 or only the point cloud decoder 30.

[0086] In some instances, antenna 42 can be used to transmit or receive encoded bitstreams of point cloud data. Additionally, in some instances, display device 45 can be used to present the point cloud data. Processing circuitry 46 can include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. Point cloud decoding system 40 can also include an optional processor 43, which similarly can include ASIC logic, a graphics processor, a general-purpose processor, etc. Furthermore, memory storage 44 can be any type of memory, such as volatile memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.). In a non-limiting example, memory storage 44 can be implemented using high-speed cache memory. In other instances, processing circuitry 46 can include memory (e.g., cache, etc.) for implementing image buffers, etc.

[0087] In some instances, the point cloud encoder 20 implemented via logic circuitry may include (e.g., implemented via processing circuitry 46 or memory storage 44) an image buffer and (e.g., implemented via processing circuitry 46) a graphics processing unit. The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the point cloud encoder 20 implemented via processing circuitry 46 to implement a reference... Figure 3 The point cloud decoder 20 and / or any other encoder system or subsystem described herein, and various modules thereof. Logic circuitry may be used to perform the various operations discussed herein.

[0088] In some instances, the point cloud decoder 30 can be implemented in a similar manner via the processing circuitry 46 to implement the reference. Figure 3 The point cloud decoder 30 and / or any other decoder system or subsystem described herein may be represented by various modules. In some instances, the logic circuit-implemented point cloud decoder 30 may include an image buffer (implemented via processing circuitry 46 or memory storage 44) and a graphics processing unit (e.g., implemented via processing circuitry 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the point cloud decoder 30 implemented via processing circuitry 46.

[0089] In some instances, antenna 42 can be used to receive an encoded bitstream of point cloud data. As discussed herein, the encoded bitstream may contain data, indicators, index values, mode selection data, etc., related to the encoded point cloud frame, such as data related to encoding segmentation (e.g., transform coefficients or quantized transform coefficients, optional indicators, and / or data defining encoding segmentation). Point cloud decoding system 40 may also include a point cloud decoder 30 coupled to antenna 42 for decoding the encoded bitstream. Display device 45 is used to present the point cloud frame.

[0090] It should be understood that, for the example described with reference to point cloud encoder 20 in this application embodiment, point cloud decoder 30 can be used to perform the reverse process. Regarding signaling syntax elements, point cloud decoder 30 can be used to receive and parse such syntax elements, and accordingly decode the associated point cloud data. In some examples, point cloud encoder 20 can entropy-encode syntax elements into an encoded point cloud bitstream. In such instances, point cloud decoder 30 can parse such syntax elements and accordingly decode the associated point cloud data.

[0091] It should be noted that the encoding and decoding process described in the embodiments of this application exists in most point cloud codecs, such as the codecs corresponding to MPEG-2, MPEG-4, and AI-based end-to-end point cloud encoding.

[0092] Figure 4 This is a schematic diagram of a point cloud encoding / decoding algorithm framework provided in an embodiment of this application. See also... Figure 4 Point clouds (i.e., point cloud data) contain the three-dimensional coordinates of points in space (e.g., [{x1,y1,z1}, {x2,y2,z2}, ...]). Point cloud encoding and decoding are performed based on an N-ary tree. During encoding, the encoder (or encoder) constructs an N-ary tree based on the point cloud, for example, an octree. The encoder then compresses the point cloud by traversing the nodes in the N-ary tree. Specifically, for each node traversed, the encoder performs context prediction and entropy encoding (simultaneously) according to the encoding method provided in this application embodiment to encode the occupancy information of the traversed nodes into the bitstream, thus obtaining the compressed binary file (i.e., the compressed point cloud file). During decoding, the decoder (or decoder) decodes the occupancy information of each node one by one through context prediction and entropy decoding according to the decoding method provided in this application embodiment, thereby reconstructing the N-ary tree. Based on the reconstructed N-ary tree, the decompressed point cloud data (reconstructed point cloud data), i.e., the reconstructed point cloud, is obtained.

[0093] The following describes a point cloud encoding method provided in an embodiment of this application. It should be noted that, in conjunction with... Figure 2 or Figure 3 The point cloud decoding system shown, or Figure 4The point cloud encoding and decoding algorithm framework shown below, any of the point cloud encoding methods mentioned below can be executed by the encoder in the point cloud decoding system, or by the encoding end in the point cloud encoding and decoding algorithm framework.

[0094] Figure 5 This is a flowchart illustrating a point cloud encoding method provided in an embodiment of this application. This method is applied to an encoder. Please refer to... Figure 5 The method includes the following steps.

[0095] Step 501: Determine the N-ary tree, which is used to describe the space in which the point cloud is located.

[0096] Where N is an integer greater than 1.

[0097] In this embodiment, an N-ary tree is constructed based on the point cloud, and the point cloud is encoded and decoded using the constructed N-ary tree. As described above, N can be a constant or a variable. When N is a constant, the number of child nodes of each parent node in the N-ary tree is N. When N is a variable, some parent nodes in the N-ary tree have N child nodes, while the number of child nodes of other parent nodes is not equal to N (greater than N or less than N). For example, in the process of constructing the N-ary tree through recursive spatial partitioning, the value of N can be changed with the depth of the tree or the granularity of the minimum space, etc.

[0098] Figure 6 This is a schematic diagram of an N-ary tree structure provided in an embodiment of this application. Figure 6 In the N-ary tree shown, N is always equal to 8, and each node except the leaf nodes has eight child nodes. The space corresponding to a white-filled node contains no nodes, while the space corresponding to a black-filled node contains nodes. The space containing no nodes does not need further subdivision; that is, white-filled nodes have no child nodes. Figure 7 This is a schematic diagram of another N-ary tree structure provided in an embodiment of this application. Figure 6 The difference is that, in Figure 7 In the N-ary tree shown, N is a variable. For example... Figure 7 As shown, some nodes have eight child nodes, while others have four child nodes.

[0099] It's important to note that an N-ary tree is used to describe the space in which the point cloud resides, i.e., the overall space in which the point cloud exists. The root node of the N-ary tree describes the overall space (or large space) in which the point cloud resides, while the nodes in the N-ary tree other than the root node describe local spaces (or subspaces or smaller spaces). A local space is a part of the space in which the point cloud resides. In other words, each node in an N-ary tree corresponds to a space: the root node corresponds to the overall space, and each other node corresponds to a local space. Points in the point cloud exist in the spaces corresponding to nodes with child nodes and some leaf nodes; points in the point cloud do not exist in the spaces corresponding to nodes without child nodes (i.e., the other part of the leaf nodes).

[0100] Furthermore, the occupancy status of each node in the constructed N-ary tree is known, and this occupancy status indicates whether a point in the point cloud exists within the corresponding node. The occupancy information of each node in the N-ary tree, excluding the leaf nodes, is also known, and this occupancy information represents the distribution of multiple points in space corresponding to the multiple child nodes of the current node. In other words, both the node's occupancy status and occupancy information represent the true information of points within the space where the point cloud resides.

[0101] Step 502: Input the input data corresponding to the current node into the context prediction network to obtain the probability distribution of the current node. The input data includes the occupancy rate of each of the surrounding nodes in one or more of the current node's surrounding nodes. The occupancy rate represents the probability of a point existing in the space corresponding to the surrounding node.

[0102] Here, the current node is a node to be encoded in the N-ary tree. The probability distribution of the current node represents the probability of points existing in multiple spaces corresponding to the multiple child nodes of the current node. Each child node of the current node corresponds one-to-one with one of these multiple spaces. The probability distribution of the current node is essentially the probability distribution of the occupancy status of its multiple child nodes.

[0103] In this embodiment, point clouds are compressed using an entropy coding method based on N-ary trees and context prediction. The context prediction process is as follows: after determining the N-ary tree, the nodes in the N-ary tree are traversed. Each time a node is reached, its input data is input into the context prediction network to obtain the probability distribution of the current node. Optionally, the N-ary tree can be traversed using breadth-first search, depth-first search, or other methods. The input data for the current node includes the occupancy rate of one or more of its neighboring nodes, where the occupancy rate represents the probability of a point existing in the space corresponding to that neighboring node.

[0104] Before inputting the current node's input data into the context prediction network, it is necessary to determine one or more of the current node's neighboring nodes. It should be noted that this application does not limit which nodes in the N-ary tree are considered neighboring nodes of the current node; in other words, there are many methods for determining neighboring nodes, and different encoding / decoding algorithms may determine the same or different neighboring nodes.

[0105] In one implementation, one or more surrounding nodes of the current node include the current node's sibling nodes and the child nodes of the current node's sibling nodes. The sibling nodes of the current node share the same parent node as the current node. It is evident that the spatial distribution corresponding to the surrounding nodes is close to the spatial distribution corresponding to the current node. By utilizing the probability of points existing in the spatial distribution close to the current node, the probability distribution of the current node can be predicted, thus improving the accuracy of the probability distribution prediction.

[0106] In this embodiment, the one or more surrounding nodes include one or more first surrounding nodes. The parent node of each first surrounding node is an encoded node. The occupancy rate of each first surrounding node is a first value or a second value. The first value indicates that there are no points in the corresponding space, and the second value indicates that there are points in the corresponding space. For example, the first value is 0 and the second value is 1, or the first value is 0 and the second value is 10.

[0107] Optionally, the one or more surrounding nodes may further include one or more second surrounding nodes, the parent node of which is an uncoded node, and the occupancy rate of the second surrounding nodes is a predetermined value, which is greater than a first value and less than a second value. That is, in this embodiment, the probability distribution prediction can be based not only on the relevant information of some coded surrounding nodes but also on the relevant information of some uncoded surrounding nodes. This provides rich features for the probability distribution prediction and improves the prediction accuracy of the context prediction network to a certain extent.

[0108] It should be noted that if the parent node of a surrounding node is an encoded node, then the occupancy rate of that surrounding node is equivalent to its actual occupancy status, i.e., whether a point actually exists in the corresponding space. If the parent node of a surrounding node is an unencoded node, then the occupancy rate of that surrounding node is a predetermined value. Thus, during the subsequent decoding process, the occupancy status of the child nodes of decoded nodes is known, and this status can be directly used as the occupancy rate of the child nodes. For undecoded nodes, the occupancy status of the child nodes is unknown, and a predetermined value can be used as the occupancy rate of the child nodes. Furthermore, in the implementation where the parent node of the current node is an encoded node, the occupancy status of the current node is also known during the subsequent decoding process. Therefore, optionally, the surrounding nodes of the current node also include the current node itself.

[0109] Optionally, the predetermined value can be a preset value or a value determined by a certain method. For example, assuming the first value is 0 and the second value is 1, the predetermined value can be preset to 0.3, 0.5, or 0.8, etc. Alternatively, the occupancy rate of the second surrounding node can be determined based on the occupancy rate of its encoded surrounding nodes. For example, the average occupancy rate of the encoded surrounding nodes of the second surrounding node can be determined as the occupancy rate of the second surrounding node. Here, the encoded surrounding nodes of the second surrounding node include its encoded sibling nodes, or the encoded surrounding nodes of the second surrounding node include its encoded sibling nodes and their child nodes. Alternatively, a random number between the first and second values ​​can be generated as the occupancy rate of the second surrounding node. In addition, other methods can be used to determine the predetermined value, and this embodiment of the application does not limit this.

[0110] In one implementation, the current node's one or more surrounding nodes include one or more first surrounding nodes and one or more second surrounding nodes. Assuming these one or more surrounding nodes include the current node's sibling nodes and the child nodes of the current node's sibling nodes, then the one or more first surrounding nodes include the current node's sibling nodes and the child nodes of the current node's encoded sibling nodes, and the one or more second surrounding nodes include the child nodes of the current node's uncoded sibling nodes. For example, when traversing the nodes in the N-ary tree in breadth-first order, if the current node is neither the first nor the last child node of its parent node, some of the current node's sibling nodes are encoded nodes, and the rest are uncoded nodes. If the current node is the first child node of its parent node, all of the current node's sibling nodes are uncoded nodes. If the current node is the last child node of its parent node, all of the current node's sibling nodes are encoded nodes.

[0111] For example, assuming the N-ary tree constructed from the point cloud is an octree, and the breadth-first traversal of the octree is performed with a predetermined value of 0.5, the current node is the third child node of its parent node, and the surrounding nodes of the current node include the current node, its sibling nodes, and the child nodes of its sibling nodes. Then, the occupancy rate of the current node and its seven sibling nodes is equal to the corresponding occupancy status, which is either 0.0 or 1.0. The first two sibling nodes of the current node are encoded nodes, and the occupancy rate of their child nodes is equal to the corresponding occupancy status. The last five sibling nodes of the current node are uncoded nodes, and the occupancy rate of their child nodes is 0.5.

[0112] Optionally, in this embodiment, the occupancy rate of the one or more surrounding nodes is represented by floating-point numbers of one or more channels, and the one or more surrounding nodes correspond one-to-one with the one or more channels. In other words, the context prediction network has multiple input data channels, the occupancy rate of each surrounding node is represented by a floating-point number, and one occupancy rate is input to the context prediction network through one channel. For example, if the current node has a total of 7 surrounding nodes, the 7 occupancy rates (represented by floating-point numbers) of these 7 surrounding nodes are input to the context prediction network through 7 channels respectively.

[0113] For example, taking an octree, assuming the first value is 0.0, the second value is 1.0, and the predetermined value is 0.5, the surrounding nodes of the current node include the current node and its sibling nodes. Then, the occupancy rate of the current node and its seven sibling nodes is equivalent to their respective occupancy rates. For example, the occupancy rates of these eight nodes are 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, and 0.0, respectively. When encoding the current node, these eight floating-point numbers are input to the context prediction network through eight channels. These eight floating-point numbers can represent the occupancy information of the current node's parent node, that is, the distribution of multiple points in space corresponding to the multiple child nodes of the parent node.

[0114] As mentioned above, this scheme introduces floating-point numbers ranging from a first value (e.g., 0.0) to a second value (e.g., 1.0) to represent the probability of a point existing in the space corresponding to a node. This allows the occupancy rate to represent not only the presence or absence of a point, but also, for nodes whose occupancy status is not yet fully determined, a predetermined value can be used as the occupancy rate for input into the context prediction network. Essentially, this improves the prediction accuracy and overall compression ratio of the context prediction network by introducing more features.

[0115] Optionally, in this embodiment, the input data corresponding to the current node further includes one or more of the following: location information of the space corresponding to the current node, octant information of the current node, and level information of the current node. That is, the input data of the context prediction network includes not only features related to octant but also features of other modalities unrelated to octant. The location information of the space corresponding to the current node is the center coordinates or coordinates of any vertex of the space corresponding to the current node. The level information of the current node refers to the depth of the current node within the N-ary tree. The octant information of the current node refers to the octant information of the space corresponding to the current node within the space corresponding to the parent node of the current node. The octant information of the current node is used to indicate the relative position of the space corresponding to the current node within the space corresponding to the parent node. Taking an octant tree as an example, the parent node of the current node includes eight child nodes, and the space corresponding to the parent node includes eight subspaces, i.e., divided into eight octants. For example, the octants corresponding to these eight child nodes are 0 to 7, and these eight octants correspond one-to-one with these eight subspaces.

[0116] It should be noted that there can be multiple network frameworks and structures for context prediction networks, and this application does not limit them in the embodiments. One implementation method will be introduced below.

[0117] In one implementation, the context prediction network includes a feature fusion network and a probability distribution prediction network. The process of determining the probability distribution of the current node through the context prediction network is as follows: the input data corresponding to the current node is input into the feature fusion network to obtain fused features, and the fused features are input into the probability distribution prediction network to obtain the probability distribution of the current node.

[0118] Figure 8 This is a schematic diagram of the structure of a context prediction network provided in an embodiment of this application. See also... Figure 8 The context prediction network comprises a feature fusion network and a probability distribution prediction network. The input data corresponding to the current node is fed into the feature fusion network, which processes the input data to obtain fused features (also called latent features). These fused features are high-dimensional features. The fused features are then fed into the probability distribution prediction network to predict the probability distribution of the current node. The input data corresponding to the current node includes the occupancy rate of each of its one or more neighboring nodes. Assuming that the neighboring nodes of the current node include the child nodes of its parent node and the child nodes of its sibling nodes, the occupancy rates of the multiple channels included in the input data are, for example, 0.0, 0.0, 1.0, ..., 0.0, 1.0, ..., 0.5, 0.5. The input data corresponding to the current node also includes features unrelated to occupancy rates, such as the spatial location information L(x,y,z)∈R of the current node.3 The current node's octave O∈{0,…,7}, the current node's depth L∈{0,…,T}, where R 3 Let T represent a real number, and let T represent the depth of the N-ary tree.

[0119] The feature fusion network and probability distribution prediction network can have various network structures, and this application does not limit them in the embodiments. The following describes one implementation method.

[0120] In one implementation, the feature fusion network includes a multi-layer perceptron (MLP) network, and the probability distribution prediction network includes a long short-term memory (LSTM) network. The MLP network within the feature fusion network is used to fuse various input features. The LSTM network within the probability distribution prediction network is used to predict the probability distribution of the current node through recurrent convolutions. An LSTM network is a recurrent neural network capable of processing sequential data and recursively iterating along the sequence's progression.

[0121] Figure 9 This is a schematic diagram of the framework of an LSTM network provided in an embodiment of this application. See also... Figure 9 The LSTM network architecture consists of multiple cascaded LSTM networks for recursive iterative processing. Figure 9 The image shows two LSTM networks. The internal structure of the first subnetwork A is the same as that of the second subnetwork. The input of the LSTM network at time t includes x. t h t-1 and C t-1 The output includes h t and C t Among them, x t h represents the input fusion features. t Let C be a state vector, representing a certain prediction result. t This is an intermediate variable representing a summary of the information above, and it is updated in each iteration. C t The update (deletion or addition) is handled by the gate function (i.e. Figure 9 The gate function (σ) determines the value. C is updated via the gate function. t This allows the network to selectively utilize the preceding information. Here, h0 is the initialized vector, and tanh is a trigonometric function.

[0122] Figure 10 This is a schematic diagram of another context prediction network structure provided in an embodiment of this application. For example... Figure 10As shown, the feature fusion network consists of three cascaded MLP networks, and the probability distribution prediction network consists of three LSTM networks (structure as shown). Figure 9 (As shown) and an MLP network. The input and output dimensions of each network included in the context prediction network are as follows: Figure 10 As shown in the diagram, the input dimension *n* of the first MLP network is equal to the dimension of the input data of the current node. First, three cascaded MLP networks fuse the input data (containing various features) corresponding to the current node, combining these features into a 128-dimensional latent feature (i.e., the fused feature). Then, a three-layer cascaded LSTM network integrates the latent features and other information of the current node. Each LSTM layer converts the prediction result *h*... t Passed into the next layer of the network, based on h t And the updated C t Proceed to the next iteration. The prediction results are adjusted in the MLP layer after the last LSTM network, and finally a 256-dimensional probability distribution is output, which is the probability that points exist in multiple spaces corresponding to multiple child nodes of the current node.

[0123] To reiterate, the above... Figure 8 and Figure 10 The structure of the context prediction network shown is only for illustrative purposes to illustrate the embodiments of this application and is not intended to limit the embodiments of this application. In the embodiments of this application, the structure of the context prediction network can also be other network structures, such as convolutional neural networks, support vector machines, etc.

[0124] In this embodiment of the application, the context prediction network is a pre-trained network, and the training process of the context prediction network will be described below.

[0125] First, training data is acquired, including N-ary tree samples and the known probability distribution of each sample node (excluding leaf nodes) within these samples. Second, a context prediction network is trained based on the training data. During training, the context prediction network predicts the probability distribution of the current sample node in the N-ary tree samples, obtaining an estimated probability distribution. Based on the known and estimated probability distributions of the current sample node, a loss value is determined, and the network parameters are adjusted accordingly. When the loss value converges to a first threshold, or when the current training iterations reach a second threshold, the current context prediction network is considered the one obtained upon completion of training. Optionally, the training of the context prediction network can be performed online or offline. Online training allows the encoder to obtain a high-precision context prediction network in a timely manner.

[0126] Step 503: Based on the probability distribution of the current node, encode the occupancy information of the current node into the code stream. The occupancy information represents the distribution of multiple spatial points corresponding to multiple child nodes of the current node.

[0127] In this embodiment, after obtaining the probability distribution of the current node, the occupancy information of the current node is encoded into the bitstream based on the probability distribution. The occupancy information represents the distribution of multiple spatial points corresponding to multiple child nodes of the current node. In this embodiment, taking an octree as an example, the occupancy information of the current node to be encoded is represented by one occupancy byte, or by eight occupancy bytes. It should be noted that the distribution represented by the occupancy information is real information.

[0128] For example, after obtaining the probability distribution of the current node, the probability distribution and occupancy information of the current node are fed into an entropy encoder for entropy encoding to obtain the bitstream of the current node. The entropy encoder can be an arithmetic encoder or other encoders.

[0129] It should be noted that the current node is a non-leaf node in the N-ary tree to be encoded; that is, encoding only the non-leaf nodes in the N-ary tree is required. For leaf nodes, encoding the occupancy information of the leaf node's parent node into the bitstream essentially encodes the leaf node's occupancy status into the bitstream. In the subsequent decoding process, the entire N-ary tree can be reconstructed based on the occupancy information of other nodes besides the leaf nodes, and thus the point cloud can be reconstructed. In other words, when the occupancy information of multiple nodes in the N-ary tree, excluding the leaf nodes, is encoded into the bitstream, the point cloud encoding based on the N-ary tree is complete.

[0130] As can be seen from the foregoing, point clouds may also contain one or more of the following information: color, classification value, intensity value, and time of points in space. The embodiments of this application do not limit the encoding method for incorporating this information into the bitstream.

[0131] In summary, in this embodiment, an N-ary tree is used to describe the space where the point cloud is located. During the encoding and decoding process of the point cloud, instead of converting the occupancy information represented by eight bits into decimal numbers and inputting it into the context prediction network, the occupancy rates of each neighboring node of the current node are input into the context prediction network to predict the probability distribution of the current node. Here, the occupancy rate represents the probability of a point existing in the space corresponding to the corresponding neighboring node. It is evident that this scheme, by inputting the occupancy rates of each neighboring node into the context prediction network, prevents the context prediction network from mistakenly believing that the spaces corresponding to each neighboring node have different weights, thus accurately understanding the occupancy rate of each neighboring node. This improves the prediction accuracy of the context prediction network, thereby enhancing the encoding and decoding performance. In simple terms, this scheme includes a data preprocessing process, such as representing the features related to occupancy or occupancy rate in the network's input data as multi-channel values ​​(e.g., floating-point numbers), enabling the network to correctly understand the input data.

[0132] Figure 11 This is a flowchart of a point cloud decoding method provided in an embodiment of this application, which is applied to a decoder. It should be noted that... Figure 11 The decoding method shown is the same as Figure 5 The encoding methods shown are compatible. Please refer to [the documentation / reference]. Figure 11 The method includes the following steps.

[0133] Step 1101: Determine one or more neighboring nodes of the current node in the N-ary tree to be reconstructed. This N-ary tree is used to describe the space in which the point cloud is located.

[0134] Here, the current node is a node to be decoded in an N-ary tree, where N is an integer greater than 1. For an introduction to N-ary trees, please refer to the relevant content in the example; it will not be repeated here.

[0135] In the encoding process of this application embodiment, point clouds are compressed based on N-ary trees. Correspondingly, in the decoding process, the N-ary tree needs to be reconstructed based on the bitstream to reconstruct the point cloud. Specifically, the decoder sequentially decodes each node in the N-ary tree to be reconstructed, excluding the leaf nodes, based on the bitstream. For the current node to be decoded, one or more surrounding nodes of the current node need to be determined first. It should be noted that this application embodiment does not limit which nodes in the N-ary tree to be reconstructed are the surrounding nodes of the current node. In other words, there are many methods for determining surrounding nodes, and the surrounding nodes determined by different encoding and decoding algorithms may be the same or different. However, the surrounding nodes of the current node during the encoding process are consistent with the surrounding nodes of the current node during the decoding process.

[0136] In one implementation, one or more surrounding nodes of the current node include the current node's sibling nodes and the child nodes of the current node's sibling nodes. The sibling nodes of the current node share the same parent node as the current node. Optionally, the current node also includes the current node itself. It is evident that the spaces corresponding to the surrounding nodes are geographically close to the space corresponding to the current node. By utilizing the probability of points existing in spaces geographically close to the current node, the probability distribution of the current node can be predicted, thus improving the accuracy of the probability distribution prediction.

[0137] Step 1102: Input the input data of the current node into the context prediction network to obtain the probability distribution of the current node. The input data includes the occupancy rate of each of the one or more surrounding nodes. The occupancy rate represents the probability of a point existing in the space corresponding to the surrounding node.

[0138] The probability distribution of the current node represents the probability that points exist in multiple spaces corresponding to the multiple child nodes of the current node.

[0139] In this embodiment, after determining one or more neighboring nodes of the current node, the input data of the current node is fed into a context prediction network to obtain the probability distribution of the current node. The input data includes the occupancy rate of each of the one or more neighboring nodes. That is, similar to the process of determining the probability distribution of the current node through context prediction during encoding, the probability distribution of the current node is also determined through context prediction during decoding.

[0140] The one or more surrounding nodes include one or more first surrounding nodes, whose parent nodes are decoded nodes. The occupancy rate of the first surrounding nodes is either a first value or a second value, where the first value indicates that no point exists in the corresponding space, and the second value indicates that a point exists in the corresponding space. Optionally, the one or more surrounding nodes also include one or more second surrounding nodes, whose parent nodes are undecoded nodes. The occupancy rate of the second surrounding nodes is a predetermined value, which is greater than the first value and less than the second value. Optionally, the predetermined value is a preset value or a value determined by a certain method. The specific implementation method for determining the predetermined value is described in the relevant introduction during the encoding process and will not be repeated here.

[0141] Optionally, in this embodiment, the occupancy rate of the one or more surrounding nodes is represented by floating-point numbers of one or more channels, and the one or more surrounding nodes correspond one-to-one with the one or more channels. In other words, the context prediction network has multiple input data channels, the occupancy rate of each surrounding node is represented by a floating-point number, and one occupancy rate is input to the context prediction network through one channel. For example, if the current node has a total of 8 surrounding nodes, the 8 occupancy rates (represented by floating-point numbers) of these 8 surrounding nodes are input to the context prediction network through 8 channels respectively.

[0142] Optionally, in this embodiment, the input data corresponding to the current node further includes one or more of the following: spatial location information of the current node, quadrant of the current node, and depth of the current node. That is, the input data of the context prediction network includes not only features related to occupancy but also features of other modalities unrelated to occupancy. The descriptions of the spatial location information, quadrant of the current node, and depth of the current node can be found in the aforementioned encoding process and will not be repeated here. It should be noted that the input data corresponding to the current node during the encoding process is the same as the input data corresponding to the current node during the decoding process.

[0143] In this embodiment, the context prediction network in the decoding process is the same as the context prediction network in the encoding process. Exemplarily, the context prediction network includes a feature fusion network and a probability distribution prediction network. The process of determining the probability distribution of the current node through the context prediction network is as follows: the input data corresponding to the current node is input into the feature fusion network to obtain fused features, and the fused features are input into the probability distribution prediction network to obtain the probability distribution of the current node. For a detailed description of the feature fusion network and the probability distribution prediction network, please refer to the relevant content in the encoding method embodiment. Furthermore, the training process of the context prediction network can also refer to the foregoing embodiments, and will not be repeated here.

[0144] Step 1103: Based on the probability distribution of the current node, parse the occupancy information of the current node from the bitstream. This occupancy information represents the distribution of multiple spatial points corresponding to multiple child nodes of the current node.

[0145] In this embodiment, after determining the probability distribution of the current node, the occupancy information of the current node is parsed from the bitstream based on the probability distribution. This occupancy information represents the distribution of multiple spatial points corresponding to multiple child nodes of the current node. In this embodiment, taking an octree as an example, the decoded occupancy information of the current node is represented by one occupancy byte, or by eight occupancy bytes. It should be noted that the distribution represented by the occupancy information is real information.

[0146] For example, after determining the probability distribution of the current node, the probability distribution and bitstream of the current node are sent to an entropy decoder for entropy decoding to obtain the occupancy information of the current node. The entropy decoder can be an arithmetic decoder or other decoders.

[0147] Step 1104: Reconstruct the N-ary tree based on the occupancy information of multiple nodes, including the current node, to reconstruct the point cloud.

[0148] In this embodiment, the current node is a non-leaf node to be decoded in the N-ary tree to be reconstructed, meaning that the occupancy information of the non-leaf nodes in the N-ary tree can be parsed from the bitstream. For leaf nodes, the occupancy information of their parent nodes already contains the occupancy status of the leaf nodes. After parsing the occupancy information of each non-leaf node, the N-ary tree can be reconstructed, and then the point cloud can be reconstructed based on the N-ary tree. That is, when the occupancy information of the last non-leaf node in the N-ary tree is parsed from the bitstream, the N-ary tree is reconstructed, and then the point cloud is reconstructed. In other words, the N-ary tree is reconstructed based on the occupancy information of multiple nodes, including the current node, to reconstruct the point cloud; these multiple nodes are the multiple non-leaf nodes in the N-ary tree. It should be noted that the structure of the N-ary tree reconstructed during the decoding process is the same as the structure of the N-ary tree constructed during the encoding process.

[0149] As can be seen from the foregoing, point clouds may also contain one or more of the following information: color, classification value, intensity value, and time of points in space. The embodiments of this application do not limit the decoding method for parsing the point cloud information from the bitstream.

[0150] In summary, in this embodiment, an N-ary tree is used to describe the space where the point cloud is located. During the encoding and decoding process of the point cloud, instead of converting the occupancy information represented by eight bits into decimal numbers and inputting it into the context prediction network, the occupancy rates of each neighboring node of the current node are input into the context prediction network to predict the probability distribution of the current node. Here, the occupancy rate represents the probability of a point existing in the space corresponding to the corresponding neighboring node. It is evident that this scheme, by inputting the occupancy rates of each neighboring node into the context prediction network, prevents the context prediction network from mistakenly believing that the spaces corresponding to each neighboring node have different weights, thus accurately understanding the occupancy rate of each neighboring node. This improves the prediction accuracy of the context prediction network, thereby enhancing the encoding and decoding performance.

[0151] Figure 12 This is a schematic diagram of the structure of a point cloud encoding device 1200 provided in an embodiment of this application. The encoding device 1200 can be implemented as part or all of an encoder by software, hardware, or a combination of both. The encoder can be... Figures 2 to 4 Any of the encoders shown. See also Figure 12The encoding device 1200 includes: a determination module 1201, a prediction module 1202, and an encoding module 1203.

[0152] The module 1201 is used to determine an N-ary tree, which is used to describe the space where the point cloud is located, where N is an integer greater than 1;

[0153] The prediction module 1202 is used to input the input data corresponding to the current node into the context prediction network to obtain the probability distribution of the current node. The input data includes the occupancy rate of each of the one or more surrounding nodes of the current node. The occupancy rate represents the probability of a point existing in the space corresponding to the corresponding surrounding node. The current node is a node to be encoded in an N-ary tree. The probability distribution of the current node represents the probability of a point existing in the multiple spaces corresponding to the multiple child nodes of the current node.

[0154] The encoding module 1203 is used to encode the occupancy information of the current node into the code stream based on the probability distribution of the current node. The occupancy information represents the distribution of multiple spatial points corresponding to multiple child nodes of the current node.

[0155] Optionally, the one or more surrounding nodes include one or more first surrounding nodes, the parent node of the first surrounding node is an encoded node, and the occupancy rate of the first surrounding node is a first value or a second value, where the first value indicates that there are no points in the corresponding space, and the second value indicates that there are points in the corresponding space.

[0156] One or more peripheral nodes also include one or more second peripheral nodes. The parent node of the second peripheral node is an uncoded node. The occupancy rate of the second peripheral node is a predetermined value, which is greater than the first value and less than the second value.

[0157] Optionally, the context prediction network includes a feature fusion network and a probability distribution prediction network;

[0158] Prediction module 1202 includes:

[0159] The fusion submodule is used to input the input data corresponding to the current node into the feature fusion network to obtain fused features;

[0160] The prediction submodule is used to input the fused features into the probability distribution prediction network to obtain the probability distribution of the current node.

[0161] Optionally, the occupancy of the one or more peripheral nodes is represented by floating-point numbers of one or more channels, and the one or more peripheral nodes correspond one-to-one with the one or more channels.

[0162] In this embodiment, an N-ary tree is used to describe the space where the point cloud is located. During the encoding and decoding process of the point cloud, instead of converting the occupancy information represented by eight bits into decimal numbers and inputting it into the context prediction network, the occupancy rates of each neighboring node of the current node are input into the context prediction network to predict the probability distribution of the current node. Here, the occupancy rate represents the probability of a point existing in the space corresponding to the corresponding neighboring node. It is evident that this scheme, by inputting the occupancy rates of each neighboring node into the context prediction network, prevents the context prediction network from mistakenly believing that the spaces corresponding to each neighboring node have different weights, thus accurately understanding the occupancy rates of each neighboring node. This improves the prediction accuracy of the context prediction network, thereby enhancing the encoding and decoding performance.

[0163] It should be noted that the point cloud encoding device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the point cloud encoding device and the point cloud encoding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0164] Figure 13 This is a schematic diagram of the structure of a point cloud decoding device 1300 provided in an embodiment of this application. The decoding device 1300 can be implemented as part or all of a decoder by software, hardware, or a combination of both. The decoder can be... Figures 2 to 4 Any of the decoders shown. See also Figure 13 The decoding device 1300 includes: a determination module 1301, a prediction module 1302, a parsing module 1303, and a reconstruction module 1304.

[0165] The determination module 1301 is used to determine one or more surrounding nodes of the current node in the N-ary tree to be reconstructed. The N-ary tree is used to describe the space where the point cloud is located. The current node is a node to be decoded in the N-ary tree, and N is an integer greater than 1.

[0166] The prediction module 1302 is used to input the input data of the current node into the context prediction network to obtain the probability distribution of the current node. The input data includes the occupancy rate of each of the surrounding nodes in one or more surrounding nodes. The occupancy rate represents the probability of a point existing in the space corresponding to the corresponding surrounding node. The probability distribution of the current node represents the probability of a point existing in the multiple spaces corresponding to the multiple child nodes of the current node.

[0167] The parsing module 1303 is used to parse the occupancy information of the current node from the code stream based on the probability distribution of the current node. The occupancy information represents the distribution of multiple spatial points corresponding to multiple child nodes of the current node.

[0168] Reconstruction module 1304 is used to reconstruct an N-ary tree based on the occupancy information of multiple nodes, including the current node, in order to reconstruct the point cloud.

[0169] Optionally, the one or more surrounding nodes include one or more first surrounding nodes, the parent node of the first surrounding node is a decoded node, and the occupancy rate of the first surrounding node is a first value or a second value, where the first value indicates that there is no point in the corresponding space, and the second value indicates that there is a point in the corresponding space.

[0170] The one or more surrounding nodes also include one or more second surrounding nodes, the parent node of the second surrounding node is an undecoded node, and the occupancy rate of the second surrounding node is a predetermined value, which is greater than the first value and less than the second value.

[0171] Optionally, the context prediction network includes a feature fusion network and a probability distribution prediction network;

[0172] Prediction module 1302 includes:

[0173] The input data corresponding to the current node is input into the feature fusion network to obtain the fused features;

[0174] The fused features are input into the probability distribution prediction network to obtain the probability distribution of the current node.

[0175] Optionally, the occupancy of the one or more peripheral nodes is represented by floating-point numbers of one or more channels, and the one or more peripheral nodes correspond one-to-one with the one or more channels.

[0176] In this embodiment, an N-ary tree is used to describe the space where the point cloud is located. During the encoding and decoding process of the point cloud, instead of converting the occupancy information represented by eight bits into decimal numbers and inputting it into the context prediction network, the occupancy rates of each neighboring node of the current node are input into the context prediction network to predict the probability distribution of the current node. Here, the occupancy rate represents the probability of a point existing in the space corresponding to the corresponding neighboring node. It is evident that this scheme, by inputting the occupancy rates of each neighboring node into the context prediction network, prevents the context prediction network from mistakenly believing that the spaces corresponding to each neighboring node have different weights, thus accurately understanding the occupancy rates of each neighboring node. This improves the prediction accuracy of the context prediction network, thereby enhancing the encoding and decoding performance.

[0177] It should be noted that the point cloud decoding device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the point cloud decoding device and the point cloud decoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0178] Figure 14 This is a schematic block diagram illustrating one implementation of the decoding apparatus 1400 used in embodiments of this application. The acquisition apparatus 1400 may include a processor 1401, a memory 1402, and a bus system 1403. The processor 1401 and the memory 1402 are connected via the bus system 1403. The memory 1402 stores instructions, and the processor 1401 executes the instructions stored in the memory 1402 to perform various point cloud encoding or decoding methods described in embodiments of this application, particularly methods for filtering current image blocks based on their block size. To avoid repetition, further details are omitted here.

[0179] In this embodiment, the processor 1401 may be a central processing unit (CPU), or it may be other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0180] The memory 1402 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as memory 1402. Memory 1402 may include code and data 14021 accessed by processor 1401 using bus 1403. Memory 1402 may further include an operating system 14023 and an application program 14022, which includes at least one program that allows processor 1401 to execute the point cloud encoding or decoding methods described in the embodiments of this application (especially the point cloud encoding and decoding methods described in the embodiments of this application). For example, application program 14022 may include applications 1 to N, which further include a point cloud encoding or decoding application (hereinafter referred to as a point cloud decoding application) that executes the point cloud encoding or decoding methods described in the embodiments of this application.

[0181] In addition to the data bus, the bus system 1403 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 1403 in the figure.

[0182] Optionally, the encoding device 1400 may also include one or more output devices, such as a display 1404. In one example, the display 1404 may be a haptic display that combines a display with a haptic unit capable of operatively sensing touch input. The display 1404 may be connected to the processor 1401 via a bus 1403.

[0183] It should be noted that the decoding device 1400 can execute the point cloud encoding method in the embodiments of this application, and can also execute the point cloud decoding method in the embodiments of this application.

[0184] Those skilled in the art will appreciate that the functionality described in conjunction with the various illustrative logic blocks, modules, and algorithmic steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality described by the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may comprise a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium facilitating the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this way, the computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. The data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in the embodiments of this application. A computer program product may comprise a computer-readable medium.

[0185] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. Furthermore, any connection is properly referred to as computer-readable media. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are specifically directed to non-temporary tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, DVDs, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. The combination of the above items should also be included in the scope of computer-readable media.

[0186] Instructions can be executed by one or more processors, such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described in the various illustrative logic blocks, modules, and steps described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Moreover, the techniques can be fully implemented in one or more circuit or logic elements. In one example, the various illustrative logic blocks, units, and modules in encoder 100 and decoder 200 can be understood as corresponding circuit devices or logic elements.

[0187] The technology of this application can be implemented in a wide variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). The various components, modules, or units described in the embodiments of this application are intended to emphasize functional aspects of the apparatus for performing the disclosed technology, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units can be combined with suitable software and / or firmware within a codec hardware unit, or provided via an interoperable hardware unit (comprising one or more processors as described above).

[0188] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0189] It should be understood that "at least one" as mentioned herein refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., are not necessarily different.

[0190] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A point cloud encoding method, characterized in that, The method includes: Determine an N-ary tree, which is used to describe the space in which the point cloud is located, where N is an integer greater than 1; The input data corresponding to the current node is input into the context prediction network to obtain the probability distribution of the current node. The input data includes the occupancy rate of each of the one or more surrounding nodes of the current node. The occupancy rate represents the probability of a point existing in the space corresponding to the corresponding surrounding node. The current node is a node to be encoded in the N-ary tree. The probability distribution of the current node represents the probability of a point existing in the multiple spaces corresponding to the multiple child nodes of the current node. Based on the probability distribution of the current node, the occupancy information of the current node is encoded into the code stream. The occupancy information represents the distribution of multiple spatial points corresponding to multiple child nodes of the current node. The one or more surrounding nodes include one or more first surrounding nodes and one or more second surrounding nodes. The parent node of the first surrounding node is an encoded node. The occupancy rate of the first surrounding node is a first value or a second value. The first value indicates that there is no point in the corresponding space, and the second value indicates that there is a point in the corresponding space. The parent node of the second surrounding node is an uncoded node. The occupancy rate of the second surrounding node is a predetermined value. The predetermined value is greater than the first value and less than the second value. The predetermined value is the average occupancy rate of the encoded surrounding nodes of the second surrounding node.

2. The method as described in claim 1, characterized in that, The context prediction network includes a feature fusion network and a probability distribution prediction network; The step of inputting the input data corresponding to the current node into the context prediction network to obtain the probability distribution of the current node includes: The input data corresponding to the current node is input into the feature fusion network to obtain fused features; The fused features are input into the probability distribution prediction network to obtain the probability distribution of the current node.

3. The method as described in claim 1 or 2, characterized in that, The occupancy rate of the one or more peripheral nodes is represented by a floating-point number of one or more channels, and the one or more peripheral nodes correspond one-to-one with the one or more channels.

4. A method for decoding point clouds, characterized in that, The method includes: Determine one or more neighboring nodes of the current node in the N-ary tree to be reconstructed. The N-ary tree is used to describe the space where the point cloud is located. The current node is a node to be decoded in the N-ary tree. N is an integer greater than 1. The input data of the current node is input into the context prediction network to obtain the probability distribution of the current node. The input data includes the occupancy rate of each of the one or more surrounding nodes. The occupancy rate represents the probability of a point existing in the space corresponding to the corresponding surrounding node. The probability distribution of the current node represents the probability of a point existing in the multiple spaces corresponding to the multiple child nodes of the current node. Based on the probability distribution of the current node, the occupancy information of the current node is parsed from the bitstream. The occupancy information represents the distribution of multiple spatial points corresponding to multiple child nodes of the current node. The N-ary tree is reconstructed based on the occupancy information of multiple nodes, including the current node, to reconstruct the point cloud; The one or more surrounding nodes include one or more first surrounding nodes and one or more second surrounding nodes. The parent node of the first surrounding node is a decoded node. The occupancy rate of the first surrounding node is a first value or a second value. The first value indicates that there is no point in the corresponding space, and the second value indicates that there is a point in the corresponding space. The parent node of the second surrounding node is an undecoded node. The occupancy rate of the second surrounding node is a predetermined value. The predetermined value is greater than the first value and less than the second value. The predetermined value is the average occupancy rate of the decoded surrounding nodes of the second surrounding node.

5. The method as described in claim 4, characterized in that, The context prediction network includes a feature fusion network and a probability distribution prediction network; The step of inputting the input data of the current node into the context prediction network to obtain the probability distribution of the current node includes: The input data corresponding to the current node is input into the feature fusion network to obtain fused features; The fused features are input into the probability distribution prediction network to obtain the probability distribution of the current node.

6. The method as described in claim 4 or 5, characterized in that, The occupancy rate of the one or more peripheral nodes is represented by a floating-point number of one or more channels, and the one or more peripheral nodes correspond one-to-one with the one or more channels.

7. A point cloud encoding device, characterized in that, The device includes: One or more processors; A non-transitory computer-readable storage medium coupled to the processor and storing program instructions executable by the processor, wherein, when executed by the processor, the program instructions cause the encoding device to perform the method as described in any one of claims 1-3.

8. A point cloud decoding device, characterized in that, The device includes: One or more processors; A non-transitory computer-readable storage medium coupled to the processor and storing program instructions executable by the processor, wherein, when executed by the processor, the program instructions cause the decoding device to perform the method as described in any one of claims 4-6.

9. An encoder, characterized in that, Includes processing circuitry for performing the method as described in any one of claims 1-3.

10. A decoder, characterized in that, Includes processing circuitry for performing the method as described in any one of claims 4-6.

11. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

12. A computer program product, characterized in that, The computer program product stores computer instructions, which, when executed by a processor, implement the steps of the method described in any one of claims 1-6.

Images