Point cloud data processing method, device, equipment and medium

Abstract

The present application discloses a method, apparatus, device, and medium for processing point cloud data. The method comprises: obtaining a compressed bitstream of point cloud data; when the node to be decoded is the first node in a first geometric structure of the point cloud data, determining decoded neighbor nodes that have a spatially adjacent relationship with the node to be decoded; when the decoded neighbor node is the second node in a second geometric structure different from the first geometric structure, determining target occupancy information of the decoded neighbor node; decoding node geometry information of the node to be decoded in the compressed bitstream based on the target occupancy information; the node geometry information includes either partition tree occupancy information or prediction tree residual information associated with the structural type of the first geometric structure. By using the present application, the decoding efficiency of the compressed bitstream can be improved when geometric encoding is performed based on a hybrid tree structure.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a point cloud data processing method, apparatus, device, and medium. Background Art

[0002] In the existing point cloud coding architecture, a block structure coding method can be used to implement geometric coding of point cloud data, that is, the macroblock (Largest Coding Unit, LCU) is used as the basic coding unit, and each macroblock can be encoded using a specific geometric structure (or geometric coding method, such as partition tree, prediction tree, etc.).

[0003] In practice, the inventors discovered that the compressed bitstream of point cloud data obtained using a block-structured encoding method may contain both bitstreams containing geometric information encoded based on partitioning trees and bitstreams containing geometric information encoded based on prediction trees. In this case, if the node to be decoded is a partitioning tree node, only the placeholders of neighboring nodes that are also partitioning tree nodes are considered to decode the placeholder code of the node itself. Similarly, if the node to be decoded is a prediction tree node, only the node construction within the node is considered. This failure to fully consider the relevant information of neighboring nodes in different types of geometric structures results in low decoding efficiency of the compressed bitstream. Summary of the Invention

[0004] The embodiments of the present application provide a point cloud data processing method, apparatus, device, and medium, which can improve the decoding efficiency of compressed code streams when geometric encoding is performed based on a hybrid tree structure.

[0005] An embodiment of the present application provides a method for processing point cloud data, including:

[0006] Get the compressed code stream of point cloud data;

[0007] When the node to be decoded is the first node in the first geometric structure of the point cloud data, determining a decoded neighbor node having a spatial adjacent relationship with the node to be decoded;

[0008] When the decoded neighbor node is a second node in a second geometric structure different from the first geometric structure, determining target placeholder information of the decoded neighbor node;

[0009] The node geometry information of the node to be decoded in the compressed code stream is decoded based on the target occupancy information; the node geometry information includes any one of the partition tree occupancy information and the prediction tree residual information associated with the structure type of the first geometric structure.

[0010] An embodiment of the present application provides a method for processing point cloud data, including:

[0011] When the node to be encoded is the first node in the first geometric structure of the point cloud data, determining an encoded neighboring node having a spatial adjacent relationship with the node to be encoded;

[0012] When the encoded neighbor node is a second node in a second geometric structure different from the first geometric structure, determining target placeholder information of the encoded neighbor node;

[0013] The node geometry information of the node to be encoded is encoded based on the target occupancy information to obtain a compressed code stream of the point cloud data; the node geometry information includes any one of the partitioning tree occupancy information and the prediction tree residual information associated with the structure type of the first geometric structure.

[0014] An embodiment of the present application provides a point cloud data processing device, including:

[0015] The code stream acquisition module is used to obtain the compressed code stream of point cloud data;

[0016] a neighbor determination module, configured to determine, when the node to be decoded is the first node in the first geometric structure of the point cloud data, decoded neighbor nodes that have a spatially adjacent relationship with the node to be decoded;

[0017] an occupancy determination module, configured to determine target occupancy information of the decoded neighbor node when the decoded neighbor node is a second node in a second geometric structure different from the first geometric structure;

[0018] A geometry decoding module is used to decode the node geometry information of the node to be decoded in the compressed code stream based on the target occupancy information; the node geometry information includes any one of the partitioning tree occupancy information and the prediction tree residual information associated with the structure type of the first geometric structure.

[0019] Wherein, when the structure type of the first geometric structure is a partition tree type, the first node as the node to be decoded is a node in the partition tree corresponding to the partition tree type, and when the structure type of the second geometric structure is a prediction tree type, the second node as the decoded neighbor node is a node in the prediction tree corresponding to the prediction tree type;

[0020] The placeholder determination module includes:

[0021] A first strategy acquisition unit, configured to acquire a first hybrid decoding optimization strategy associated with the first node based on first spatial decoding data information of the second node located in the prediction tree;

[0022] The first placeholder determination unit is configured to determine target placeholder information of decoded neighbor nodes based on a first hybrid decoding optimization strategy.

[0023] The first placeholder determination unit includes:

[0024] A parent node search subunit is configured to determine the partition layer where the parent node of the node to be decoded is located as the parent node layer, and based on the first hybrid decoding optimization strategy, search for the neighboring parent nodes of the parent node in the parent node layer; if the neighboring parent nodes found exist in the prediction tree, then the neighboring parent nodes in the prediction tree are used as the decoded neighbor nodes;

[0025] The first placeholder determination subunit is configured to set the value of the placeholder code of the decoded neighbor node to a first code value, and determine target placeholder information of the decoded neighbor node based on the placeholder code of the decoded neighbor node having the first code value.

[0026] The first placeholder determination unit includes:

[0027] A sub-node search sub-unit is configured to use the division layer where the node to be decoded is located as a sub-node layer, and based on the first hybrid decoding optimization strategy, search for neighboring sub-nodes of the node to be decoded in the sub-node layer;

[0028] The second placeholder determination sub-unit is used to determine the target placeholder information of the decoded neighbor node based on the placeholder code of the target neighbor child node, if there is a parent node located in the prediction tree among the parent nodes of the found neighbor child node. When the parent node located in the prediction tree is used as the decoded neighbor node, the neighbor child node in the decoded neighbor node is used as the target neighbor child node.

[0029] The placeholder code of the target neighbor subnode is a default code value; the default code value is the first code value or the second code value.

[0030] The placeholder code of the target neighbor subnode is determined by the partition tree type adopted by the coding unit to which the decoded neighbor node belongs.

[0031] The placeholder code of the target neighbor child node is determined by the real placeholder code obtained after partitioning the decoded neighbor node based on the target partition tree type; the target partition tree type includes one of an octree partition type, a quadtree partition type, and a binary tree partition type.

[0032] The placeholder code of the target neighbor child node is determined by the decoding information associated with the decoded neighbor node.

[0033] The decoding information includes point cloud statistical information of the decoded neighbor node; the placeholder code of the target neighbor child node is determined by the point cloud statistical information and a statistical threshold associated with the point cloud statistical information; the point cloud statistical information includes one of the number of point cloud points in the decoded neighbor node and the point cloud density in the decoded neighbor node.

[0034] Among them, the decoding information includes the reconstruction point of the prediction tree where the decoded neighbor node is located; when the spatial distance between the reconstruction point of the target neighbor sub-node and the node to be decoded is the target spatial distance, the placeholder code of the target neighbor sub-node is determined by the first code value obtained when the target spatial distance is less than the distance threshold.

[0035] The decoding information includes a placeholder code of a reference decoded neighbor node of the decoded neighbor node; and the placeholder code of the target neighbor child node is determined by the placeholder code of the reference decoded neighbor node.

[0036] The decoding information includes the child node placeholder code of the partition tree neighbor child node that has the same parent node as the node to be decoded; the partition tree neighbor child node is the decoded neighbor child node in the partition tree; the placeholder code of the target neighbor child node is determined by the child node placeholder code.

[0037] In which, when the child node placeholder code indicates that the partition tree neighbor child node located at the child node layer is in an occupied state, the target neighbor child node located at the child node layer is in an occupied state, and the placeholder code of the target neighbor child node is determined by the first code value obtained when the target neighbor child node is in an occupied state.

[0038] The partition tree neighbor subnodes include the target partition tree neighbor subnode having the smallest distance to the decoded neighbor node; and the placeholder code of the target neighbor subnode is determined by the child node placeholder code of the target partition tree neighbor subnode.

[0039] Among them, the placeholder code of the target neighbor child node is determined by the association relationship between the division depth of the node to be decoded and the depth threshold; the association relationship includes one of the first association relationship when the division depth is less than the depth threshold and the second association relationship when the division depth is greater than the depth threshold.

[0040] Among them, the neighbor node types of the target neighbor sub-node include one or more of coplanar neighbor sub-nodes that are coplanar with the node to be decoded, collinear neighbor sub-nodes that are colinear with the node to be decoded, co-point neighbor sub-nodes that have a common point with the node to be decoded, and close neighbor sub-nodes that are L sub-node side lengths away from the node to be decoded in the negative direction of the target dimension of the coordinate system where the node to be decoded is located; L is a positive integer greater than 1.

[0041] Wherein, when the structure type of the first geometric structure is a prediction tree type, the first node as the node to be decoded is a node in a first prediction tree corresponding to the prediction tree type, and when the structure type of the second geometric structure is a partitioning tree type, the second node as the decoded neighbor node is a node in a partitioning tree corresponding to the partitioning tree type;

[0042] The placeholder determination module includes:

[0043] A second strategy acquisition unit, configured to acquire a second hybrid decoding optimization strategy associated with the first node based on second spatial decoding data information of the second node located in the partitioning tree;

[0044] The second placeholder determining unit is configured to use the placeholder code of the decoded neighboring node as the target placeholder information of the decoded neighboring node based on a second hybrid decoding optimization strategy.

[0045] The target occupancy information is used to determine the reconstruction point of the decoded neighboring node; the reconstruction point of the decoded neighboring node is used as the starting point of the second prediction tree corresponding to the node to be decoded or the independent reference point of the node to be decoded; the second prediction tree is determined by reconstructing the first prediction tree using the reconstruction point of the decoded neighboring node; the independent reference point is independent of the first prediction tree;

[0046] The geometric decoding module is specifically configured to use the first prediction reference point or the independent reference point in the second prediction tree as the target reference point, and decode the prediction tree residual information of the node to be decoded based on the reconstructed coordinate position information of the target reference point.

[0047] Wherein, when the structure type of the first geometric structure is a prediction tree type, the first node as the to-be-decoded node is a node in a first prediction tree corresponding to the prediction tree type, and when the decoded neighbor node is a third node in a third geometric structure having the same structure type as the first geometric structure, the third node as the decoded neighbor node is a node in a third prediction tree corresponding to the prediction tree type;

[0048] The device also includes:

[0049] The residual decoding module is used to determine the target reference point based on the starting point of the third prediction tree or the ending point of the third prediction tree, and decode the prediction tree residual information of the node to be decoded based on the reconstructed coordinate position information of the target reference point.

[0050] Among them, the residual decoding module is specifically used to reconstruct the first prediction tree through the target starting point when the starting point of the third prediction tree or the ending point of the third prediction tree is used as the target starting point, so as to obtain the reconstructed prediction tree corresponding to the node to be decoded; the starting point of the reconstructed prediction tree is the target starting point; and the second prediction reference point in the reconstructed prediction tree is used as the target reference point.

[0051] The target reference point is a starting point of the third prediction tree or an ending point of the third prediction tree, and the target reference point is independent of the first prediction tree.

[0052] An embodiment of the present application provides a point cloud data processing device, including:

[0053] a node determination module, configured to determine, when the node to be encoded is the first node in the first geometric structure of the point cloud data, an encoded neighboring node having a spatially adjacent relationship with the node to be encoded;

[0054] an information determination module, configured to determine target placeholder information of the encoded neighbor node when the encoded neighbor node is a second node in a second geometric structure different from the first geometric structure;

[0055] A geometric encoding module is used to encode the node geometric information of the node to be encoded based on the target occupancy information to obtain a compressed code stream of the point cloud data; the node geometric information includes any one of the partitioning tree occupancy information and the prediction tree residual information associated with the structure type of the first geometric structure.

[0056] In one aspect, an embodiment of the present application provides a computer device, including: a processor and a memory;

[0057] The processor is connected to a memory, wherein the memory is used to store a computer program. When the computer program is executed by the processor, the computer device executes the method provided in the embodiment of the present application.

[0058] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method provided by the embodiment of the present application.

[0059] In one aspect, embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in the embodiments of the present application.

[0060] In an embodiment of the present application, a decoding device can obtain a compressed code stream of point cloud data, and when the node to be decoded is the first node in a first geometric structure of the point cloud data, it can determine a decoded neighbor node that has a spatially adjacent relationship with the node to be decoded; further, when the decoded neighbor node is the second node in a second geometric structure different from the first geometric structure, the decoding device can determine the target occupancy information of the decoded neighbor node, and then can decode the node geometric information of the node to be decoded in the compressed code stream based on the target occupancy information; wherein, the node geometric information may include any one of the partitioning tree occupancy information and prediction tree residual information associated with the structural type of the first geometric structure. It can be seen that in the process of decoding the compressed code stream, when the geometric structure of any decoded node to be decoded (i.e., the first geometric structure mentioned above) is different from the geometric structure of the corresponding decoded neighbor node (i.e., the second geometric structure mentioned above), the decoding device can make full use of the target occupancy information of the decoded neighbor node to decode the node geometric information of the node to be decoded, rather than only considering the neighbor nodes in the geometric structure of the same structural type or the relevant information within the node to be decoded. In this way, when the first geometric structure is a partition tree and the second geometric structure is a prediction tree, the decoding efficiency of the partition tree occupancy information can be improved, or when the first geometric structure is a prediction tree and the second geometric structure is a partition tree, the decoding efficiency of the prediction tree residual information can be improved, thereby improving the decoding efficiency of the compressed code stream as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0062] Figure 1 This is a schematic diagram of the architecture of a point cloud media system provided by an embodiment of the present application;

[0063] Figure 2a This is a schematic diagram of a point cloud coding framework provided by an embodiment of the present application;

[0064] Figure 2b This is a schematic diagram of a point cloud decoding framework provided by an embodiment of the present application;

[0065] Figure 3 is a schematic diagram of a point cloud bounding box provided in an embodiment of the present application;

[0066] Figure 4a This is a schematic diagram of octree partitioning provided by an embodiment of the present application;

[0067] Figure 4b This is a schematic diagram of octree partitioning provided by an embodiment of the present application;

[0068] Figure 5 is a schematic diagram of an asymmetric point cloud bounding box provided in an embodiment of the present application;

[0069] Figure 6a This is a quadtree partitioning diagram provided by an embodiment of the present application;

[0070] Figure 6b This is a binary tree partitioning diagram provided in an embodiment of the present application;

[0071] Figure 7a This is a schematic diagram of an AVS prediction tree coding structure provided by an embodiment of the present application;

[0072] Figure 7b This is a schematic diagram of an MPEG prediction tree coding structure provided by an embodiment of the present application;

[0073] Figure 8 This is a block structure coding diagram provided by an embodiment of the present application;

[0074] Figure 9 This is a flow chart of a point cloud data processing method provided in an embodiment of the present application;

[0075] Figure 10 This is a schematic diagram of a parent node layer adjacency relationship provided in an embodiment of the present application;

[0076] Figure 11 This is a schematic diagram of a sub-node layer adjacency relationship provided in an embodiment of the present application;

[0077] Figure 12 This is a schematic diagram of a block-structured hybrid tree coding scenario provided by an embodiment of the present application;

[0078] Figure 13 This is a schematic diagram of a block-structured hybrid tree coding scenario provided by an embodiment of the present application;

[0079] Figure 14 This is a schematic diagram of a parent node layer adjacency relationship provided in an embodiment of the present application;

[0080] Figure 15 This is a schematic diagram of a parent node layer adjacency relationship provided in an embodiment of the present application;

[0081] Figure 16 This is a flow chart of a point cloud data processing method provided in an embodiment of the present application;

[0082] Figure 171 is a schematic structural diagram of a point cloud data processing device provided in an embodiment of the present application;

[0083] Figure 18 1 is a schematic structural diagram of a point cloud data processing device provided in an embodiment of the present application;

[0084] Figure 19 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0085] Figure 20 It is a structural diagram of a point cloud data processing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0086] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0087] The following is an introduction to some technical terms involved in the embodiments of this application:

[0088] 1. Immersive Media

[0089] Immersive media (also known as immersive media) refers to media files that can provide immersive media content, so that business objects immersed in the media content can obtain visual, auditory and other sensory experiences in the real world. Immersive media can be divided into 3DoF media, 3DoF+ media and 6DoF media according to the degree of freedom (DoF) of business objects when consuming media content. Among them, point cloud media is a typical 6DoF media. In the subsequent embodiments of this application, immersive media is explained as point cloud media as an example. In addition, the business objects here can be users (i.e. viewers) who consume immersive media (such as point cloud media).

[0090] 2. Point Cloud

[0091] A point cloud is a collection of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud contains at least 3D position information and, depending on the application scenario, may also contain color, material, or other information. Typically, each point in a point cloud has the same number of additional attributes.

[0092] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes, and therefore have a wide range of applications, including virtual reality (VR) games, computer-aided design (CAD), geographic information systems (GIS), autonomous navigation systems (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, and three-dimensional reconstruction of biological tissues and organs.

[0093] Point clouds are primarily acquired through computer generation, 3D (3-Dimensional) laser scanning, and 3D photogrammetry. Computers can generate point clouds of virtual 3D objects and scenes. 3D scanning can obtain point clouds of static, real-world 3D objects or scenes, generating millions of point clouds per second. 3D photography can obtain point clouds of dynamic, real-world 3D objects or scenes, generating tens of millions of point clouds per second. Furthermore, in the medical field, MRI (Magnetic Resonance Imaging), CT (Computed Tomography), and electromagnetic positioning information can be used to obtain point clouds of biological tissues and organs. These technologies reduce the cost and time required to acquire point cloud data while improving data accuracy. Changes in point cloud data acquisition methods have made it possible to acquire large amounts of point cloud data. With the continuous accumulation of large-scale point cloud data, efficient storage, transmission, publication, sharing, and standardization of point cloud data have become key to point cloud applications.

[0094] 3. DoF (degrees of freedom):

[0095] In this application, DoF refers to the degrees of freedom that a business object supports when watching immersive media (such as point cloud media) and generates content interaction, which can include 3DoF (three degrees of freedom), 3DoF+ and 6DoF (six degrees of freedom). Among them, 3DoF refers to the three degrees of freedom of rotation of the business object's head around the x-axis, y-axis, and z-axis. 3DoF+ is based on the three degrees of freedom, and the business object also has the freedom of limited movement along the x-axis, y-axis, and z-axis. 6DoF is based on the three degrees of freedom, and the business object also has the freedom of free movement along the x-axis, y-axis, and z-axis.

[0096] 4. ISOBMFF (ISO Based Media File Format): It is a media file encapsulation standard. A typical ISOBMFF file is an MP4 (Moving Picture Experts Group 4) file.

[0097] The embodiments of the present application relate to point cloud media data processing technology. Some concepts in the point cloud media data processing process are introduced below.

[0098] It can be understood that the complete processing process of point cloud media from acquisition to consumption can specifically include: point cloud acquisition, point cloud compression (PCC), point cloud file encapsulation, point cloud file transmission, point cloud file decapsulation, point cloud decoding and final video presentation.

[0099] Among them, point cloud acquisition refers to the collection of point cloud data from different angles through multiple cameras. Point cloud encoding refers to the conversion of files in the original video format into files in another video format through compression technology. From the perspective of the acquisition method of point cloud data, point cloud data can be divided into two methods: camera capture and computer generation. Due to different statistical characteristics, the corresponding compression encoding methods may also be different. Commonly used compression encoding methods can include HEVC (High Efficiency Video Coding, international video coding standard HEVC / H.265), VVC (Versatile Video Coding, international video coding standard VVC / H.266), AVS (Audio Video Coding Standard, China National Video Coding Standard), AVS3 (the third-generation video coding standard launched by the AVS standard group), MPEG (Moving Picture Expert Group, international audio and video coding standard), etc.

[0100] After point cloud encoding, the encoded data stream (i.e., point cloud code stream) needs to be encapsulated and transmitted to the business object. Point cloud file encapsulation refers to storing the encoded and compressed point cloud code stream in a file in a certain format according to the encapsulation format (or container, or file container). Common encapsulation formats include AVI format (Audio Video Interleaved) or ISOBMFF format. In one embodiment, the point cloud code stream is encapsulated in a file container according to a file format such as ISOBMFF to form a point cloud file (also called a media file, encapsulation file, or video file). The point cloud file can be composed of multiple tracks (Tracks), such as a video track, an audio track, and a subtitle track.

[0101] After the encoding device (or content production device) performs the above encoding and file encapsulation processes, it can transmit the point cloud file to the decoding device (or content consumption device). The decoding device can then present the final media content after performing inverse operations such as decapsulation and decoding. The point cloud file can be sent to the decoding device based on various transmission protocols, including but not limited to: DASH (dynamic adaptive streaming over HTTP, dynamic adaptive streaming over HTTP is an adaptive bitrate streaming technology), HLS (HTTP Live Streaming, dynamic bitrate adaptive transmission) protocol, SMTP (Smart Media Transport Protocol, intelligent media transmission protocol), TCP (Transmission Control Protocol, transmission control protocol), etc.

[0102] It is understood that the file decapsulation process of the decoding device is the inverse of the file encapsulation process described above. The decoding device can decapsulate the point cloud file according to the file format requirements when it was encapsulated to obtain a point cloud stream. The decoding process of the decoding device is also the inverse of the encoding process. For example, the decoding device can decode the point cloud stream to restore the media content.

[0103] For easier understanding, see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a point cloud media system provided by an embodiment of the present application. Figure 1As shown, the point cloud media system may include an encoding device (e.g., encoding device 100A) and a decoding device (e.g., decoding device 100B). The encoding device may refer to a computer device used by a point cloud media provider (e.g., a content producer of the point cloud media). The computer device may be a terminal (e.g., a PC (Personal Computer), a smart mobile device (e.g., a smartphone), etc.) or a server. The server may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The decoding device may refer to a computer device used by a point cloud media user (e.g., a viewer of the point cloud media, i.e., a business object). The computer device may be a terminal (e.g., a PC (Personal Computer), a smart mobile device (e.g., a smartphone), a VR device (e.g., a VR helmet, VR glasses, etc.), a smart home appliance, a vehicle-mounted terminal, an aircraft, etc.), and the computer device may be integrated with a client. The client herein may be a client capable of displaying data information such as text, images, audio, and video, including but not limited to a multimedia client (e.g., a video client), a social client (e.g., an instant messaging client), an information client (e.g., a news client), an entertainment client (e.g., a game client), a shopping client, an in-car client, a browser, etc. The client may be a standalone client or an embedded sub-client integrated into a client (e.g., a social client), and the like, without limitation.

[0104] It is understood that the data processing technology for point cloud media involved in this application can be implemented using cloud technology; for example, using a cloud server as an encoding device. Cloud technology refers to a hosting technology that unifies hardware, software, network, and other resources within a wide area network or local area network to achieve data computing, storage, processing, and sharing.

[0105] The data processing process of point cloud media includes the data processing process on the encoding device side and the data processing process on the decoding device side.

[0106] The data processing process on the encoding device side mainly includes: (1) the acquisition and production process of the media content of the point cloud media; (2) the encoding and file packaging process of the point cloud media. The data processing process on the decoding device side mainly includes: (1) the file decapsulation and decoding process of the point cloud media; (2) the rendering process of the point cloud media. In addition, the transmission process of the point cloud media between the encoding device and the decoding device can be carried out based on various transmission protocols, which may include but are not limited to: DASH protocol, HLS protocol, SMTP protocol, TCP protocol, etc.

[0107] The following will be combined Figure 1 , each process involved in the data processing of point cloud media is introduced respectively.

[0108] 1. Data processing on the encoding device side:

[0109] (1) The acquisition and production process of media content of point cloud media.

[0110] 1) The process of acquiring media content of point cloud media.

[0111] The media content of point cloud media is obtained by capturing the real-world audio-visual scene using a capture device. In one implementation, the capture device can refer to a hardware component within the encoding device, such as a microphone, camera, or sensor on a terminal. In another implementation, the capture device can also be a hardware device connected to the encoding device, such as a camera connected to a server, which provides the encoding device with a service for acquiring the media content of point cloud media. The capture device can include, but is not limited to, audio equipment, video equipment, and sensor equipment. Audio equipment can include audio sensors, microphones, etc. Video equipment can include standard cameras, stereo cameras, light field cameras, etc. Sensor equipment can include laser equipment, radar equipment, etc. There can be multiple capture devices, which are deployed at specific locations in real space to simultaneously capture audio and video content from different angles within the space, with the captured audio and video content synchronized in both time and space. In embodiments of the present application, the media content in a three-dimensional space captured by capture devices deployed at specific locations to provide a multi-degree-of-freedom (e.g., 6DoF) viewing experience can be referred to as point cloud media.

[0112] For example, take the video content of point cloud media as an example. Figure 1 As shown, a visual scene 10A (e.g., a real-world visual scene) can be captured by a camera array connected to an encoding device 100A, or by a camera device having multiple cameras and sensors connected to the encoding device 100A. The captured result can be source point cloud data 10B (i.e., video content of the point cloud media).

[0113] 2) The production process of media content of point cloud media.

[0114] It should be understood that the production process of the media content of the point cloud media involved in the embodiments of this application can be understood as the process of producing the content of the point cloud media, and the content production of the point cloud media here is mainly produced by content in the form of point cloud data captured by cameras or camera arrays deployed in multiple locations. For example, the encoding device can convert the point cloud media from a three-dimensional representation to a two-dimensional representation. The point cloud media here can include geometric information, attribute information, placeholder information, and atlas data. Point cloud media generally requires specific processing before encoding, such as the point cloud data needs to be cut and mapped before encoding.

[0115] In addition, it should be noted that since panoramic videos can be captured by capture devices, such videos are processed by encoding devices and transmitted to decoding devices for corresponding data processing. Business objects on the decoding device side need to perform some specific actions (such as head rotation) to view 360-degree video information, while performing non-specific actions (such as moving the head) cannot obtain corresponding video changes, and the VR experience is not good. Therefore, it is necessary to provide additional depth information that matches the panoramic video to enable business objects to obtain better immersion and a better VR experience, which involves 6DoF production technology. When business objects can move more freely in a simulated scene, it is called 6DoF. When using 6DoF production technology to produce video content of point cloud media, capture devices generally use laser equipment, radar equipment, etc. to capture point cloud data in space.

[0116] (2) The process of encoding and file packaging of point cloud media.

[0117] The captured audio content can be directly encoded to form an audio stream for point cloud media. The captured video content can be encoded to produce a video stream for point cloud media. It should be noted that if 6DoF production technology is used, a specific encoding method (such as a point cloud compression method based on traditional video encoding) must be used during the video encoding process. The audio and video streams are encapsulated in a file container according to the point cloud media file format (such as ISOBMFF) to form a media file resource for point cloud media. This media file resource can be a media file or a media fragment forming a point cloud media file. The point cloud media file metadata is recorded using Media Presentation Description (MPD) information in accordance with the requirements of the point cloud media file format. Metadata here is a general term for information related to the presentation of point cloud media. This metadata may include descriptions of the media content, descriptions of the viewport, and signaling information related to the presentation of the media content. It can be understood that the encoding device will store the media presentation description information and media file resources formed after the data processing process. The media presentation description information and media file resources here can be encapsulated into a point cloud file for sending to the decoding device according to a specific media file format.

[0118] Specifically, in one embodiment, the captured audio is encoded into a corresponding audio bitstream. The point cloud media's geometric information, attribute information, and placeholder information can be encoded using traditional video encoding methods, while the point cloud media's atlas data can be entropy encoded. The encoded media is then encapsulated in a file container according to a specific format (such as ISOBMFF) and combined with metadata describing the media content attributes and viewport metadata to form a media file or an initialization segment and media segments according to a specific media file format.

[0119] For example, Figure 1As shown, the encoding device 100A performs point cloud media encoding on one or more data frames in the source point cloud data 10B, for example, using geometry-based point cloud compression (G-PCC), thereby obtaining an encoded point cloud code stream 10E (i.e., a video code stream, such as a G-PCC code stream), including a geometry code stream (i.e., a code stream obtained after encoding the geometric information) and an attribute code stream (i.e., a code stream obtained after encoding the attribute information). Subsequently, the encoding device 100A can encapsulate the one or more encoded code streams into a media file 10F for local playback according to a specific media file format (such as ISOBMFF), or encapsulate them into a segment sequence 10Fs containing an initialization segment and multiple media segments for streaming. In addition, the file encapsulator in the encoding device 100A can also add relevant metadata to the media file 10F or the segment sequence 10Fs. Furthermore, the encoding device 100A may use a certain transmission mechanism (such as DASH, SMT) to transmit the segment sequence 10Fs to the decoding device 100B, or transmit the media file 10F to the decoding device 100B. In some embodiments, the decoding device 100B may be a player.

[0120] 2. Data processing on the decoding device side:

[0121] (3) The process of decapsulating and decoding point cloud media files.

[0122] The decoding device can dynamically and adaptively obtain the point cloud media's media file resources and corresponding media presentation description information from the encoding device based on recommendations from the encoding device or according to the needs of the business object on the decoding device. For example, the decoding device can determine the business object's viewing direction and position based on the head / eye position information of the business object, and then dynamically request the corresponding media file resources from the encoding device based on the determined viewing direction and position. The media file resources and media presentation description information are transmitted from the encoding device to the decoding device via a transport mechanism (such as DASH and SMT). The file decapsulation process on the decoding device is the inverse of the file encapsulation process on the encoding device. The decoding device decapsulates the media file resources according to the point cloud media file format (e.g., ISOBMFF) to obtain audio and video streams. The decoding process on the decoding device is the inverse of the encoding process on the encoding device. The decoding device decodes the audio stream to restore the audio content, and the decoding device decodes the video stream to restore the video content.

[0123] For example, Figure 1As shown, the media file 10F output by the file encapsulator in the encoding device 100A is the same as the media file 10F' input to the file decapsulator in the decoding device 100B. The file decapsulator performs file decapsulation processing on the media file 10F' or the received fragment sequence 10F's, and extracts the encoded point cloud code stream 10E', while parsing the corresponding metadata. Subsequently, the point cloud media decoding can be performed on the point cloud code stream 10E' to obtain the decoded video signal 10D', and the point cloud data (i.e., the restored video content) can be reconstructed from the video signal 10D'. Among them, the media file 10F and the media file 10F' may include a track format definition, which may include constraints on the basic streams contained in the samples in the track.

[0124] (4) Rendering process of point cloud media.

[0125] The decoding device renders the audio content obtained by audio decoding and the video content obtained by video decoding according to the rendering-related metadata in the media presentation description information corresponding to the media file resource. Once the rendering is completed, the playback output of the content is realized.

[0126] For example, Figure 1 As shown, the decoding device 100B can render the aforementioned reconstructed point cloud data based on the current viewing position, viewing direction or viewport, and display it on the screen of a head-mounted display or any other display device. The current viewport can be determined by various types of sensors. For example, the sensors here may include head detection sensors, and possibly position detection sensors or eye detection sensors. In addition to being used by the decoding device 100B to obtain the appropriate portion of the decoded point cloud data, the current viewing position or viewing direction can also be used for decoding optimization. In addition, in the viewport-related transmission, the current viewing position and viewing direction will also be passed to the strategy module in the decoding device 100B, which can determine the track to be received based on the current viewing position and viewing direction.

[0127] As can be seen from the above, the decoding device can dynamically obtain the media file resources corresponding to the point cloud media from the encoding device side. Since the media file resources are obtained by the encoding device encoding and encapsulating the captured audio and video content, after the decoding device receives the media file resources returned by the encoding device, it needs to first decapsulate the media file resources to obtain the corresponding audio and video code stream, and then decode the audio and video code stream, and finally present the decoded audio and video content to the business object. The point cloud media here can include but is not limited to V-PCC (Video-based Point Cloud Compression, point cloud compression based on traditional video coding) point cloud media and G-PCC (point cloud compression based on geometric models) point cloud media.

[0128] In the embodiment of the present application, the data representing each point (which may be referred to as a point cloud point) in the point cloud captured from the three-dimensional space can be collectively referred to as point cloud data, that is, each point cloud point can be represented by the corresponding point cloud data. For point cloud data, it can be divided into dense point cloud and sparse point cloud based on the data acquisition method; it can be divided into static point cloud and dynamic point cloud based on the time series type of the data. The embodiment of the present application does not limit the point cloud data type. It can be understood that the point cloud data here may include geometric information and attribute information of each point cloud point, wherein the geometric information may specifically include the three-dimensional position information of the point cloud point in the three-dimensional space (also referred to as coordinate position information). For example, the geometric position of the point cloud point can be represented by the x, y, and z coordinates of the point cloud point in the coordinate system of the three-dimensional space. In addition, the coordinate position information of each point cloud point can be collectively referred to as the spatial coordinate position information of the point cloud data; the attribute information may include the attributes of the point cloud point, such as the corresponding scalar features (such as reflectivity) and vector features (such as color).

[0129] Among them, the current mainstream point cloud coding technology can be divided into geometric structure-based point cloud coding and projection-based point cloud coding for different point cloud data types. For example, in geometric structure-based point cloud coding, a specific geometric structure (such as an octree) can be used to represent point cloud data, and different coding processing flows are used for the corresponding geometric information and attribute information. It should be noted that the embodiment of this application will perform relevant optimizations at the point cloud encoding and decoding level. For ease of understanding, the main processing procedures of point cloud encoding and decoding will be explained below using geometric structure-based point cloud coding as an example.

[0130] Please also see Figure 2a , Figure 2a This is a schematic diagram of a point cloud coding framework provided by an embodiment of the present application. The point cloud coding framework is applicable to the geometric structure-based point cloud coding G-PCC in MPEG and the point cloud coding standard AVS-PCC in AVS, and both G-PCC and AVS-PCC can be used for static sparse point clouds. Figure 2a As shown, the point cloud coding framework can specifically include geometric information coding (also known as geometric structure coding) and attribute information coding. Figure 2a , the main operations and processing of geometric information encoding and attribute information encoding are introduced respectively.

[0131] 1. The main processing process of geometric information encoding:

[0132] (1) Pre-processing: The pre-processing of geometric information may include coordinate transformation and voxelization, that is, by scaling and translation operations, the spatial coordinate position information of the point cloud data in the 3D space is converted from floating-point form to integer form, and its minimum geometric position is moved to the coordinate origin. First, the original geometric information (that is, the original spatial coordinate position information) can be coordinate transformed so that all point cloud points after the coordinate transformation are contained in a polyhedron (such as a hexahedron) bounding box, which can also be called a point cloud bounding box. The origin and size of the point cloud bounding box can be determined by the original geometric information. For example, assuming that the floating-point coordinates of the k-th point (that is, the k-th point cloud point) of the input point cloud (that is, the spatial coordinate position information of the k-th point) are expressed as (x k ,y k ,z k ), where k = 0, ..., K-1, K is the number of points in the point cloud, and the coordinate point (x min ,y min ,z min ,) and (x max ,y max ,z max ,) can be expressed as follows:

[0133] x min =min(x 0 ,x 1 ,…,x K-1 ) (1)

[0134] y min =min(y 0 ,y 1 ,…,y K-1 ) (2)

[0135] z min =min(z 0 ,z 1 ,…,z K-1 ) (3)

[0136] x max =max(x 0 ,x 1 ,…,x K-1 ) (4)

[0137] y max =max(y 0 ,y 1 ,…,y K-1 ) (5)

[0138] z max=max(z 0 ,z 1 ,…,z K-1 ) (6)

[0139] Among them, the function min(s 0 ,s 1 ,…,s K-1 ) means taking the minimum value of the current input, function max(s 0 ,s 1 ,…,s K-1 ) means taking the maximum value of the current input. The point cloud bounding box represents the largest polyhedron (e.g., a cuboid) that contains all the points in the input point cloud. The origin of the point cloud bounding box (x origin ,y origin ,z origin ) can be calculated as follows:

[0140] x origin =int(floor(x min )) (7)

[0141] y origin =int(floor(y min )) (8)

[0142] z origin =int(floor(z min )) (9)

[0143] Correspondingly, the size of the point cloud bounding box in the x, y, and z directions can be calculated as follows:

[0144] BoudingBoxSize x =int(x max -x origin )+1 (10)

[0145] BoudingBoxSize y =int(y max -y origin )+1 (11)

[0146] BoudingBoxSize z =int(z max -z origin )+1 (12)

[0147] Among them, int(s) is the integer function, and floor(s) function returns the largest integer value less than or equal to s.

[0148] For easier understanding, please refer to Figure 3 , Figure 3Schematic diagram of a point cloud bounding box provided in an embodiment of the present application. Figure 3 The point cloud bounding box shown is a rectangular parallelepiped containing all relevant point cloud points. In some scenarios, the point cloud bounding box constructed can also be a cube with equal length, width and height. The embodiment of the present application does not limit the shape of the point cloud bounding box.

[0149] Secondly, the geometric information after coordinate transformation can be quantized (geometry quantization). Here, geometric quantization primarily serves a scaling purpose (e.g., rounding the coordinates of point cloud points). The degree of quantization is typically determined by the quantization parameter (QP). A larger QP value means that coefficients with a larger value range will be quantized into the same output, which generally results in greater distortion and a lower bitrate. Conversely, a smaller QP value means that coefficients with a smaller value range will be quantized into the same output, which generally results in less distortion and a higher bitrate. In point cloud coding, geometric quantization is performed directly on the coordinate information of the point. As can be appreciated, after this quantization rounding, multiple point cloud points with the same location (i.e., the same geometric information) may exist, known as duplicate points. In this case, the decision to remove duplicate points can be made based on relevant parameters. Furthermore, multiple points with the same quantized location but different attributes can be merged into a single point. The attributes associated with the single point can be calculated by the attribute transformation module. The process of geometric quantization, removing duplicate points, and assigning attributes to the remaining points is called voxelization.

[0150] (2) Geometry encoding: Geometry encoding contains multiple modes that can be used under different conditions.

[0151] (a) Geometric coding based on partitioning tree: The partitioning tree in the embodiment of the present application may specifically include various types of tree data structures such as octree (OT), quadtree (QT), binary tree (BT), etc., among which octree is the most common three-dimensional point cloud geometric coding method. For example, in G-PCC and AVS-PCC, the octree structure is mainly used to divide the point cloud. When the length, width and height of the bounding box of the three-dimensional scene are asymmetric, in some cases the octree can be simplified to quadtree or binary tree division. In the embodiment of the present application, the geometric coding based on octree, the geometric coding based on quadtree, and the geometric coding based on binary tree are collectively referred to as partitioning tree coding methods. For ease of understanding, the octree is used as an example for introduction. For point cloud data in 3D space, the octree partitioning method is to evenly divide the pre-set point cloud bounding box layer by layer. If the point cloud bounding box of the three-dimensional scene is a size of (2 N-1 ,2 N-1,2 N-1 ) cube (i.e. cube), the cube is called the root node of the octree, and the root node is partitioned by octree, that is, the cube can be recursively divided into 8 sub-cubes, corresponding to the 8 child nodes of the octree (i.e. each sub-cube is called the child node of the root node). If one of the child nodes contains at least one 3D point (i.e. point cloud point), then this child node will be further divided into 8 smaller child nodes (i.e. sub-cubes), until it is divided into the smallest unit, i.e. a 1×1×1 sub-cube (i.e. unit cube). The entire partitioning process adopts breadth-first traversal (Breath First Search, BFS, also known as breadth-first search or horizontal-first search). Here N represents the maximum number of octree partition levels. In this way, except for the leaf node (i.e. unit cube), each node in the octree has eight child nodes. For each child node of the octree, one bit is used to indicate whether it is occupied (i.e., whether it contains a point in the point cloud). For example, a first code value (e.g., '1') indicates that the child node is occupied (i.e., the child node contains at least one point in the point cloud), and a second code value (e.g., '0') indicates that the child node is not occupied (i.e., the child node does not contain any point). This bit can be called a placeholder code or occupancy code. It can be understood that the occupied child node will be further divided into eight child nodes, and the code stream of the geometric information can be determined based on the obtained placeholder code information. In addition, the embodiments of the present application do not limit the specific values of the first code value and the second code value.

[0152] Among them, the construction of the octree can be realized based on the Morton code. For example, the 3D coordinate information of the point cloud data can be converted into the corresponding Morton code by querying the Morton sequence table or through relevant calculations. Then, according to the sorting of each Morton code, the corresponding nodes of each layer of the octree are obtained starting from the root node (i.e., the 0th layer) in a breadth-first manner until all nodes become leaf nodes. The construction of the octree is completed. It can be understood that each time the octree node is divided, the placeholder code of the node will contain eight bits (b7b6b5b4b3b2b1b0), each bit representing the occupancy of the eight child nodes of the node.

[0153] For easier understanding, see Figure 4a-4b , Figure 4a-4b This is a schematic diagram of octree partitioning provided by an embodiment of the present application. Figure 4a The cube shown can be divided into 8 sub-cubes, corresponding to the 8 child nodes of the octree. Assume that its placeholder code can be as follows Figure 4a The sequence coding shown is to start from position 0 and code to position 7 in sequence. It can be understood that other coding sequences can be designed as needed in actual applications, which is not limited here. Figure 4bAs shown, assuming that in the node corresponding to the cube, the child node at position 0 (which can be called child node 0) and the child node at position 7 (which can be called child node 7) are occupied, their corresponding placeholder codes can be 10000001; further, the child node at position 0 and the child node at position 7 can be divided into 8 smaller child nodes respectively, so as to obtain the placeholder codes corresponding to child node 0 and child node 7 according to the occupancy of each child node, for example, the placeholder code corresponding to child node 0 is 00011000, and the placeholder code corresponding to child node 7 is 10001000.

[0154] It is understandable that if the point cloud bounding box of the 3D scene is asymmetrical in length, width and height, that is, the point cloud bounding box is a size of A rectangular parallelepiped, where D x ≠D y ≠D z ,like Figure 5 As shown, Figure 5 is a schematic diagram of an asymmetric point cloud bounding box provided in an embodiment of the present application. Figure 5 The point cloud bounding box shown can be a cuboid of any size to better adapt to the shape of the 3D scene or object. Since the point cloud bounding box may not be a symmetrical cube, in some cases, the nodes may not (or cannot) be divided along all directions. In other words, the octree can be simplified to a quadtree or binary tree partition. For ease of understanding, please refer to Figure 6a , Figure 6a This is a quadtree partitioning diagram provided by the embodiment of the present application. Figure 6a As shown, the quadtree partitioning can be performed along any two of the three directions of x, y, and z, for example Figure 6a The results of quadtree partitioning along the xy, xz, and yz directions are shown in turn. It can be understood that only 4 bits need to be encoded in the quadtree encoding mode, which saves 4 bits compared to the octree encoding. For example, Figure 4a Compared with the octree partitioning method shown in Figure 6a As shown in the first figure, the placeholder information obtained by quadtree partitioning along the xy axis can only encode the bits corresponding to positions 0, 2, 4, and 6, while the bits corresponding to the skipped positions 1, 3, 5, and 7 can be inferred to be 0 at the decoding end. Similarly, please refer to Figure 6b , Figure 6b This is a binary tree partitioning diagram provided by the embodiment of the present application. Figure 6b As shown, the binary tree partition can be divided along any of the three directions of x, y, and z, for example Figure 6bThe results of binary tree partitioning along the x, y, and z directions are shown in turn. It can be understood that only 2 bits are needed to encode in the binary tree encoding mode, which saves 6 bits compared to the octree encoding. For example, Figure 4a Compared with the octree partitioning method shown in Figure 6b As shown in the first figure, the placeholder information obtained by binary tree partitioning along the x-axis can only encode the bits corresponding to positions 0 and 4, and the remaining 6 bits skipped can be inferred to be 0 at the decoding end.

[0155] It is understandable that in actual applications, corresponding partitioning strategies can be formulated as needed to perform corresponding binary tree, quadtree or octree partitioning when specified conditions are met. The embodiments of the present application do not limit the partitioning strategy adopted.

[0156] (b) Predictive tree-based geometric coding: This approach, also known as predictive geometry coding, is a parallel approach to partition tree-based geometric coding. It offers advantages such as support for low-latency applications or streaming media, and low-complexity decoding. It first defines a prediction structure on the point cloud, which can be described by a prediction tree, where each point in the point cloud is associated with a node in the prediction tree. Each node can only be predicted by one or more ancestors in the prediction tree (i.e., the nodes preceding it in the prediction tree). The prediction mode is selected based on the prediction mode, and the specific prediction mode used is not specified here.

[0157] For example, optionally, in AVS, the prediction tree can take a single chain structure. For ease of understanding, see Figure 7a , Figure 7a This is a schematic diagram of an AVS prediction tree coding structure provided by an embodiment of the present application. Figure 7a As shown, first, the root node can be selected based on the specified root node selection rule, and then the point closest to the root node is searched in the KD (K-Dimensional) tree determined by all the remaining points as the only child node of the root node, and the child node is removed from the KD tree at the same time; then the nearest neighbor point of the child node is searched in the KD tree in the same way as its child node; and so on, the construction of the entire single-chain prediction tree is completed. After all points are linked into a single prediction tree, each point can be predicted based on the signal value (i.e., coordinate information) of its previous point. As an option for point cloud signal prediction, this method can be set to act on the entire original point cloud data, or on the octree child node (i.e., combined with geometric encoding based on the partitioning tree) or a subset of point cloud data obtained by other means, which is not limited here.

[0158] For example, in MPEG, a parent-child structure is used to construct a prediction tree. For ease of understanding, see Figure 7b , Figure 7b This is a schematic diagram of an MPEG prediction tree coding structure provided by an embodiment of the present application. Figure 7b As shown in , we can also first select the root vertex based on the specified root node selection rule, and then search the nearest neighbor point in the constructed tree one by one in order as the parent (i.e., ancestor). If the number of children (also known as child nodes) of the parent is less than the preset maxChildNum (i.e., the maximum number of child nodes, also known as the child node number threshold), then it can be used as a child point of the node (e.g., Figure 7b Branch nodes (Branchvertex) with one or more children (or children) are linked into the prediction tree, resulting in a multi-link structure. Furthermore, the predictive coding technology in G-PCC uses distance search between points to build a prediction tree. Each point is connected to other points, indicating the number of points it is linked to and one of the four prediction modes it uses. These four prediction modes are as follows:

[0159] 1) No prediction;

[0160] 2) Use parent point prediction, that is, the previous point;

[0161] 3) Use parent point and grandparent point prediction;

[0162] 4) Use parent, grandparent and grand-grandparent point predictions.

[0163] It can be understood that each node in the prediction tree can be predicted according to the prediction mode adopted by it, and the specific prediction mode adopted here is not limited. For a node x in the prediction tree, after determining its reference point (such as the previous point of the node x), the geometric coordinate reconstruction value of the reference point can be used to predict the node x, thereby obtaining the predicted geometric coordinates, and then the original geometric coordinates of the node x (that is, the coordinate position information of the node x) and the predicted geometric coordinates can be differentiated to obtain the corresponding residual information. The embodiment of the present application can refer to the residual information as the prediction tree residual information, and finally the code stream of the geometric information can be determined based on the obtained prediction tree residual information.

[0164] (c) Trisoup-based geometric encoding: Divide the point cloud into blocks of a certain size, locate the intersection points of the point cloud surfaces at the edges of the blocks and construct triangles, and compress the geometric information by encoding the intersection points.

[0165] (3) Geometry entropy encoding: Statistical compression encoding is performed on the placeholder information (also called partition tree placeholder information) of the partition tree (such as an octree) or the prediction tree residual information, and finally a binary (0 or 1) compressed code stream (i.e., geometric code stream) is output. Statistical compression encoding here is a lossless encoding method that can effectively reduce the bit rate required to express the same signal. Common statistical compression encoding methods include context-based binary arithmetic coding (CABAC, Content Adaptive Binary Arithmetic Coding).

[0166] 2. The main processing process of attribute information encoding:

[0167] (1) Attribute Recoloring: In the case of lossy encoding, after the geometric information is encoded, the encoding device needs to decode and reconstruct the geometric information, that is, restore the coordinate information of each point in the 3D point cloud. Then, the attribute information corresponding to one or more neighboring points in the original point cloud can be found as the attribute information of the reconstructed point (that is, the unencoded attribute information is matched with the reconstructed geometric information).

[0168] (2) Attribute transformation coding (Transform): Attribute transformation coding includes four attribute transformation modes that can be used under different conditions.

[0169] (a) Prediction: Based on information such as distance or spatial relationship, the neighboring points of the point to be coded are determined as prediction points among the coded points, and the predicted value of the point is calculated according to the set criteria. The difference between the current point attribute value and the predicted value is calculated as the attribute residual information, and the attribute residual information is quantized, transformed (optional), and entropy coded;

[0170] (b) Transformation: Attribute information is grouped and transformed using transformation methods such as DCT (Discrete Cosine Transform) and Haar transform (an orthogonal normalization transform). The transform coefficients are quantized. Attribute reconstruction values are obtained through inverse quantization and inverse transformation. The difference between the original attribute and the reconstructed attribute value is calculated to obtain attribute residual information and quantized. The quantized transform coefficients and attribute residual information are encoded.

[0171] (c) Prediction transformation: transformation is performed using the residual signal obtained by prediction;

[0172] (d) Transformation prediction: predicting the transformed parameter information;

[0173] (3) Attribute quantization: The degree of quantization is usually determined by the quantization parameter (QP). For example, in predictive transform coding and lifting transform coding, the residual value is quantized and then entropy coded; in RAHT, the transform coefficients are quantized and then entropy coded.

[0174] (4) Attribute entropy coding: Quantized attribute residual information or transform coefficients are generally compressed using run length coding and arithmetic coding to obtain the attribute bitstream. The corresponding coding mode, quantization parameters, and other information are also encoded using an entropy encoder.

[0175] For further information, please see Figure 2b , Figure 2b Schematic diagram of a point cloud decoding framework provided by an embodiment of the present application. Similarly, the point cloud decoding framework is applicable to G-PCC and AVS-PC. Figure 2b As shown, the point cloud decoding framework can specifically include geometric code stream decoding and attribute code stream decoding. It can be understood that the decoding process is inverse to the encoding process, that is, the geometric code stream decoding process is inverse to the above-mentioned geometric information encoding process, and the attribute code stream decoding is inverse to the above-mentioned attribute information encoding process. Figure 2b , respectively introduces the main operations and processing of geometric code stream decoding and attribute code stream decoding.

[0176] 1. The main processing process of geometric code stream decoding:

[0177] According to the above-mentioned geometric information encoding process, it can be understood that after obtaining the geometric code stream (i.e., the aforementioned compressed code stream), the decoding device first performs geometric entropy decoding to obtain the corresponding partition tree occupancy information or prediction tree residual information, so that the partition tree can be reconstructed based on the obtained partition tree occupancy information, or the prediction tree can be reconstructed based on the obtained prediction tree residual information, and then the voxelized geometric information (such as quantized geometric information) can be obtained based on the reconstructed partition tree or prediction tree. After performing post-processing (including inverse voxelization and inverse coordinate transformation) operations, the reconstructed 3D point position information (i.e., the reconstructed geometric information) can be obtained.

[0178] 2. The main processing process of attribute code stream decoding:

[0179] According to the above-mentioned attribute information encoding process, it can be understood that after obtaining the attribute code stream, the decoding device first performs entropy decoding on the attribute code stream to obtain various mode information and quantized attribute information, and then dequantizes it to obtain attribute residual information or transformation coefficients, so that the reference signal can be confirmed according to the adopted attribute transformation mode to obtain reconstructed attribute information, and the reconstructed attribute information is sequentially corresponded one by one with the reconstructed geometric information, thereby outputting the reconstructed point cloud data.

[0180] It is understandable that in the above Figure 2a In the point cloud coding framework shown in the figure, a block structure coding method can be used. This method introduces a concept similar to the coding block in video coding, dividing the three-dimensional space into several The size of the 3D macroblock is controlled by non-overlapping coding macroblocks (LCUs). Each macroblock is used as a basic coding unit. There is no limit on the number of macroblocks obtained by the division. The user can set the division depth of the division tree (such as octree), or control the size of the 3D macroblock through related parameters (such as nodeSizeLog2, i.e. node size parameter). There is no limit on the size of the macroblock. For example, when the size of each node of the octree is And the following conditions are met:

[0181] nodeSizeLog2>max(d x ,d y ,d z ) (13)

[0182] Each node is encoded as a macroblock. The partition depth here refers to the depth of the geometric partition. The depth of the root node can be defined as 0, and the depth value increases by 1 each time the partition is performed.

[0183] For easier understanding, see Figure 8 , Figure 8 This is a block structure coding diagram provided by the embodiment of the present application. Figure 8 As shown in the figure, in the 2D quadtree partitioning scenario, assuming that the encoding starts from the second layer in the order of macroblocks, each macroblock can be regarded as a separate tree partitioning structure. In the current implementation, a breadth-first traversal can be used inside each coded macroblock. For example, inside macroblock 1, the second layer (node 1) is traversed first, then the third layer (node 2, node 3), and the fourth layer (node 4, node 5) are traversed. This block structure encoding method has high flexibility. For example, different geometric encoding methods can be used for each macroblock according to its characteristics. For example, Figure 8 The macroblock 1 shown uses geometric coding based on partition tree. Figure 8Macroblock 2 shown uses prediction tree-based geometric coding. This embodiment of the application does not limit the geometric coding method used for macroblocks. Furthermore, attribute coding can be performed after geometric coding of a macroblock is complete. This eliminates the need to wait for the entire point cloud to be geometrically encoded before performing attribute coding, thereby increasing the flexibility of the coding method.

[0184] Based on this, to address the problem of low coding and decoding efficiency caused by using different geometric coding methods for different macroblocks, the embodiments of the present application provide a geometric coding and decoding optimization method based on a hybrid tree structure. When encoding the partition tree, the coding information of the prediction tree can be used to deduce the occupancy of neighbor nodes and child nodes, thereby improving the coding efficiency of the partition tree occupancy code; when encoding the prediction tree, the occupancy of the partition tree can be used to improve the coding efficiency of the prediction tree. Similarly, when decoding the partition tree or the prediction tree, the decoding efficiency can also be correspondingly improved.

[0185] For ease of understanding and explanation, the embodiments of the present application collectively refer to the partition tree and the prediction tree as geometric structures (or tree-shaped data structures). The two have different structural types, wherein the structural type corresponding to the partition tree can be referred to as a partition tree type, and the structural type corresponding to the prediction tree can be referred to as a prediction tree type. The partition tree here can include but is not limited to geometric structures based on spatial partitioning such as octrees, quadtrees, and binary trees. It should be noted that since the encoding process and the decoding process are mutually inverse, many similar names in the embodiments of the present application are actually different expressions of the same information or content at the encoding end and the decoding end. For example, at the encoding end, the embodiments of the present application can refer to the node to be encoded currently as a node to be encoded, and the node to be encoded can be any node in the current macroblock to be encoded; at the same time, the node that has a spatial adjacent relationship with the node to be encoded and has been encoded can be referred to as a node that has been encoded. Similarly, at the decoding end, the embodiments of the present application can refer to the node to be decoded currently as a node to be decoded, and the node to be decoded can be any node in the current macroblock to be decoded; at the same time, the node that has a spatial adjacent relationship with the node to be decoded and has been decoded can be referred to as a node that has been decoded. It should be noted that the spatial adjacent relationships in the embodiments of the present application can be diverse, such as coplanar, colinear, co-point, etc. Therefore, there are many possible neighbor node types of the encoded neighbor nodes (or decoded neighbor nodes), which are not limited here.

[0186] It can be understood that, at the encoding end, when the node to be encoded is the first node in the first geometric structure of the point cloud data, the encoding device can determine the encoded neighboring nodes that have a spatially adjacent relationship with the node to be encoded; further, when the encoded neighboring node is the second node in the second geometric structure different from the first geometric structure, the encoding device can determine the target occupancy information of the encoded neighboring node, and then encode the node geometric information of the node to be encoded based on the target occupancy information, thereby obtaining a compressed code stream of the point cloud data (i.e., the aforementioned geometric code stream). The node geometric information here may include any one of the partition tree occupancy information and the prediction tree residual information associated with the structural type of the first geometric structure. Among them, the first geometric structure and the second geometric structure may have different structural types, and the specific structural types of the two are not limited here. For example, when the structural type of the first geometric structure is a partition tree type, the structural type of the second geometric structure is a prediction tree type, and the node geometric information of the node to be encoded at this time is the partition tree placeholder information; for another example, conversely, when the structural type of the first geometric structure is a prediction tree type, the structural type of the second geometric structure is a partition tree type, and the node geometric information of the node to be encoded at this time is the prediction tree residual information; wherein, the first node can be the node currently to be encoded in the first geometric structure, and accordingly, the second node can be a node that has a spatial adjacent relationship with the first node in the second geometric structure and has been encoded. It can be seen that in the process of encoding the geometric information of point cloud data, when the geometric structure where any encoded node to be encoded is located (i.e., the first geometric structure mentioned above) is different from the geometric structure where the corresponding encoded neighbor node is located (i.e., the second geometric structure mentioned above), the encoding device can make full use of the target occupancy information of the encoded neighbor node to encode the node geometric information of the node to be encoded, rather than only considering the neighbor nodes in the geometric structure of the same structural type or the relevant information within the node to be encoded. In this way, when the first geometric structure is a partitioning tree and the second geometric structure is a prediction tree, the encoding efficiency of the partitioning tree occupancy information can be improved, or when the first geometric structure is a prediction tree and the second geometric structure is a partitioning tree, the encoding efficiency of the prediction tree residual information can be improved, thereby improving the encoding efficiency of the geometric information as a whole.

[0187] Similarly, it can be understood that at the decoding end, the decoding device can obtain the compressed code stream of the point cloud data, and when the node to be decoded is the first node in the first geometric structure of the point cloud data, the decoded neighbor node that has a spatial adjacent relationship with the node to be decoded can be determined; further, when the decoded neighbor node is the second node in the second geometric structure different from the first geometric structure, the decoding device can determine the target occupancy information of the decoded neighbor node, and then can decode the node geometry information of the node to be decoded in the compressed code stream based on the target occupancy information; wherein, the node geometry information may include any one of the partitioning tree occupancy information and prediction tree residual information associated with the structural type of the first geometric structure. It can be seen that in the process of decoding the compressed code stream, when the geometric structure of any decoded node to be decoded (i.e., the first geometric structure mentioned above) is different from the geometric structure of the corresponding decoded neighbor node (i.e., the second geometric structure mentioned above), the decoding device can make full use of the target occupancy information of the decoded neighbor node to decode the node geometric information of the node to be decoded, rather than only considering the neighbor nodes in the geometric structure of the same structural type or the relevant information within the node to be decoded. In this way, when the first geometric structure is a partition tree and the second geometric structure is a prediction tree, the decoding efficiency of the partition tree occupancy information can be improved, or when the first geometric structure is a prediction tree and the second geometric structure is a partition tree, the decoding efficiency of the prediction tree residual information can be improved, thereby improving the decoding efficiency of the compressed code stream as a whole.

[0188] In summary, the point cloud geometry encoding and decoding optimization method based on a hybrid partitioning tree and a prediction tree provided in an embodiment of the present application can improve the placeholder code encoding efficiency of the partitioning tree based on the nodes encoded by the prediction tree; conversely, it can also improve the prediction efficiency of the prediction tree based on the occupied code situation of the partitioning tree.

[0189] It should be understood that the method provided in the embodiment of the present application can be applied to 3D point cloud encoding and decoding products, including but not limited to the server side (i.e., encoding device side), player side (i.e., decoding device side) and intermediate nodes (e.g., SMT (Smart Media Transport) receiving entity, SMT sending entity) of the point cloud media system. The specific process of geometric encoding optimization performed by the encoding device and the specific process of geometric decoding optimization performed by the decoding device can be found in the following. Figures 9-16 Description of the corresponding embodiment.

[0190] Further, see Figure 9 , Figure 9 This is a flow chart of a method for processing point cloud data provided by an embodiment of the present application. The method can be implemented by a decoding device in a point cloud media system (e.g., the above Figure 1The method may be performed by the decoding device 100B in the corresponding embodiment. For example, the decoding device may be a terminal integrated with a client (such as a video client). The method may include at least the following steps S101 to S104:

[0191] Step S101, obtaining a compressed code stream of point cloud data;

[0192] It can be understood that after encoding the point cloud data, the encoding device can obtain the corresponding point cloud code stream, and can encapsulate it into a point cloud file and send it to the decoding end for processing, wherein the point cloud code stream may include the compressed code stream (i.e., the geometric code stream) obtained after encoding the geometric information and the attribute code stream obtained after encoding the attribute information. Accordingly, after receiving the point cloud file, the decoding device can decapsulate it to obtain the corresponding point cloud code stream, and then obtain the compressed code stream when decoding the point cloud code stream. The specific process of the encoding device generating the compressed code stream can be found in the following Figure 16 In addition, it should be noted that since the embodiment of the present application focuses on optimizing the geometric encoding and decoding process, the specific process of attribute encoding and decoding will not be described in detail here.

[0193] Step S102, when the node to be decoded is the first node in the first geometric structure of the point cloud data, determining decoded neighbor nodes that have a spatial adjacent relationship with the node to be decoded;

[0194] For ease of understanding and distinction, the embodiment of the present application may refer to the geometric structure to be decoded as the first geometric structure, and accordingly, the node to be decoded in the first geometric structure may be referred to as the first node. The structural type of the first geometric structure will not be limited here, that is, the structural type of the first geometric structure may be either a partitioning tree type or a prediction tree type. Regardless of the structural type of the first geometric structure, when the node to be decoded is the first node in the first geometric structure, the first node as the node to be decoded can always be determined by the spatial coordinate position information of the point cloud data, and the determination process has been executed on the encoding device side, that is, when the encoding device encodes the point cloud data, it can pre-process the original coordinate position information of all point cloud data, and can construct the required partitioning tree or prediction tree based on the coordinate position information obtained after pre-processing, so that all relevant nodes including the first node can be determined (the specific process can be found in the above). Figure 2a description in the corresponding embodiment).

[0195] It can be understood that when the node to be decoded is the first node mentioned above, there may be multiple situations in which the decoded neighboring nodes have a spatial adjacent relationship with the node to be decoded. For example, from the perspective of spatial structure, the spatial adjacent relationship here may include one or more of the parent node layer adjacent relationship when it has a direct adjacent relationship with the parent node of the node to be decoded (which can also be understood as having an indirect adjacent relationship with the node to be decoded) and the child node layer adjacent relationship when it has a direct adjacent relationship with the node to be decoded (which can also be called the child node to be decoded). The position of the node to be decoded in its parent node is not limited here. For example, assuming the above Figure 4a The node shown is the parent node of the node to be decoded, then the node to be decoded can be located at any position from position 0 to position 7. The parent node layer adjacent relationship may include but is not limited to a first coplanar adjacent relationship that is coplanar with the parent node of the node to be decoded, a first colinear adjacent relationship that is colinear with the parent node of the node to be decoded, etc. Accordingly, the embodiment of the present application may collectively refer to nodes that have a parent node layer adjacent relationship with the node to be decoded and have been decoded as neighboring parent nodes. The child node layer adjacent relationship may include but is not limited to a second coplanar adjacent relationship that is coplanar with the node to be decoded, a second colinear adjacent relationship that is colinear with the node to be decoded, a co-point adjacent relationship that has a common point with the node to be decoded, and a close adjacent relationship that is a specified distance away from the node to be decoded in a specified direction, etc. Accordingly, the embodiment of the present application may collectively refer to nodes that have a child node layer adjacent relationship with the node to be decoded and have been decoded as neighboring child nodes. It can be understood that subsequent decoding devices can use the relevant information (such as placeholder information) of any one or more decoded neighbor nodes among these neighbor parent nodes and neighbor child nodes to decode the node to be decoded, and the structural type of the geometric structure where the decoded neighbor node is located is not limited here, that is, the decoded neighbor node can be either a partition tree node or a prediction tree node.

[0196] For easier understanding, please refer to Figure 10 , Figure 10 This is a schematic diagram of a parent node layer adjacent relationship provided by an embodiment of the present application. Figure 10 As shown, it is assumed that the middle black filling block is the parent node of the node to be decoded (i.e. Figure 10 The node with Morton number 13 in the middle) is centered on the parent node. This approach has a larger receptive field and can utilize up to 18 adjacent parent blocks (i.e., neighboring parent nodes) that have been decoded around it. Specifically, it can include 6 coplanar neighboring parent nodes (such as Figure 10 The node with Morton number 4 in the parent node) and the 12 collinear neighbor parent nodes that are collinear with the parent node (such as Figure 10It can be understood that the coplanar neighbor parent node and the parent node of the node to be decoded have the above-mentioned first coplanar neighbor relationship, while the collinear neighbor parent node and the parent node of the node to be decoded have the above-mentioned first collinear neighbor relationship. In practical applications, it can be obtained from Figure 10 The 18 neighbor parent nodes shown are used to determine the decoded neighbor nodes required for the current node to be decoded.

[0197] For easier understanding, please refer to Figure 11 , Figure 11 This is a schematic diagram of a sub-node layer adjacency relationship provided by an embodiment of the present application. Figure 11 As shown in the figure, taking the node with the shortest edge length in the x dimension as an example, assuming that the gray node in the figure is the current child node to be decoded (i.e., the node to be decoded), the dotted box node is the current node (i.e., the parent node of the node to be decoded), and the remaining solid box nodes are the neighboring child nodes selected by each child node to be decoded (also known as the reference node of the child node to be decoded). Figure 11 The example shows the neighbor subnodes of the subnode to be decoded (i.e., subnode 0-subnode 7) at each position under the octree breadth-first traversal division method. For example, the neighbor subnodes of the subnode to be decoded at each position can specifically include 3 coplanar neighbor subnodes in the left front and lower direction of the same layer as the subnode to be decoded, 3 collinear neighbor subnodes, 1 co-point neighbor subnode, and a close neighbor subnode at a distance of 2 subnode side lengths from the subnode to be decoded in the negative direction of the dimension with the shortest node side length (such as the x dimension). Among them, the left front and lower direction can be the negative direction of the coordinate axis of each dimension in the coordinate system where the node to be decoded is located. It can be understood that the coplanar neighbor subnode here has the above-mentioned second coplanar neighbor relationship with the subnode to be decoded, the collinear neighbor subnode has the above-mentioned second collinear neighbor relationship with the subnode to be decoded, the co-point neighbor subnode has the above-mentioned co-point neighbor relationship with the subnode to be decoded, and the close neighbor subnode has the above-mentioned close neighbor relationship with the subnode to be decoded. Optionally, the child node to be decoded may also have neighboring child nodes located at other positions, which are not listed here one by one.

[0198] It is understood that for any node to be decoded, the decoding device can determine, based on decoding requirements, decoded neighbor nodes that have a certain spatial proximity relationship with the node to be decoded. The number of decoded neighbor nodes here can be one or more, and the spatial proximity relationship between different decoded neighbor nodes and the node to be decoded can be any of the spatial proximity relationships listed above, without limitation here. Furthermore, the embodiments of the present application do not limit the structural type of the geometric structure in which the decoded neighbor nodes are located.

[0199] It should be noted that, taking the first node in the first geometric structure as node x, when the node x is a node in the partition tree (i.e., a partition tree node), since the partition tree node has a real parent node, child node, and other hierarchical divisions, the parent node of the node x can be understood as the polyhedron where the node x is located (see the above Figure 4a-4b When the node x is a node in the prediction tree (i.e., a prediction tree node), since the prediction tree node does not have a parent node, child node, or other hierarchical divisions, the parent node of the node x can be understood as the macroblock where the node x is located (see the above Figure 7a-7b description in the corresponding embodiment).

[0200] Step S103, when the decoded neighbor node is a second node in a second geometric structure different from the first geometric structure, determining target placeholder information of the decoded neighbor node;

[0201] As can be seen from the above step S102, for any node to be decoded, there may be multiple situations for the decoded neighbor nodes that have a certain spatial adjacent relationship with it, that is, the structural type of the geometric structure where the node to be decoded is located and the structural type of the geometric structure where its decoded neighbor nodes are located may be the same or different. For example, assuming that the node to be decoded is node A1, whether node A1 is a partition tree node or a prediction tree node, if the number of its decoded neighbor nodes is multiple (such as including node A2, node A3, etc.), then these multiple decoded neighbor nodes may only include one type of neighbor node, for example, node A2 and node A3 are both partition tree nodes or both prediction tree nodes; or, these multiple decoded neighbor nodes may include multiple types of neighbor nodes at the same time, for example, node A2 is a partition tree node, and node A3 is a prediction tree node. The embodiment of the present application mainly optimizes the decoding method when the structural type of the geometric structure where the node to be decoded is located is different from the structural type of the geometric structure where the decoded neighbor node is located. For the scenario where the structural type of the geometric structure where the node to be decoded is located is the same as the structural type of the geometric structure where the decoded neighbor node is located, the existing decoding method can still be used, and the embodiment of the present application will not elaborate on this. Based on this, taking a decoded neighbor node as an example, when the decoded neighbor node is a second node in a second geometric structure different from the first geometric structure, the decoding device can determine the target occupancy information of the decoded neighbor node to provide more effective information for the subsequent decoding of the node to be decoded. The second geometric structure here refers to a geometric structure of a different structural type from the first geometric structure, and accordingly, the second node refers to a node that has a certain spatial proximity to the first node in the second geometric structure and has been decoded.

[0202] It can be understood that the embodiment of the present application provides two different hybrid decoding optimization strategies for the specific structure types corresponding to the first geometric structure and the second geometric structure. These two hybrid decoding optimization strategies will be described below respectively.

[0203] Optionally, in one embodiment, when the structural type of the first geometric structure is a partitioning tree type, the first node as the node to be decoded is a node in the partitioning tree corresponding to the partitioning tree type (i.e., a partitioning tree node), and when the structural type of the second geometric structure is a prediction tree type, the second node as the decoded neighbor node is a node in the prediction tree corresponding to the prediction tree type (i.e., a prediction tree node). At this time, in the spatial adjacent relationship, the spatial adjacent relationship between the first node in the partitioning tree and the second node in the prediction tree can be used as the first spatial adjacent relationship, and the first spatial adjacent relationship here can include any one or more of the parent node layer adjacent relationship and the child node layer adjacent relationship exemplified above. When determining that the decoded neighbor node is the second node in the above-mentioned prediction tree, further, the decoding device can obtain a first hybrid decoding optimization strategy associated with the first node based on the first spatial decoding data information of the second node in the prediction tree, and then can determine the target placeholder information of the decoded neighbor node based on the first hybrid decoding optimization strategy. Among them, the first spatial decoding data information here may include some spatial relationships or related decoding information of the second node itself; since the prediction tree node itself does not have real placeholder code information (because the prediction tree node is not actually divided), the embodiment of the present application can infer or determine the placeholder status of the prediction tree node as a decoded neighbor node based on the first hybrid decoding optimization strategy; and the first hybrid decoding optimization strategy in the embodiment of the present application will correspond to different strategy contents for different situations (such as the multiple first spatial decoding data information possessed by the second node). These strategy contents can be used simultaneously or separately, and are not limited here.

[0204] For easier understanding, please refer to Figure 12 , Figure 12 This is a schematic diagram of a block-structured hybrid tree coding scenario provided by an embodiment of the present application. Figure 12 As shown, taking the partition tree as an octree as an example, the gray-filled blocks in the figure represent prediction tree nodes (such as node B), the colorless filled blocks represent partition tree nodes (such as node A), and the remaining unfilled parts are all empty (that is, the placeholder code is 0). Assuming that the current node to be decoded is node A, the decoding device can decode the placeholder code of node A (that is, the partition tree placeholder information) by looking up the placeholder information (that is, the target placeholder information) of the corresponding decoded neighbor node (for example, node B). In addition, optionally, the decoding device can determine the context model of node A based on the placeholder information of the decoded neighbor nodes to decode the placeholder code of node A.

[0205] Among them, optionally, the decoding device can determine the partition layer where the parent node of the node to be decoded is located as the parent node layer (such as the partition layer where the parent node of the above-mentioned node A is located), and can search for the neighbor parent node of the parent node of the node to be decoded in the parent node layer based on the first hybrid decoding optimization strategy; if there is a neighbor parent node located in the prediction tree among the neighbor parent nodes found, then the neighbor parent node located in the prediction tree can be used as the decoded neighbor node. At this time, the above-mentioned first spatial adjacent relationship can specifically include the parent node layer adjacent relationship between the decoded neighbor node and the node to be decoded (such as the above-mentioned first coplanar adjacent relationship, first collinear adjacent relationship, etc.); then, the value of the placeholder code of the decoded neighbor node can be set to the first code value (for example, 1), and the target placeholder information of the decoded neighbor node can be determined based on the placeholder code of the decoded neighbor node with the first code value. With the above Figure 12 Taking the node A shown as an example, the decoding device can search for the occupancy status of the neighboring parent node of the parent node of node A. If the neighboring parent node found is a prediction tree node (such as the parent node of the above-mentioned node B, the prediction tree node as the neighboring parent node here can be understood as the LCU level), then its occupancy status is 1.

[0206] Optionally, the decoding device can use the partition layer where the node to be decoded is located (such as the partition layer where the above-mentioned node A is located) as a child node layer, and then based on the first hybrid decoding optimization strategy, in the child node layer, search for the neighbor child nodes of the node to be decoded; if the parent node of the neighbor child node found (such as the above-mentioned node B) has a parent node located in the prediction tree, then when the parent node located in the prediction tree is used as the decoded neighbor node, the neighbor child node in the decoded neighbor node can be used as the target neighbor child node (such as the above-mentioned node B), and then the target placeholder information of the decoded neighbor node can be determined based on the placeholder code of the target neighbor child node. Figure 12 Taking node A shown as an example, the decoding device can search for the occupancy of node A's neighboring child nodes. Assuming that the partition depth of the current node A is D (D is a positive integer), when the parent node of some or all of node A's neighboring child nodes (such as node B mentioned above) is a prediction tree node, the occupancy of these neighboring child nodes in the prediction tree node can be obtained to parse the placeholder code of the current node A. For such situations, the first hybrid decoding optimization strategy provided by the embodiment of the present application includes but is not limited to the following:

[0207] (1) The placeholder code of the target neighbor child node may be a default code value, which may be a first code value (e.g., 1) or a second code value (e.g., 0). In other words, the decoding device may default to the placeholder code of the target neighbor child node, for example, directly defaulting the placeholder code of the aforementioned node B to 1 or 0.

[0208] (2) The placeholder code of the target neighbor subnode can be determined by the partition tree type adopted by the coding unit (such as a macroblock) to which the decoded neighbor node belongs. That is, when the block structure coding method is adopted, the decoding device can determine the placeholder code of the target neighbor subnode according to the block structure partitioning method. Optionally, when the partition depth D of the node to be decoded is in the first partition range, the partition tree type is determined by the first geometric partition strategy. The first geometric partition strategy here can include one of a binary tree partition strategy and a quadtree partition strategy for partitioning the point cloud bounding box. The partition tree type determined when the binary tree partition strategy is used to partition the point cloud bounding box is a binary tree partition type. Similarly, the partition tree type determined when the quadtree partition strategy is used to partition the point cloud bounding box is a quadtree partition type. Optionally, when the division depth is within the second division range, the division tree type is determined by a second geometric division strategy, where the second geometric division strategy may include one of a binary tree division strategy, a quadtree division strategy, and an octree division strategy for dividing the point cloud bounding box. The division tree type determined when the point cloud bounding box is divided using the octree division strategy is an octree division type. The point cloud bounding box is obtained by preprocessing the spatial coordinate position information of the point cloud data (see the above Figure 2a (See the relevant description in the corresponding embodiment). It can be understood that the above-mentioned first partition range is different from the second partition range. The specific partition range is not limited here. At the same time, the judgment condition between the placeholder code of the target neighbor child node and the partition tree type when the partition depth is D is also not limited. For example, if the partition tree type when the partition depth is D is a binary tree partition type, the value of the placeholder code of the target neighbor child node can be the first code value (such as 1); for another example, if the partition tree type when the partition depth is D is an octree partition type, the value of the placeholder code of the target neighbor child node can be the second code value (such as 0).

[0209] In some embodiments, different partitioning orders can be defined by defining two partitioning parameters, specifically including a first partitioning parameter K and a second partitioning parameter M, where 0≤K≤max(d x ,d y ,d z )-min(d x ,d y ,d z ), 0≤M≤min(d x ,d y ,d z), the specific values of the first partition parameter K and the second partition parameter M are not limited here. The first partition parameter K can be used to indicate the above-mentioned first partition range (i.e., 0 to K layers), that is, in the first K layers of partitioning, the first geometric partitioning strategy (such as binary tree partitioning strategy or quadtree partitioning strategy) can be adopted. The second partition parameter M can represent the minimum partitioning size corresponding to the quadtree partitioning strategy and the binary tree partitioning strategy, that is, the last M layers of partitioning must adopt the octree partitioning strategy. In other words, after the K layers (the above-mentioned second partitioning range), the second geometric partitioning strategy (such as binary tree partitioning strategy, quadtree partitioning strategy or octree partitioning strategy) can be adopted. Among them, when d x ≠d y ≠d z , the binary tree partitioning strategy may take precedence over the quadtree partitioning strategy. Optionally, in the first K-layer partitioning (i.e., when the partitioning depth D is within the first partitioning range), the first geometric partitioning strategy shown in Table 1 may be used for partitioning. After the K-layer partitioning (i.e., when the partitioning depth D is within the second partitioning range), the second geometric partitioning strategy shown in Table 2 may be used for partitioning. When the conditions in Tables 1 and 2 are not met, the octree partitioning strategy may be executed.

[0210] Table 1 Implicit geometric partitioning decision method (first K layers)

[0211] QT along the xy axis Along the xz axis QT QT along the yz axis condition <![CDATA[d z <d x =d y ]]> <![CDATA[d y <d x =d z ]]> <![CDATA[d x <d y =d z ]]> Along the x-axis BT Along the y-axis BT Along the z-axis BT condition <![CDATA[d y <d x andd z <d x ]]> <![CDATA[d x <d y andd z <d y ]]> <![CDATA[d x <d z andd y <d z ]]>

[0212] Table 2 Implicit geometric partitioning decision method (after K layers)

[0213]

[0214] As can be seen from Table 1 and Table 2 above, the corresponding geometric partitioning strategy can be executed only when the specified conditions are met. The embodiment of the present application can also support other different geometric partitioning strategies, which will not be described here.

[0215] (3) The placeholder code of the target neighbor subnode can be determined by the real placeholder code obtained by partitioning the decoded neighbor node based on the target partition tree type. In other words, the decoding device can partition the decoded neighbor node to obtain its real placeholder code. The target partition tree type here can include any one of an octree partition type, a quadtree partition type, and a binary tree partition type, which is not limited here. For example, the decoded neighbor node can be partitioned by an octree, and the obtained 8-bit real placeholder code can be used to determine the placeholder code of the target neighbor subnode, but the placeholder code does not need to be encoded or decoded.

[0216] (4) The placeholder code of the target neighbor subnode can be determined by the decoding information associated with the decoded neighbor node. In other words, the decoding device can determine the placeholder code of the target neighbor subnode based on the decoding information associated with the decoded neighbor node (which can be understood as some information related to the decoded neighbor node obtained by previous decoding), including but not limited to the following methods:

[0217] (a) The above-mentioned decoding information may include point cloud statistical information of the decoded neighbor node, and the placeholder code of the target neighbor child node may be determined by the point cloud statistical information and a statistical threshold associated with the point cloud statistical information, wherein the point cloud statistical information here may include one of the number of point cloud points in the decoded neighbor node and the point cloud density in the decoded neighbor node. That is, the decoding device can determine the placeholder code of the target neighbor subnode based on the point cloud statistical information of the decoded neighbor node. For example, when it is detected that the number of point cloud points in the decoded neighbor node is greater than the set first statistical threshold, the placeholder code of the target neighbor subnode can be determined to be a first code value (such as 1). Conversely, when it is detected that the number of point cloud points in the decoded neighbor node is less than or equal to the first statistical threshold, the placeholder code of the target neighbor subnode can be determined to be a second code value (such as 0). For another example, similarly, when it is detected that the point cloud density in the decoded neighbor node is greater than the set second statistical threshold, the placeholder code of the target neighbor subnode can be determined to be a first code value (such as 1). Conversely, when it is detected that the point cloud density in the decoded neighbor node is less than or equal to the second statistical threshold, the placeholder code of the target neighbor subnode can be determined to be a second code value (such as 0). The specific values of the first statistical threshold and the second statistical threshold are not limited here, and can be set as needed in actual applications.

[0218] (b) The above-mentioned decoding information may include the reconstruction point of the prediction tree where the decoded neighbor node is located. It can be understood that after the prediction tree is decoded, multiple corresponding reconstruction points can be obtained, such as the reconstruction point of the target neighbor sub-node. At this time, the decoding device can make a judgment based on the spatial distance between the reconstruction point of the prediction tree and the node to be decoded (or the reference neighbor sub-node of the node to be decoded) and the preset distance threshold, and the specific value of the distance threshold is not limited here. That is to say, when the spatial distance between the reconstruction point of the target neighbor sub-node and the node to be decoded is the target spatial distance, the placeholder code of the target neighbor sub-node is determined by the first code value obtained when the target spatial distance is less than the distance threshold. The embodiment of the present application provides two distance determination methods, which are explained below:

[0219] Optionally, taking the target neighbor child node in the prediction tree as an example, assume that the spatial distance between the reconstruction point of the target neighbor child node and the node to be decoded is the first target spatial distance (e.g., d1). If the reconstruction point found by the decoding device in the prediction tree, whose spatial distance from the node to be decoded is less than the distance threshold (e.g., the distance threshold is dis, and d1 < dis), is the reconstruction point of the target neighbor child node, then at this time, the occupancy code of the target neighbor child node can be determined by the first code value (such as 1) obtained when the first target spatial distance is less than the distance threshold. In the embodiments of the present application, since the specific position information of the node to be decoded is unknown, the geometric center point of the node to be decoded or the vertex of the point cloud bounding box where it is located, etc., can be used as the reference coordinate point (such as P1) of the node to be decoded. Then, the decoding device can search for the reconstruction points in the prediction tree. If there exists a certain reconstruction point P2 (such as the reconstruction point of the aforementioned target neighbor child node) whose spatial distance to P1 is less than the preset distance threshold, the occupancy code of the neighbor child node corresponding to the reconstruction point P2 can be determined as 1.

[0220] Alternatively, optionally, still taking the target neighbor child node in the prediction tree as an example, assume that the spatial distance between the reconstruction point of the target neighbor child node and the reference neighbor child node of the node to be decoded is the second target spatial distance (e.g., d2). If the reconstruction point found by the decoding device in the prediction tree, whose spatial distance from the reference neighbor child node is less than the distance threshold (e.g., the distance threshold is dis, and d2 < dis), is the reconstruction point of the target neighbor child node, then at this time, the occupancy code of the target neighbor child node can be determined by the first code value (such as 1) obtained when the second target spatial distance is less than the distance threshold. In the embodiments of the present application, the reference neighbor child node can specifically be any real neighbor child node adjacent to the node to be decoded (regardless of the structural type of the geometric structure where it is located), or a non-real neighbor child node (such as the center point of the point cloud bounding box) only used for distance determination here. Similarly, here, the geometric center point of the reference neighbor child node or the vertex or center point of the point cloud bounding box where it is located, etc., can be used as the reference coordinate point (such as P3) of the node to be decoded. Then, the decoding device can search for the reconstruction points in the prediction tree. If there exists a certain reconstruction point P2 (such as the reconstruction point of the aforementioned target neighbor child node) whose spatial distance to P3 is less than the preset distance threshold, the occupancy code of the neighbor child node corresponding to the reconstruction point P2 can be determined as 1.

[0221] It can be understood that when multiple (such as S, and S is a positive integer) neighbor child nodes all belong to the aforementioned decoded neighbor nodes (that is, when the number of target neighbor child nodes is S), S similar distance determinations can be made.

[0222] (c) The above-mentioned decoding information may include a placeholder code of a reference decoded neighbor node of the decoded neighbor node. The reference decoded neighbor node here may specifically be another decoded neighbor node of the decoded neighbor node. The reference decoded neighbor node may be in a partition tree or a prediction tree, which is not limited here. In this case, the placeholder code of the target neighbor child node may be determined by the placeholder code of the reference decoded neighbor node. For example, please refer again to Figure 12 , when node B is a decoded neighbor node of node A, assuming Figure 12 Node C in the example is a neighbor node of node B, and node C is a partition tree node. In this case, node C can be used as a reference decoded neighbor node of node B. The decoding device can infer the placeholder code of node B based on the placeholder code of node C. For example, when the placeholder code of node C is 1, the placeholder code of node B is also inferred to be 1.

[0223] (d) The above decoding information may include the child node placeholder code of the neighbor child node of the partition tree that has the same parent node as the node to be decoded. The neighbor child node of the partition tree here may be the neighbor child node that has been decoded in the partition tree. In this case, the placeholder code of the target neighbor child node can be determined by the child node placeholder code. For example, Figure 12 As shown, assuming that node D and node A have the same parent node (i.e., both are partition tree nodes), and node D is a neighbor child node of node A in the same child node layer, then node D can be a neighbor child node or a same-layer child node of node B. The decoding device can infer the placeholder status of the neighbor prediction tree node (such as node B) based on the placeholder code of the partition tree neighbor child node of node A (such as node D), which may include but is not limited to the following situations:

[0224] (d1) Optionally, when the above-mentioned child node placeholder code indicates that a partition tree neighbor child node (such as node D) located in the child node layer where the node to be decoded (such as node A) is located is in an occupied state, the target neighbor child node (such as node B) located in the same child node layer is also in an occupied state (or, when the partition tree neighbor child node is a neighbor child node of the target neighbor child node, it is predicted that the target neighbor child node is also in an occupied state), and the placeholder code of the target neighbor child node can be determined by the first code value (such as 1) obtained when the target neighbor child node is in an occupied state. That is, when the decoding device detects that only the child nodes of the same layer are occupied in the parent node of the node to be decoded, it can be inferred that only the child nodes of the same layer (such as the target neighbor child node) are occupied in the decoded neighbor nodes.

[0225] (d2) Optionally, the above partition tree neighbor child nodes may include target partition tree neighbor child nodes with the smallest distance (such as l2 distance (i.e., Euclidean distance), Manhattan distance, etc., which are not limited here) from the decoded neighbor nodes. At this time, the occupancy code of the target neighbor child node can be determined by the child node occupancy codes of the target partition tree neighbor child nodes. That is to say, the decoding device can make a determination based on the occupancy situation of the partition tree neighbor child node with the closest distance to the decoded neighbor node (i.e., the target partition tree neighbor child node).

[0226] (5) Optionally, the occupancy code of the target neighbor child node can be determined by the association relationship between the partition depth (i.e., the aforementioned partition depth D) where the node to be decoded is located and the depth threshold. Here, the association relationship may include one of the first association relationship when the partition depth is less than the depth threshold and the second association relationship when the partition depth is greater than the depth threshold. That is to say, the decoding device can determine the occupancy code of the target neighbor child node according to the partition depth D. For example, assuming the depth threshold is set to D1, the association relationship when D < D1 can be the first association relationship, and the association relationship when D > D1 can be the second association relationship. The first neighbor point search method determined based on the first association relationship and the second neighbor point search method determined based on the second association relationship can be the same or different. For example, the first neighbor point search method and the second neighbor point search method can be any one of the above-described contents (1)-(4) or other similar methods not exemplified here, which are not limited here.

[0227] In addition, in the embodiments of the present application, the context model for decoding the partition tree occupancy information of the node to be decoded is determined by the occupancy code information of the neighbor nodes. It can be understood that various context models (such as context models for sparse point clouds and context models for dense point clouds, etc.) are designed in the current point cloud coding framework to perform entropy coding and entropy decoding on the relevant occupancy codes. The definition of neighbor nodes (including neighbor parent nodes, neighbor child nodes, etc.) also needs to be specifically analyzed according to the different context models adopted. Here, the target neighbor child node is taken as an example for illustration: The neighbor node type of the target neighbor child node may include, but is not limited to, coplanar neighbor child nodes coplanar with the node to be decoded, collinear neighbor child nodes collinear with the node to be decoded, concurrent neighbor child nodes concurrent with the node to be decoded, and near neighbor child nodes at a distance of L child node side lengths in the negative direction of the target dimension of the coordinate system where the node to be decoded is located, where L is a positive integer greater than 1 (such as L = 2). Reference can be made to the above Figure 11Here, the target dimension can be any dimension of the coordinate system of the node to be decoded. For example, the target dimension can be the dimension with the shortest node side length among the three dimensions x, y, and z. This is not limited here. It is understood that when there are a large number of neighbor nodes, the placeholder codes of different neighbor nodes can be considered in combination or independently. This is not limited here.

[0228] Optionally, in another embodiment, when the structural type of the first geometric structure is a prediction tree type, the first node as the node to be decoded is a node in the first prediction tree corresponding to the prediction tree type (i.e., a prediction tree node), and when the structural type of the second geometric structure is a partitioning tree type, the second node as the decoded neighbor node is a node in the partitioning tree corresponding to the partitioning tree type (i.e., a partitioning tree node). At this time, in the spatial adjacent relationship, the spatial adjacent relationship between the first node in the first prediction tree and the second node in the partitioning tree can be used as a second spatial adjacent relationship. The second spatial adjacent relationship here can also include one or more of the parent node layer adjacent relationship and the child node layer adjacent relationship exemplified above. For details, please refer to the above explanation of the first spatial adjacent relationship, which will not be repeated here. When determining that the decoded neighbor node is the second node in the above partitioning tree, further, the decoding device can obtain a second hybrid decoding optimization strategy associated with the first node based on the second spatial decoding data information of the second node in the partitioning tree, and then can use the placeholder code of the decoded neighbor node as the target placeholder information of the decoded neighbor node based on the second hybrid decoding optimization strategy. Wherein, the second spatial decoding data information here can include decoding information related to the second node. It can be understood that since the partition tree node itself has real placeholder code information, the placeholder code of the decoded neighbor node can be directly used without performing a series of inferences like the prediction tree node.

[0229] For easier understanding, please refer to Figure 13 , Figure 13 This is a schematic diagram of a block-structured hybrid tree coding scenario provided by an embodiment of the present application. Figure 13As shown in the figure, the gray-filled blocks in the figure represent prediction tree nodes (such as nodes E and G), the non-colored filled blocks represent partition tree nodes (such as node F), and the remaining unfilled parts are empty (i.e., the placeholder code is 0). Assuming that the current node to be decoded is node E, the decoding device can decode the prediction tree residual information of node E based on the partition tree occupancy code information or prediction tree decoding information of the corresponding decoded neighbor nodes (such as nodes F and G). Here, taking the decoded neighbor node as a partition tree node (such as node F) as an example, when node F is the decoded neighbor node of the current node E, the decoding device can quickly obtain the reconstruction point determined by the placeholder code of node F, and then use the reconstruction point to decode node E. The specific process can be seen in the following step S104.

[0230] Step S104 , decoding the node geometry information of the node to be decoded in the compressed code stream based on the target occupancy information; the node geometry information includes any one of the partition tree occupancy information and the prediction tree residual information associated with the structure type of the first geometric structure.

[0231] It can be understood that after obtaining the placeholder information of the relevant neighboring nodes, the decoding device can use this placeholder information (including the above-mentioned target placeholder information and the placeholder information of other neighboring nodes located in the same geometric structure as the first geometric structure) to decode the node geometry information of the node to be decoded in the compressed code stream to achieve reconstruction of the geometry information of the node to be decoded. However, as can be seen from the above-mentioned step S103, when the structure type of the first geometric structure is a partition tree type or a prediction tree type, the target placeholder information obtained is different, and the node geometry information used for decoding is also different. Therefore, the relevant decoding processes of the partition tree placeholder information and the prediction tree residual information code will be explained below.

[0232] Optionally, in one embodiment, when the node to be decoded is a partition tree node and the decoded neighbor node is a prediction tree node, the target placeholder information of the decoded neighbor node can be inferred through the above-mentioned step S103, and the partition tree placeholder information of the node to be decoded can be decoded in combination with the placeholder information of other decoded partition tree neighbor nodes. Optionally, the embodiment of the present application can use a context model to decode the partition tree placeholder information, that is, after the decoding device obtains the target placeholder information of the decoded neighbor node, it can determine the target context model associated with the partition tree placeholder information of the node to be decoded based on the target placeholder information and the placeholder information of other decoded partition tree neighbor nodes, and then entropy decoding can be performed on the partition tree placeholder information based on the target context model. Subsequently, the placeholder code obtained after entropy decoding can be used to reconstruct the coordinate position information of the node to be decoded.

[0233] To facilitate understanding, the embodiments of the present application will briefly introduce two methods for determining context models (including a first context model determination method and a second context model determination method). The above-mentioned target context model can be a context model determined by the first context model determination method, or a context model determined by the second context model determination method. Other context models can also be used, which are not limited here.

[0234] For example, the first context model determination method may include neighbor predictions at the child node layer and neighbor predictions at the parent node layer. For the neighbor predictions at the child node layer, under the octree breadth-first traversal division method, the neighbor reference information (i.e., placeholder information) that can be obtained when decoding the child node to be decoded of the current point (i.e., the parent node of the child node to be decoded) may specifically include the placeholder information of the neighbor child nodes in the three directions of left, front, and bottom (i.e., the negative direction of each coordinate axis). As mentioned above Figure 11 As shown, the context model design of the sub-node layer can be as follows: For the sub-node layer where the sub-node to be decoded is located, the occupancy of 3 coplanar neighbor sub-nodes, 3 collinear neighbor sub-nodes, 1 co-point neighbor sub-node in the left front and lower direction of the same layer as the sub-node to be decoded, and the close neighbor sub-nodes at the negative direction of the dimension with the shortest node side length (such as the x dimension) with two sub-node side lengths away from the current sub-node to be decoded can be searched. Among them, the occupancy of the 3 coplanar neighbor sub-nodes, 3 collinear neighbor sub-nodes, and the close neighbor sub-nodes at the negative direction of the dimension with the shortest node side length with two sub-node side lengths away from the current sub-node to be decoded can be given priority for detailed consideration. It can be understood that the occupancy of these 7 nodes is 2 7 = 128 cases. If not all are unoccupied, there are 2 7 -1 = 127 possible cases, and one context can be assigned to each. If all seven nodes are unoccupied, the occupancy of the shared neighboring child nodes is further considered. This shared neighboring child node has two possibilities: occupied or unoccupied. Similarly, a separate context can be assigned to the occupied shared neighboring child node. If this shared neighboring child node is also unoccupied, the occupancy of the parent node's neighbors, as described below, is considered. Thus, the shared neighboring child nodes of the child node to be decoded correspond to a total of 127 + 2 - 1 = 128 contexts.

[0235] For the neighbor prediction of the parent node layer, if the above 8 neighbor sub-nodes of the child node to be decoded are not occupied, you can refer to Figure 14 , Figure 14 This is a schematic diagram of a parent node layer adjacency relationship provided by an embodiment of the present application. Figure 14 The four groups of neighbor parent nodes shown can be understood as the above Figure 10 The subset of 18 neighbor parent nodes shown in . Figure 14The figure shows the occupancy of the four groups of neighboring parent nodes in the parent node layer where the parent node of the child node to be decoded is located. The dotted frame node is the parent node of the child node to be decoded (also known as the current node), and the solid frame node is the neighboring parent node. The upper right rear direction shown in the figure is the positive direction of each coordinate axis. For the parent node layer (also known as the current node layer), the three coplanar neighboring parent nodes of the upper right rear of the parent node are first considered. There are 2 occupancy situations of the three coplanar neighboring parent nodes of the upper right rear of the parent node. 3 = 8 possibilities, assign a context to each of the cases where not all are not occupied, and then consider the position of the child node to be decoded at the parent node (as mentioned above Figure 4a ), this group of coplanar neighbor parent nodes provides a total of (8-1) × 8 = 56 contexts. If none of the three coplanar neighbor parent nodes immediately above and to the right of the parent node are occupied, then the remaining three groups of neighbor parent nodes at the parent node level are considered. Specifically, the node distance between the most recently occupied neighbor parent node and the parent node can be considered. The specific correspondence between the distribution of neighbor parent nodes and node distance is shown in Table 3.

[0236] Table 3

[0237] Parent node layer occupancy Node distance Occupancy by the left front and lower coplanar neighbors or the right upper and rear collinear neighbors 1 The left front and lower coplanar neighbors and the right upper and rear collinear neighbors are not occupied and the left front and lower collinear neighbors are occupied 2 None of the four neighbor groups at the current node level are occupied 3

[0238] As can be seen from Table 3, there are three possible values for node distance, and one context can be assigned to each of these three values. Considering the position of the child node to be decoded at its parent node, there are a total of 3×8=24 contexts. At this point, the first context model determination method has assigned a total of 128+56+24=208 context models. In this way, the decoding device can determine the target model index based on the placeholder codes of the neighboring child nodes of the child node to be decoded at the child node layer and the placeholder codes of the neighboring parent nodes at the parent node layer (which may include the target placeholder information of the prediction tree node as the decoded neighbor node and the placeholder information of other decoded partition tree neighbor nodes). Therefore, based on the target model index, the target context model of the child node to be decoded can be determined from the above 208 context models.

[0239] For another example, the second context model determination method can use a two-layer context reference relationship configuration, as shown in formula (14), the first layer is the occupancy of the decoded adjacent blocks (i.e., neighboring parent nodes) adjacent to the parent node of the current child node to be decoded (i.e., ctxIdxParent), and the second layer is the occupancy of the adjacent decoded blocks (i.e., neighboring child nodes) at the same division depth as the child node to be decoded (i.e., ctxIdxChild).

[0240] idx=LUT[ctxIdxParent][ctxIdxChild] (14)

[0241]

[0242]

[0243] First, for each child node to be decoded, the ctxIdxChild of the second layer is as shown in formula (15), Indicates the three neighboring child nodes with a l2 distance (i.e., Euclidean distance) of 1 from the child node to be decoded (as mentioned above Figure 11 Secondly, for the first layer’s ctxIdxParent, for the relative positions of different child nodes to be decoded, we can find the neighboring parent nodes that are coplanar and colinear with them by looking up the table, and calculate ctxIdxParent according to formula (16) based on their occupancy. Figure 15 , Figure 15 This is a schematic diagram of a parent node layer adjacent relationship provided by an embodiment of the present application. Figure 15 As shown, each subgraph shows the relative position relationship of the 6 neighbor parent nodes found by the i-th child node, including 3 coplanar neighbor parent nodes (P i,0 ,P i,1 ,P i,2 ) and 3 collinear neighbor parent nodes (P i,3 ,P i,4 ,P i,5 The positional relationship between each child node and its neighboring parent node can be obtained by the above Table 3. The following Table 4 describes the relationship between child node i and its neighboring parent node j. The numbers in Table 4 correspond to Figure 10 The Morton number in the method takes into account the different child node positions and the geometric center rotation symmetry. The method used in formula (16) is the combination of the occupancy of the three coplanar neighbor parent nodes and the sum of the number of occupancy of the three collinear neighbor parent nodes. Therefore, the maximum number of context models used in this method is 2 3 ×2 5 = 256. Similarly, in this manner, the decoding device can determine the target model index based on the placeholder codes of the neighboring child nodes of the child node to be decoded at the child node layer and the placeholder codes of the neighboring parent nodes at the parent node layer (which may include the target placeholder information of the prediction tree node as the decoded neighbor node and the placeholder information of other decoded partition tree neighbor nodes), so as to determine the target context model of the child node to be decoded in the above 256 context models based on the target model index.

[0244] Table 4

[0245]

[0246]

[0247] It can be understood that the decoding device can use the target context model determined by any of the above methods to perform entropy decoding on the partition tree occupancy information of the child node to be decoded, thereby saving bit rate and improving decoding efficiency.

[0248] Optionally, in another embodiment, when the node to be decoded is a prediction tree node and the decoded neighbor node is a partition tree node, the target occupancy information of the decoded neighbor node obtained in step S103 above can be used, and combined with the decoding information of other decoded prediction tree neighbor nodes (such as the coordinate position information of the reconstruction point), to decode the prediction tree residual information of the node to be decoded. Specifically, the target occupancy information of the decoded neighbor node can be used to determine the reconstruction point of the decoded neighbor node. After obtaining the reconstruction point of the decoded neighbor node, the decoding device can use the reconstruction point as the starting point (i.e., the root node) of the second prediction tree corresponding to the node to be decoded or an independent reference point of the node to be decoded. That is to say, optionally, the original first prediction tree can be reconstructed by the reconstruction point of the decoded neighbor node, so that a second prediction tree with the reconstruction point as the starting point can be obtained; or, optionally, the reconstruction point of the decoded neighbor node can be directly used as an independent reference point independent of the first prediction tree. Furthermore, the decoding device can use the first prediction reference point or the independent reference point in the second prediction tree as the target reference point, and can decode the prediction tree residual information of the node to be decoded based on the reconstructed coordinate position information of the target reference point, and then use the decoded prediction tree residual information to reconstruct the coordinate position information of the node to be decoded. Among them, the embodiment of the present application does not limit the prediction mode adopted by the encoding end, and the structure of the second prediction tree can be determined by the adopted prediction mode. Therefore, accordingly, the reconstructed second prediction tree may be Figure 7a The single chain structure shown in FIG, then the first prediction reference point at this time can be the previous node of the node to be decoded in the second prediction tree; or, optionally, the reconstructed second prediction tree may also be Figure 7b In the multi-chain structure shown, the first prediction reference point at this time can be the previous node A1 (i.e., the parent point) of the node to be decoded in the second prediction tree, or node A1 and the previous node A2 of node A1 (i.e., the grandparent point), or node A1, node A2 and the previous node A3 of node A2 (i.e., the grand-grandparent point), which is not limited in this embodiment of the present application.

[0249] In addition, it can be understood that when the structure type of the first geometric structure is a prediction tree type, the first node to be decoded is a node in the first prediction tree corresponding to the prediction tree type (such as the above Figure 13), and when the decoded neighbor node is the third node in the third geometric structure having the same structural type as the first geometric structure, the third node as the decoded neighbor node is a node in the third prediction tree corresponding to the prediction tree type (such as the above Figure 13 ), that is, at this time, the node to be decoded and the decoded neighboring node are both prediction tree nodes, and the first prediction tree and the third prediction tree here can be the same prediction tree or different prediction trees, which is not limited here. In this case, the decoding device can determine the target reference point based on the starting point of the third prediction tree (such as the starting point of the prediction tree where the above-mentioned node G is located) or the ending point of the third prediction tree (such as the ending point of the prediction tree where the above-mentioned node G is located), and then can decode the prediction tree residual information of the node to be decoded based on the reconstructed coordinate position information of the target reference point.

[0250] Optionally, when the starting point of the third prediction tree or the ending point of the third prediction tree is used as the target starting point, the decoding device can reconstruct the original first prediction tree using the target starting point, thereby obtaining a reconstructed prediction tree corresponding to the node to be decoded. It can be understood that the starting point of the reconstructed prediction tree is the target starting point. Further, the decoding device can use the second prediction reference point in the reconstructed prediction tree as the target reference point. The selection of the second prediction reference point here can depend on the prediction mode adopted by the encoding end. For details, please refer to the above description of the first prediction reference point, which will not be repeated here.

[0251] Alternatively, optionally, the starting point of the third prediction tree or the ending point of the third prediction tree may be directly used as the target reference point. In this case, the target reference point is independent of the first prediction tree.

[0252] As can be seen from the above, when the node to be decoded is a prediction tree node (as mentioned above Figure 13 When the node E in the decoded node is decoded, its neighboring nodes may contain prediction tree nodes (such as the above Figure 13 Node G in the partition tree (as mentioned above Figure 13 Node F in the embodiment of the present application has been described in the previous text for both cases. It can be understood that since the decoded neighbor nodes can include prediction tree nodes and partition tree nodes at the same time, the relevant hybrid decoding optimization strategies described in the previous text can be used simultaneously or separately, and are not limited here.

[0253] Among them, optionally, regarding how to determine the target reference point among the above-mentioned first prediction reference point, independent reference point, or second prediction reference point, the starting point of the third prediction tree and the ending point of the third prediction tree, the embodiment of the present application can also add corresponding reference mode selection information in the compressed code stream to indicate, so that the decoding device can determine the required target reference point by parsing the reference mode selection information.

[0254] It can be seen that in the process of decoding the compressed code stream, when the geometric structure of any decoded node to be decoded (i.e., the first geometric structure mentioned above) is different from the geometric structure of the corresponding decoded neighbor node (i.e., the second geometric structure mentioned above), the decoding device can make full use of the target occupancy information of the decoded neighbor node to decode the node geometric information of the node to be decoded, rather than only considering the neighbor nodes in the geometric structure of the same structural type or the relevant information within the node to be decoded. In this way, when the first geometric structure is a partition tree and the second geometric structure is a prediction tree, the decoding efficiency of the partition tree occupancy information can be improved, or when the first geometric structure is a prediction tree and the second geometric structure is a partition tree, the decoding efficiency of the prediction tree residual information can be improved, thereby improving the overall decoding efficiency of the compressed code stream. In addition, for the case where the node to be decoded and the decoded neighbor node are both prediction tree nodes, the embodiment of the present application has also been expanded to enrich the decoding method of the prediction tree residual information and improve the decoding efficiency of the prediction tree residual information.

[0255] Further, see Figure 16 , Figure 16 This is a flow chart of a point cloud data processing method provided by an embodiment of the present application. The method can be implemented by an encoding device in a point cloud media system (e.g., the above Figure 1 The method may be performed by the encoding device 100A in the corresponding embodiment, for example, the encoding device may be a server. The method may at least include the following steps S201 to S203:

[0256] Step S201: When the node to be encoded is the first node in the first geometric structure of the point cloud data, determining an encoded neighboring node that has a spatial adjacent relationship with the node to be encoded;

[0257] Specifically, after acquiring the point cloud data, the encoding device can encode its geometric information and attribute information separately. It is understandable that the encoding device can use a block structure encoding method to encode the geometric information, dividing the three-dimensional space into multiple macroblocks, and for each macroblock, different geometric encoding methods can be used according to its characteristics. For example, geometric encoding based on a partitioning tree or geometric encoding based on a prediction tree can be used. During the encoding process, the encoding device will follow a certain encoding order. Assuming that any node to be encoded currently encoded is the first node in the first geometric structure of the point cloud data, the encoding device can first determine the encoded neighboring nodes that have a spatially adjacent relationship with the node to be encoded, so that some information of the encoded neighboring nodes can be used to encode the node to be encoded later. The spatially adjacent relationship here can include one or more of a parent node-level adjacent relationship (also understood as an indirect adjacent relationship with the node to be encoded) when the node to be encoded has a direct adjacent relationship with the parent node of the node to be encoded (also referred to as the current node) and a child node-level adjacent relationship when the node to be encoded (also referred to as the child node to be encoded) has a direct adjacent relationship.

[0258] It is understandable that since the decoding process is the inverse of the encoding process, the specific process of this step and the explanation of the relevant terms can be found in the above Figure 9 Step S102 in the corresponding embodiment will not be described in detail here.

[0259] Step S202 , when the encoded neighbor node is a second node in a second geometric structure different from the first geometric structure, determining target placeholder information of the encoded neighbor node;

[0260] It can be understood that when the encoded neighbor node is a second node in a second geometric structure different from the first geometric structure, the embodiment of the present application can optimize the geometric coding based on the hybrid tree.

[0261] For example, optionally, in one embodiment, when the structural type of the first geometric structure is a partition tree type, the first node as the node to be encoded is a node in the partition tree corresponding to the partition tree type, and when the structural type of the second geometric structure is a prediction tree type, the second node as the encoded neighbor node is a node in the prediction tree corresponding to the prediction tree type; based on this, in the spatial neighbor relationship, the spatial neighbor relationship between the first node located in the partition tree and the second node located in the prediction tree can be used as the first spatial neighbor relationship; the encoding device can obtain the first hybrid coding optimization strategy associated with the first node based on the first spatial coding data information of the second node located in the prediction tree, and can determine the target placeholder information of the encoded neighbor node based on the first hybrid coding optimization strategy. The first spatial coding data information here may include some spatial relationships or related coding information of the second node itself.

[0262] Among them, optionally, the encoding device can determine the division layer where the parent node of the node to be encoded is located as the parent node layer, and can search for the neighbor parent node of the parent node of the node to be encoded in the parent node layer based on the first hybrid coding optimization strategy; if there is a neighbor parent node located in the prediction tree among the neighbor parent nodes found, the neighbor parent node located in the prediction tree can be used as the encoded neighbor node; then, the value of the placeholder code of the encoded neighbor node can be set to the first code value (for example, 1), and the target placeholder information of the encoded neighbor node can be determined based on the placeholder code of the encoded neighbor node with the first code value.

[0263] Among them, optionally, the decoding device can use the division layer where the node to be encoded is located as a child node layer, and then based on the first hybrid coding optimization strategy, search for the neighbor child nodes of the node to be encoded in the child node layer; if the parent node of the neighbor child node found contains a parent node located in the prediction tree, then when the parent node located in the prediction tree is used as the encoded neighbor node, the neighbor child node in the encoded neighbor node can be used as the target neighbor child node, and then the target placeholder information of the encoded neighbor node can be determined based on the placeholder code of the target neighbor child node.

[0264] Optionally, the value of the placeholder code of the target neighbor subnode may be a default code value, and the default code value may be a first code value (such as 1) or a second code value (such as 0).

[0265] Optionally, the placeholder code of the target neighbor subnode can be determined by the partition tree type adopted by the coding unit (such as a macroblock) to which the encoded neighbor node belongs. When the partition depth D of the node to be encoded is within the first partition range, the partition tree type is determined by the first geometric partition strategy; the first geometric partition strategy includes one of a binary tree partition strategy and a quadtree partition strategy for partitioning the point cloud bounding box; when the partition depth is within the second partition range, the partition tree type is determined by the second geometric partition strategy; the second geometric partition strategy includes one of a binary tree partition strategy, a quadtree partition strategy, and an octree partition strategy for partitioning the point cloud bounding box; the first partition range is different from the second partition range; the point cloud bounding box is obtained after preprocessing the spatial coordinate position information of the point cloud data.

[0266] Among them, optionally, the placeholder code of the target neighbor child node can be determined by the real placeholder code obtained after dividing the encoded neighbor node based on the target partition tree type; the target partition tree type can include one of an octree partition type, a quadtree partition type and a binary tree partition type.

[0267] Optionally, the placeholder code of the target neighbor subnode may be determined by coding information associated with the coded neighbor node.

[0268] Among them, optionally, the above-mentioned encoding information may include point cloud statistical information of the encoded neighbor node; the placeholder code of the target neighbor child node may be determined by the point cloud statistical information and a statistical threshold associated with the point cloud statistical information; the point cloud statistical information may include one of the number of point cloud points in the encoded neighbor node and the point cloud density in the encoded neighbor node.

[0269] Optionally, the above-mentioned coding information may include a reconstruction point of the prediction tree where the encoded neighbor node is located. It can be understood that after the prediction tree is reconstructed, multiple corresponding reconstruction points can be obtained, such as the reconstruction point of the target neighbor subnode. At this time, the coding device can make a judgment based on the spatial distance between the reconstruction point of the prediction tree and the node to be encoded (or the reference neighbor subnode of the node to be encoded) and a preset distance threshold, and the specific value of the distance threshold is not limited here. In other words, when the spatial distance between the reconstruction point of the target neighbor subnode and the node to be encoded is the target spatial distance, the placeholder code of the target neighbor subnode is determined by the first code value (such as 1) obtained when the target spatial distance is less than the distance threshold.

[0270] Optionally, the above-mentioned coding information may include a placeholder code of a reference coded neighbor node of the coded neighbor node; the placeholder code of the target neighbor child node may be determined by the placeholder code of the reference coded neighbor node.

[0271] Among them, optionally, the above-mentioned encoding information may include the child node placeholder code of the partition tree neighbor child node that has the same parent node as the node to be encoded; the partition tree neighbor child node is the neighbor child node that has been encoded in the partition tree; the placeholder code of the target neighbor child node can be determined by the child node placeholder code.

[0272] Among them, optionally, when the sub-node placeholder code indicates that the partition tree neighbor sub-node located in the sub-node layer where the node to be encoded is located is in an occupied state, the target neighbor sub-node located in the same sub-node layer is also in an occupied state, and the placeholder code of the target neighbor sub-node can be determined by the first code value (such as 1) obtained when the target neighbor sub-node is in an occupied state.

[0273] Among them, optionally, the partition tree neighbor sub-node may include the target partition tree neighbor sub-node with the smallest distance (such as Euclidean distance, Manhattan distance, etc., not limited here) to the encoded neighbor node; the placeholder code of the target neighbor sub-node can be determined by the child node placeholder code of the target partition tree neighbor sub-node.

[0274] Among them, optionally, the placeholder code of the target neighbor sub-node can be determined by the association relationship between the division depth of the node to be encoded and the depth threshold; the association relationship can include one of a first association relationship when the division depth is less than the depth threshold and a second association relationship when the division depth is greater than the depth threshold.

[0275] Among them, optionally, the neighbor node type of the target neighbor sub-node may include one or more of coplanar neighbor sub-nodes that are coplanar with the node to be encoded, colinear neighbor sub-nodes that are colinear with the node to be encoded, co-point neighbor sub-nodes that are co-point with the node to be encoded, and close neighbor sub-nodes that are L sub-node side lengths away from the node to be encoded in the negative direction of the target dimension of the coordinate system where the node to be encoded is located; L is a positive integer greater than 1.

[0276] Similarly, optionally, in one embodiment, when the structural type of the first geometric structure is a prediction tree type, the first node as the node to be encoded is a node in the first prediction tree corresponding to the prediction tree type, and when the structural type of the second geometric structure is a partitioning tree type, the second node as the encoded neighbor node is a node in the partitioning tree corresponding to the partitioning tree type; based on this, in the spatial neighbor relationship, the spatial neighbor relationship between the first node located in the first prediction tree and the second node located in the partitioning tree can be used as the second spatial neighbor relationship. The encoding device can obtain a second hybrid coding optimization strategy associated with the first node based on the second spatial coding data information of the second node located in the partitioning tree, and can use the placeholder code of the encoded neighbor node as the target placeholder information of the encoded neighbor node based on the second hybrid coding optimization strategy. The second spatial coding data information here can include coding information related to the second node.

[0277] It is understandable that since the decoding process is the inverse of the encoding process, the specific process of this step and the explanation of the relevant terms can be found in the above Figure 9 Step S103 in the corresponding embodiment will not be described in detail here.

[0278] Step S203, encoding the node geometry information of the node to be encoded based on the target occupancy information to obtain a compressed code stream of the point cloud data; the node geometry information includes any one of the partitioning tree occupancy information and the prediction tree residual information associated with the structure type of the first geometric structure.

[0279] It is understandable that after obtaining the above-mentioned target occupancy information, the encoding device can use the target occupancy information to encode the node geometry information of the node to be encoded, so as to achieve compression of the geometry information, thereby obtaining a compressed code stream of the point cloud data.

[0280] Optionally, in one embodiment, when the node to be encoded is a partition tree node and the encoded neighbor node is a prediction tree node, the target occupancy information of the encoded neighbor node can be inferred through the above-mentioned step S202, and the partition tree occupancy information of the node to be encoded can be encoded in combination with the occupancy information of other encoded partition tree neighbor nodes. For example, the encoding device can determine the target context model associated with the partition tree occupancy information of the node to be encoded based on the acquired target occupancy information of the encoded neighbor node and the occupancy information of other decoded partition tree neighbor nodes, and then can perform entropy encoding on the partition tree occupancy information based on the target context model to obtain a corresponding compressed code stream. The target context model here includes but is not limited to the context model determined by the aforementioned first context model determination method or the second context model determination method.

[0281] Optionally, in another embodiment, when the node to be encoded is a prediction tree node and the encoded neighbor node is a partition tree node, the target occupancy information of the encoded neighbor node obtained in the above step S202 can be combined with the encoding information of other decoded prediction tree neighbor nodes (such as the coordinate position information of the reconstructed point) to encode the prediction tree residual information of the node to be encoded, thereby obtaining a compressed code stream of the point cloud data.

[0282] For example, combining the above Figure 9 In the corresponding embodiment, step S104, in some embodiments, the target occupancy information can be used to determine the reconstruction point of the encoded neighboring node, and the reconstruction point can be used as the starting point of the second prediction tree corresponding to the node to be encoded or the independent reference point of the node to be encoded. The second prediction tree here is determined by reconstructing the first prediction tree through the reconstruction point of the encoded neighboring node, that is, the starting point of the second prediction tree is the reconstruction point; in addition, the independent reference point is independent of the first prediction tree. Based on this, the encoding device can use the first prediction reference point or the independent reference point in the second prediction tree as the target reference point, and then can perform prediction based on the reconstructed coordinate position information of the target reference point to obtain the predicted coordinate position information; then, the encoding device can determine the prediction tree residual information of the node to be encoded based on the predicted coordinate position information and the original coordinate position information of the node to be encoded, and can use the obtained prediction tree residual information as the node geometry information of the node to be encoded, and then can encode the node geometry information to obtain a compressed code stream of the point cloud data.

[0283] In addition, in some embodiments, when the structural type of the first geometric structure is a prediction tree type, the first node as the node to be encoded is a node in the first prediction tree corresponding to the prediction tree type, and when the encoded neighbor node is the third node in the third geometric structure having the same structural type as the first geometric structure, the third node as the encoded neighbor node is a node in the third prediction tree corresponding to the prediction tree type; the encoding device can determine the target reference point based on the starting point of the third prediction tree or the ending point of the third prediction tree, and then can perform prediction based on the reconstructed coordinate position information of the target reference point to obtain predicted coordinate position information; subsequently, the encoding device can determine the prediction tree residual information of the node to be encoded based on the predicted coordinate position information and the original coordinate position information of the node to be encoded, and can use the obtained prediction tree residual information as the node geometric information of the node to be encoded, and encode the node geometric information to obtain a compressed code stream of the point cloud data.

[0284] Optionally, the process of determining the target reference point based on the starting point of the third prediction tree or the ending point of the third prediction tree may be as follows: when the starting point of the third prediction tree or the ending point of the third prediction tree is used as the target starting point, the encoding device may reconstruct the first prediction tree using the target starting point, thereby obtaining a reconstructed prediction tree corresponding to the node to be encoded. It can be understood that the starting point of the reconstructed prediction tree is the target starting point. Further, the encoding device may use the second prediction reference point in the reconstructed prediction tree as the target reference point.

[0285] Alternatively, optionally, the starting point of the third prediction tree or the ending point of the third prediction tree may be directly used as the target reference point. In this case, the target reference point is independent of the first prediction tree.

[0286] It is understandable that since the decoding process is the inverse of the encoding process, the specific process of this step and the explanation of the relevant terms can be found in the above Figure 9 Step S104 in the corresponding embodiment will not be described in detail here.

[0287] It can be seen that in the process of encoding the geometric information of point cloud data, when the geometric structure where any encoded node to be encoded is located (i.e., the first geometric structure mentioned above) is different from the geometric structure where the corresponding encoded neighbor node is located (i.e., the second geometric structure mentioned above), the encoding device can make full use of the target occupancy information of the encoded neighbor node to encode the node geometric information of the node to be encoded, rather than only considering the neighbor nodes in the geometric structure of the same structural type or the relevant information within the node to be encoded. In this way, when the first geometric structure is a partitioning tree and the second geometric structure is a prediction tree, the encoding efficiency of the partitioning tree occupancy information can be improved, or when the first geometric structure is a prediction tree and the second geometric structure is a partitioning tree, the encoding efficiency of the prediction tree residual information can be improved, thereby improving the encoding efficiency of the geometric information as a whole.

[0288] See Figure 17 , is a schematic diagram of the structure of a point cloud data processing device provided in an embodiment of the present application. Figure 17 As shown, the point cloud data processing device 1 can be a computer program (including program code) running on a decoding device (for example, the above-mentioned decoding device 100B). For example, the point cloud data processing device 1 is an application software. It can be understood that the point cloud data processing device 1 can be used to execute the corresponding steps of the point cloud data processing method provided in the embodiment of the present application. Figure 17 As shown, the point cloud data processing device 1 may include: a code stream acquisition module 11, a neighbor determination module 12, an occupancy determination module 13, a geometry decoding module 14, and a residual decoding module 15;

[0289] The code stream acquisition module 11 is used to obtain the compressed code stream of the point cloud data;

[0290] a neighbor determination module 12 for determining, when the node to be decoded is the first node in the first geometric structure of the point cloud data, decoded neighbor nodes that have a spatially adjacent relationship with the node to be decoded;

[0291] an occupancy determination module 13, configured to determine target occupancy information of the decoded neighbor node when the decoded neighbor node is a second node in a second geometric structure different from the first geometric structure;

[0292] Wherein, when the structure type of the first geometric structure is a partition tree type, the first node as the node to be decoded is a node in the partition tree corresponding to the partition tree type, and when the structure type of the second geometric structure is a prediction tree type, the second node as the decoded neighbor node is a node in the prediction tree corresponding to the prediction tree type;

[0293] The placeholder determination module 13 may include: a first strategy acquisition unit 131, a first placeholder determination unit 132;

[0294] A first strategy acquisition unit 131 is configured to acquire a first hybrid decoding optimization strategy associated with the first node based on first spatial decoding data information of the second node located in the prediction tree;

[0295] The first placeholder determining unit 132 is configured to determine target placeholder information of decoded neighboring nodes based on a first hybrid decoding optimization strategy.

[0296] The first placeholder determination unit 132 may include: a parent node search subunit 1321, a first placeholder determination subunit 1322, a child node search subunit 1323, and a second placeholder determination subunit 1324;

[0297] The parent node search sub-unit 1321 is configured to determine the partition layer where the parent node of the node to be decoded is located as the parent node layer, and based on the first hybrid decoding optimization strategy, search for the neighboring parent nodes of the parent node in the parent node layer; if the neighboring parent nodes found exist in the prediction tree, then the neighboring parent nodes in the prediction tree are used as the decoded neighbor nodes;

[0298] The first placeholder determination subunit 1322 is configured to set the value of the placeholder code of the decoded neighbor node to a first code value, and determine target placeholder information of the decoded neighbor node based on the placeholder code of the decoded neighbor node having the first code value.

[0299] The child node search sub-unit 1323 is configured to use the division layer where the node to be decoded is located as a child node layer, and search for neighboring child nodes of the node to be decoded in the child node layer based on the first hybrid decoding optimization strategy;

[0300] The second placeholder determination sub-unit 1324 is used to, if the parent node of the found neighbor child node exists in the prediction tree, use the neighbor child node in the decoded neighbor node as the target neighbor child node when using the parent node in the prediction tree as the decoded neighbor node, and determine the target placeholder information of the decoded neighbor node based on the placeholder code of the target neighbor child node.

[0301] The specific implementation of the parent node search subunit 1321, the first placeholder determination subunit 1322, the child node search subunit 1323, and the second placeholder determination subunit 1324 can be found in the above Figure 9 Step S103 in the corresponding embodiment will not be described in detail here.

[0302] The placeholder code of the target neighbor subnode is a default code value; the default code value is the first code value or the second code value.

[0303] The placeholder code of the target neighbor subnode is determined by the partition tree type adopted by the coding unit to which the decoded neighbor node belongs.

[0304] The placeholder code of the target neighbor child node is determined by the real placeholder code obtained after partitioning the decoded neighbor node based on the target partition tree type; the target partition tree type includes one of an octree partition type, a quadtree partition type, and a binary tree partition type.

[0305] The placeholder code of the target neighbor child node is determined by the decoding information associated with the decoded neighbor node.

[0306] The decoding information includes point cloud statistical information of the decoded neighbor node; the placeholder code of the target neighbor child node is determined by the point cloud statistical information and a statistical threshold associated with the point cloud statistical information; the point cloud statistical information includes one of the number of point cloud points in the decoded neighbor node and the point cloud density in the decoded neighbor node.

[0307] Among them, the decoding information includes the reconstruction point of the prediction tree where the decoded neighbor node is located; when the spatial distance between the reconstruction point of the target neighbor sub-node and the node to be decoded is the target spatial distance, the placeholder code of the target neighbor sub-node is determined by the first code value obtained when the target spatial distance is less than the distance threshold.

[0308] The decoding information includes a placeholder code of a reference decoded neighbor node of the decoded neighbor node; and the placeholder code of the target neighbor child node is determined by the placeholder code of the reference decoded neighbor node.

[0309] The decoding information includes the child node placeholder code of the partition tree neighbor child node that has the same parent node as the node to be decoded; the partition tree neighbor child node is the decoded neighbor child node in the partition tree; the placeholder code of the target neighbor child node is determined by the child node placeholder code.

[0310] In which, when the child node placeholder code indicates that the partition tree neighbor child node located at the child node layer is in an occupied state, the target neighbor child node located at the child node layer is in an occupied state, and the placeholder code of the target neighbor child node is determined by the first code value obtained when the target neighbor child node is in an occupied state.

[0311] The partition tree neighbor subnodes include the target partition tree neighbor subnode having the smallest distance to the decoded neighbor node; and the placeholder code of the target neighbor subnode is determined by the child node placeholder code of the target partition tree neighbor subnode.

[0312] Among them, the placeholder code of the target neighbor child node is determined by the association relationship between the division depth of the node to be decoded and the depth threshold; the association relationship includes one of the first association relationship when the division depth is less than the depth threshold and the second association relationship when the division depth is greater than the depth threshold.

[0313] Among them, the neighbor node types of the target neighbor sub-node include one or more of coplanar neighbor sub-nodes that are coplanar with the node to be decoded, collinear neighbor sub-nodes that are colinear with the node to be decoded, co-point neighbor sub-nodes that have a common point with the node to be decoded, and close neighbor sub-nodes that are L sub-node side lengths away from the node to be decoded in the negative direction of the target dimension of the coordinate system where the node to be decoded is located; L is a positive integer greater than 1.

[0314] Wherein, when the structure type of the first geometric structure is a prediction tree type, the first node as the node to be decoded is a node in a first prediction tree corresponding to the prediction tree type, and when the structure type of the second geometric structure is a partitioning tree type, the second node as the decoded neighbor node is a node in a partitioning tree corresponding to the partitioning tree type;

[0315] The occupancy determination module 13 may include: a second strategy acquisition unit 133, a second occupancy determination unit 134;

[0316] A second strategy acquisition unit 133 is configured to acquire a second hybrid decoding optimization strategy associated with the first node based on the second spatial decoding data information of the second node located in the partitioning tree;

[0317] The second placeholder determining unit 134 is configured to use the placeholder code of the decoded neighboring node as the target placeholder information of the decoded neighboring node based on the second hybrid decoding optimization strategy.

[0318] The specific implementation of the first strategy acquisition unit 131, the first placeholder determination unit 132, the second strategy acquisition unit 133, and the second placeholder determination unit 134 can be found in the above Figure 9 Step S103 in the corresponding embodiment will not be described in detail here.

[0319] The geometry decoding module 14 is used to decode the node geometry information of the node to be decoded in the compressed code stream based on the target occupancy information; the node geometry information includes any one of the partitioning tree occupancy information and the prediction tree residual information associated with the structure type of the first geometric structure.

[0320] The target occupancy information is used to determine the reconstruction point of the decoded neighboring node; the reconstruction point of the decoded neighboring node is used as the starting point of the second prediction tree corresponding to the node to be decoded or the independent reference point of the node to be decoded; the second prediction tree is determined by reconstructing the first prediction tree using the reconstruction point of the decoded neighboring node; the independent reference point is independent of the first prediction tree;

[0321] The geometric decoding module 14 is specifically configured to use the first prediction reference point or the independent reference point in the second prediction tree as the target reference point, and decode the prediction tree residual information of the node to be decoded based on the reconstructed coordinate position information of the target reference point.

[0322] Wherein, when the structure type of the first geometric structure is a prediction tree type, the first node as the to-be-decoded node is a node in a first prediction tree corresponding to the prediction tree type, and when the decoded neighbor node is a third node in a third geometric structure having the same structure type as the first geometric structure, the third node as the decoded neighbor node is a node in a third prediction tree corresponding to the prediction tree type;

[0323] The device also includes:

[0324] The residual decoding module 15 is configured to determine a target reference point based on a starting point of the third prediction tree or an ending point of the third prediction tree, and decode the prediction tree residual information of the node to be decoded based on the reconstructed coordinate position information of the target reference point.

[0325] Among them, the residual decoding module 15 is specifically used to reconstruct the first prediction tree through the target starting point when the starting point of the third prediction tree or the ending point of the third prediction tree is used as the target starting point, so as to obtain a reconstructed prediction tree corresponding to the node to be decoded; the starting point of the reconstructed prediction tree is the target starting point; and the second prediction reference point in the reconstructed prediction tree is used as the target reference point.

[0326] The target reference point is a starting point of the third prediction tree or an ending point of the third prediction tree, and the target reference point is independent of the first prediction tree.

[0327] The specific implementation of the code stream acquisition module 11, the neighbor determination module 12, the occupancy determination module 13, the geometric decoding module 14, and the residual decoding module 15 can be referred to above. Figure 9 Steps S101 to S104 in the corresponding embodiment will not be described in detail here. In addition, the description of the beneficial effects of adopting the same method will not be described in detail either.

[0328] See Figure 18 , is a schematic diagram of the structure of a point cloud data processing device provided in an embodiment of the present application. Figure 18 As shown, the point cloud data processing device 2 can be a computer program (including program code) running on an encoding device (for example, the encoding device 100A mentioned above), for example, the point cloud data processing device 2 is an application software; it can be understood that the point cloud data processing device 2 can be used to execute the corresponding steps in the point cloud data processing method provided in the embodiment of the present application. Figure 18 As shown, the point cloud data processing device 2 may include: a node determination module 21, an information determination module 22, and a geometric encoding module 23;

[0329] a node determination module 21 for determining, when the node to be encoded is the first node in the first geometric structure of the point cloud data, an encoded neighboring node that has a spatially adjacent relationship with the node to be encoded;

[0330] an information determining module 22 for determining target placeholder information of the encoded neighbor node when the encoded neighbor node is a second node in a second geometric structure different from the first geometric structure;

[0331] The geometric encoding module 23 is used to encode the node geometric information of the node to be encoded based on the target occupancy information to obtain a compressed code stream of the point cloud data; the node geometric information includes any one of the partitioning tree occupancy information and the prediction tree residual information associated with the structure type of the first geometric structure.

[0332] The specific implementation of the node determination module 21, the information determination module 22, and the geometric coding module 23 can be found in the above Figure 16 Steps S201 to S203 in the corresponding embodiment will not be described in detail here. In addition, the description of the beneficial effects of adopting the same method will not be described in detail either.

[0333] See Figure 19 , is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 19 As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Figure 19 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

[0334] In such Figure 19 In the computer device 1000 shown, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to execute the above Figure 9 、 Figure 16The description of the point cloud data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated here either.

[0335] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores the computer program executed by the point cloud data processing device 1 and the point cloud data processing device 2 mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the above-mentioned Figure 9 、 Figure 16 The description of the point cloud data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated here. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0336] The computer-readable storage medium may be the point cloud data processing apparatus provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Furthermore, the computer-readable storage medium may include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is about to be output.

[0337] In addition, it should be noted that the present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned Figure 9 、 Figure 16 The method provided by any corresponding embodiment. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0338] For further information, see Figure 20 , Figure 20This is a schematic diagram of the structure of a point cloud data processing system provided in an embodiment of the present application. Figure 20 As shown, the point cloud data processing system 3 may include a point cloud data processing device 1a and a point cloud data processing device 2a. The point cloud data processing device 1a may be the above-mentioned Figure 17 The point cloud data processing device 1 in the corresponding embodiment can be understood that the point cloud data processing device 1a can be integrated into the above Figure 1 The encoding device 100B in the corresponding embodiment will not be described in detail here. Figure 18 The point cloud data processing device 2 in the corresponding embodiment can be understood that the point cloud data processing device 2a can be integrated into the above Figure 1 The encoding device 100A in the corresponding embodiment will not be described in detail here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the embodiment of the point cloud data processing system involved in this application, please refer to the description of the method embodiment of this application.

[0339] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0340] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0341] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A point cloud data processing method, characterized in that: include: Get the compressed code stream of point cloud data; When the node to be decoded is the first node in the first geometric structure of the point cloud data, determining a decoded neighbor node having a spatial adjacent relationship with the node to be decoded; When the decoded neighbor node is a second node in a second geometric structure different from the first geometric structure, determining target placeholder information of the decoded neighbor node; Decoding node geometry information of the to-be-decoded node in the compressed bitstream based on the target occupancy information, wherein the node geometry information includes any one of partition tree occupancy information and prediction tree residual information associated with the structure type of the first geometric structure; The step of decoding the node geometry information of the node to be decoded in the compressed code stream based on the target placeholder information includes: If the node to be decoded is a partition tree node in a partition tree when the structure type of the first geometric structure is a partition tree type, and the decoded neighbor node is a prediction tree node in a prediction tree when the structure type of the second geometric structure is a prediction tree type, decoding the partition tree placeholder information of the node to be decoded by using the target placeholder information of the decoded neighbor node and the placeholder information of other decoded partition tree neighbor nodes in the same geometric structure as the partition tree type; If the node to be decoded is a prediction tree node in the first prediction tree when the structure type of the first geometric structure is a prediction tree type, and the decoded neighbor node is a partition tree node in the partition tree when the structure type of the second geometric structure is a partition tree type, then the prediction tree residual information of the node to be decoded is decoded through the target placeholder information of the decoded neighbor node and the decoding information of other decoded prediction tree neighbor nodes located in the same geometric structure as the prediction tree type.

2. The method according to claim 1, characterized in that When the structure type of the first geometric structure is a partition tree type, the first node serving as the to-be-decoded node is a node in a partition tree corresponding to the partition tree type, and when the structure type of the second geometric structure is a prediction tree type, the second node serving as the decoded neighbor node is a node in a prediction tree corresponding to the prediction tree type; The determining target placeholder information of the decoded neighbor node includes: acquiring a first hybrid decoding optimization strategy associated with the first node based on first spatial decoding data information of the second node located in the prediction tree; Target placeholder information of the decoded neighbor node is determined based on the first hybrid decoding optimization strategy.

3. The method according to claim 2, characterized in that The determining the target placeholder information of the decoded neighbor node based on the first hybrid decoding optimization strategy includes: Determine the partition layer where the parent node of the node to be decoded is located as the parent node layer, and search for the neighboring parent node of the parent node in the parent node layer based on the first hybrid decoding optimization strategy; If there is a neighbor parent node located in the prediction tree among the neighbor parent nodes found, then the neighbor parent node located in the prediction tree is used as the decoded neighbor node; The value of the placeholder code of the decoded neighbor node is set to a first code value, and target placeholder information of the decoded neighbor node is determined based on the placeholder code of the decoded neighbor node having the first code value.

4. The method according to claim 2, characterized in that The determining the target placeholder information of the decoded neighbor node based on the first hybrid decoding optimization strategy includes: Taking the division layer where the node to be decoded is located as a sub-node layer, and searching for neighbor sub-nodes of the node to be decoded in the sub-node layer based on the first hybrid decoding optimization strategy; If the parent node of the neighbor child node found exists in the prediction tree, then when the parent node in the prediction tree is used as the decoded neighbor node, the neighbor child node in the decoded neighbor node is used as the target neighbor child node, and the target placeholder information of the decoded neighbor node is determined based on the placeholder code of the target neighbor child node.

5. The method according to claim 4, characterized in that The placeholder code of the target neighbor subnode has a default code value; the default code value is the first code value or the second code value.

6. The method according to claim 4, characterized in that The placeholder code of the target neighbor subnode is determined by the partition tree type adopted by the coding unit to which the decoded neighbor node belongs.

7. The method according to claim 4, characterized in that The placeholder code of the target neighbor child node is determined by the real placeholder code obtained after dividing the decoded neighbor node based on the target partition tree type; the target partition tree type includes one of an octree partition type, a quadtree partition type and a binary tree partition type.

8. The method according to claim 4, characterized in that The placeholder code of the target neighbor child node is determined by decoding information associated with the decoded neighbor node.

9. The method according to claim 8, characterized in that The decoding information includes point cloud statistical information of the decoded neighbor node; the placeholder code of the target neighbor child node is determined by the point cloud statistical information and a statistical threshold associated with the point cloud statistical information; the point cloud statistical information includes one of the number of point cloud points in the decoded neighbor node and the point cloud density in the decoded neighbor node.

10. The method according to claim 8, characterized in that The decoding information includes the reconstruction point of the prediction tree where the decoded neighbor node is located; when the spatial distance between the reconstruction point of the target neighbor child node and the node to be decoded is the target spatial distance, the placeholder code of the target neighbor child node is determined by the first code value obtained when the target spatial distance is less than the distance threshold.

11. The method according to claim 8, characterized in that The decoding information includes a placeholder code of a reference decoded neighbor node of the decoded neighbor node; the placeholder code of the target neighbor child node is determined by the placeholder code of the reference decoded neighbor node.

12. The method according to claim 8, characterized in that The decoding information includes a child node placeholder of a neighbor child node of a partition tree having the same parent node as the node to be decoded; the neighbor child node of the partition tree is a decoded neighbor child node in the partition tree; The placeholder code of the target neighbor child node is determined by the child node placeholder code.

13. The method according to claim 12, characterized in that When the child node placeholder code indicates that the partition tree neighbor child node located at the child node layer is in an occupied state, the target neighbor child node located at the child node layer is in the occupied state, and the placeholder code of the target neighbor child node is determined by the first code value obtained when the target neighbor child node is in the occupied state.

14. The method according to claim 12, characterized in that The partition tree neighbor sub-nodes include a target partition tree neighbor sub-node having the smallest distance to the decoded neighbor node; the placeholder code of the target neighbor sub-node is determined by the child node placeholder code of the target partition tree neighbor sub-node.

15. The method according to claim 4, characterized in that The placeholder code of the target neighbor sub-node is determined by the association relationship between the division depth of the node to be decoded and the depth threshold; the association relationship includes one of a first association relationship when the division depth is less than the depth threshold and a second association relationship when the division depth is greater than the depth threshold.

16. The method according to claim 4, characterized in that The neighbor node types of the target neighbor sub-node include one or more of coplanar neighbor sub-nodes that are coplanar with the node to be decoded, collinear neighbor sub-nodes that are colinear with the node to be decoded, co-point neighbor sub-nodes that have a common point with the node to be decoded, and close neighbor sub-nodes that are L sub-node side lengths away from the node to be decoded in the negative direction of the target dimension of the coordinate system where the node to be decoded is located; L is a positive integer greater than 1.

17. The method according to claim 1, wherein When the structure type of the first geometric structure is a prediction tree type, the first node serving as the node to be decoded is a node in a first prediction tree corresponding to the prediction tree type, and when the structure type of the second geometric structure is a partitioning tree type, the second node serving as the decoded neighbor node is a node in a partitioning tree corresponding to the partitioning tree type; The determining target placeholder information of the decoded neighbor node includes: acquiring, based on second spatial decoding data information of the second node located in the partitioning tree, a second hybrid decoding optimization strategy associated with the first node; Based on the second hybrid decoding optimization strategy, the placeholder code of the decoded neighbor node is used as the target placeholder information of the decoded neighbor node.

18. The method according to claim 17, characterized in that The target placeholder information is used to determine the reconstruction point of the decoded neighboring node; the reconstruction point of the decoded neighboring node is used as the starting point of the second prediction tree corresponding to the node to be decoded or an independent reference point of the node to be decoded; the second prediction tree is determined by reconstructing the first prediction tree using the reconstruction point of the decoded neighboring node; the independent reference point is independent of the first prediction tree; The decoding of the node geometry information of the to-be-decoded node in the compressed code stream based on the target placeholder information includes: The first prediction reference point or the independent reference point in the second prediction tree is used as a target reference point, and the prediction tree residual information of the node to be decoded is decoded based on the reconstructed coordinate position information of the target reference point.

19. The method according to claim 1, wherein When the structure type of the first geometric structure is a prediction tree type, the first node serving as the node to be decoded is a node in a first prediction tree corresponding to the prediction tree type, and when the decoded neighbor node is a third node in a third geometric structure having the same structure type as the first geometric structure, the third node serving as the decoded neighbor node is a node in a third prediction tree corresponding to the prediction tree type; The method further comprises: A target reference point is determined based on a starting point of the third prediction tree or an ending point of the third prediction tree, and prediction tree residual information of the node to be decoded is decoded based on reconstructed coordinate position information of the target reference point.

20. The method according to claim 19, characterized in that The determining of the target reference point based on the starting point of the third prediction tree or the ending point of the third prediction tree includes: When the starting point of the third prediction tree or the ending point of the third prediction tree is used as the target starting point, the first prediction tree is reconstructed through the target starting point to obtain a reconstructed prediction tree corresponding to the node to be decoded; the starting point of the reconstructed prediction tree is the target starting point; The second prediction reference point in the reconstructed prediction tree is used as the target reference point.

21. The method according to claim 19, wherein The target reference point is a starting point of the third prediction tree or an ending point of the third prediction tree, and the target reference point is independent of the first prediction tree.

22. A point cloud data processing method, characterized in that: include: When the node to be encoded is the first node in the first geometric structure of the point cloud data, determining an encoded neighboring node having a spatial adjacent relationship with the node to be encoded; When the encoded neighbor node is a second node in a second geometric structure different from the first geometric structure, determining target placeholder information of the encoded neighbor node; Encoding the node geometry information of the node to be encoded based on the target occupancy information to obtain a compressed code stream of the point cloud data; The node geometry information includes any one of partition tree placeholder information and prediction tree residual information associated with the structure type of the first geometric structure; The step of encoding the node geometry information of the node to be encoded based on the target placeholder information to obtain a compressed code stream of the point cloud data includes: If the node to be encoded is a partition tree node in a partition tree when the structure type of the first geometric structure is a partition tree type, and the encoded neighbor node is a prediction tree node in a prediction tree when the structure type of the second geometric structure is a prediction tree type, encoding the partition tree placeholder information of the node to be encoded by using the target placeholder information of the encoded neighbor node and the placeholder information of other encoded partition tree neighbor nodes in the same geometric structure as the partition tree type, to obtain a compressed code stream of the point cloud data; If the node to be encoded is a prediction tree node in the first prediction tree when the structure type of the first geometric structure is a prediction tree type, and the encoded neighbor node is a partition tree node in the partition tree when the structure type of the second geometric structure is a partition tree type, then the prediction tree residual information of the node to be encoded is encoded through the target occupancy information of the encoded neighbor node and the encoding information of other encoded prediction tree neighbor nodes located in the same geometric structure as the prediction tree type to obtain a compressed code stream of the point cloud data.

23. A point cloud data processing device, characterized in that: include: The code stream acquisition module is used to obtain the compressed code stream of point cloud data; a neighbor determination module, configured to determine, when the node to be decoded is the first node in the first geometric structure of the point cloud data, decoded neighbor nodes that have a spatially adjacent relationship with the node to be decoded; an occupancy determination module, configured to determine target occupancy information of the decoded neighbor node when the decoded neighbor node is a second node in a second geometric structure different from the first geometric structure; a geometry decoding module, configured to decode node geometry information of the to-be-decoded node in the compressed bitstream based on the target placeholder information, wherein the node geometry information includes any one of partition tree placeholder information and prediction tree residual information associated with the structure type of the first geometric structure; The geometric decoding module is specifically configured to: if the node to be decoded is a partition tree node in a partition tree when the structure type of the first geometric structure is a partition tree type, and the decoded neighbor node is a prediction tree node in a prediction tree when the structure type of the second geometric structure is a prediction tree type, decode the partition tree placeholder information of the node to be decoded using the target placeholder information of the decoded neighbor node and the placeholder information of other decoded partition tree neighbor nodes in the same geometric structure as the partition tree type; If the node to be decoded is a prediction tree node in the first prediction tree when the structure type of the first geometric structure is a prediction tree type, and the decoded neighbor node is a partition tree node in the partition tree when the structure type of the second geometric structure is a partition tree type, then the prediction tree residual information of the node to be decoded is decoded through the target placeholder information of the decoded neighbor node and the decoding information of other decoded prediction tree neighbor nodes located in the same geometric structure as the prediction tree type.

24. A point cloud data processing device, characterized in that: include: a node determination module, configured to determine, when the node to be encoded is the first node in the first geometric structure of the point cloud data, an encoded neighboring node having a spatially adjacent relationship with the node to be encoded; an information determination module, configured to determine target placeholder information of the encoded neighbor node when the encoded neighbor node is a second node in a second geometric structure different from the first geometric structure; A geometric encoding module, configured to encode the node geometric information of the node to be encoded based on the target occupancy information to obtain a compressed code stream of the point cloud data; The node geometry information includes any one of partition tree placeholder information and prediction tree residual information associated with the structure type of the first geometric structure; The geometric coding module is specifically used to: If the node to be encoded is a partition tree node in a partition tree when the structure type of the first geometric structure is a partition tree type, and the encoded neighbor node is a prediction tree node in a prediction tree when the structure type of the second geometric structure is a prediction tree type, encoding the partition tree placeholder information of the node to be encoded by using the target placeholder information of the encoded neighbor node and the placeholder information of other encoded partition tree neighbor nodes in the same geometric structure as the partition tree type, to obtain a compressed code stream of the point cloud data; If the node to be encoded is a prediction tree node in the first prediction tree when the structure type of the first geometric structure is a prediction tree type, and the encoded neighbor node is a partition tree node in the partition tree when the structure type of the second geometric structure is a partition tree type, then the prediction tree residual information of the node to be encoded is encoded through the target occupancy information of the encoded neighbor node and the encoding information of other encoded prediction tree neighbor nodes located in the same geometric structure as the prediction tree type to obtain a compressed code stream of the point cloud data.

25. A computer device, characterized in that: include: processor and memory; The processor is connected to the memory, wherein the memory is used to store a computer program, and the processor is used to call the computer program to enable the computer device to execute the method according to any one of claims 1 to 22.

26. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 22.

Images