Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

The point cloud data transmission method according to the embodiments can include the steps of obtaining point cloud data, encoding geometry information on positions of points included in the point cloud data by applying a geometry tree, encoding attribute information on attribute values of the points included in the point cloud data based on the geometry tree, and transmitting the encoded geometry information, the encoded attribute information, and signaling information, wherein the step of encoding the attribute information includes the steps of generating a level of detail (LoD) based on the geometry tree and compressing the attribute information based on the LoD.

Description

Technical Field

[0001] The implementation methods relate to methods and apparatus for processing point cloud content. Background Technology

[0002] Point cloud content is content represented by point clouds, which are collections of points belonging to a coordinate system representing three-dimensional space (or volume). Point cloud content can represent media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), extended reality (XR), and autonomous driving. However, tens of thousands to hundreds of thousands of point data points are required to represent point cloud content. Therefore, methods for efficiently processing large amounts of point data are needed. Summary of the Invention

[0003] Technical issues

[0004] The purpose of this disclosure, designed to solve the above problems, is to provide a point cloud data transmission apparatus, a point cloud data transmission method, a point cloud data receiving apparatus, and a point cloud data receiving method for efficiently transmitting and receiving point clouds.

[0005] Another object of this disclosure is to provide a point cloud data transmission device, a point cloud data transmission method, a point cloud data receiving device, and a point cloud data receiving method for dealing with waiting time and encoding / decoding complexity.

[0006] Another object of this disclosure is to provide a point cloud data transmission apparatus, a point cloud data transmission method, a point cloud data receiving apparatus, and a point cloud data receiving method for efficiently transmitting and receiving geometric point cloud compression (G-PCC) bit streams.

[0007] Another object of this disclosure is to provide a point cloud data transmission apparatus, a point cloud data transmission method, a point cloud data receiving apparatus, and a point cloud data receiving method that match the geometric coding layer with the scalable coding of the attribute coding layer by performing attribute compression considering the asymmetric point distribution.

[0008] Another object of this disclosure is to provide a point cloud data transmission apparatus, a point cloud data transmission method, a point cloud data receiving apparatus, and a point cloud data receiving method that increase the attribute compression efficiency of content with asymmetric point distribution by using a LoD-based scalable layer in scalable point cloud encoding to configure the scalable layer based on asymmetric neighbor nodes.

[0009] The purpose of this disclosure is not limited to the purposes mentioned above, and other purposes of this disclosure not mentioned above will become clear to those skilled in the art upon review of the following description.

[0010] Technical solution

[0011] The above and other objectives of this disclosure can be achieved by providing a method for transmitting point cloud data. The method may include the following steps: acquiring point cloud data; encoding geometric information of the positions of points including the point cloud data by applying a geometric tree; encoding attribute information of the attribute values ​​of the points including the point cloud data based on the geometric tree; and transmitting the encoded geometric information, the encoded attribute information, and signaling information.

[0012] In one implementation, the step of encoding the attribute information may include: generating a level of detail (LoD) based on the geometry tree; and compressing the attribute information based on the LoD.

[0013] In one implementation, the geometric tree may be generated by a combination of two or more of an octree, a quadtree, or a binary tree.

[0014] In one implementation, the step of generating the LOD may include: configuring one or more child nodes that are split from the occupying node at the current depth in the geometry tree as neighbors; and selecting a point from the points belonging to the one or more child nodes configured as neighbors, wherein configuring the neighbors and selecting the points may be performed on each of the occupying nodes at the current depth to generate an LOD corresponding to the current depth.

[0015] In one implementation, the number of child nodes configured as the neighbor can vary depending on whether the tree type at the current depth is an octree, a quadtree, or a binary tree.

[0016] In one implementation, the selection of the point may be performed based on the Morton code order of the points belonging to the one or more child nodes configured as the neighbors.

[0017] An apparatus for transmitting point cloud data according to an embodiment may include: an acquirer configured to acquire point cloud data; a geometry encoder configured to encode geometric information of the positions of points including the point cloud data by applying a geometry tree; an attribute encoder configured to encode attribute information of attribute values ​​of the points including the point cloud data based on the geometry tree; and a transmitter configured to transmit the encoded geometric information, the encoded attribute information, and signaling information.

[0018] In one embodiment, the attribute encoder may include: a LoD generator configured to generate a level of detail (LoD) based on the geometry tree; and a compressor configured to compress the attribute information based on the LoD.

[0019] In one implementation, the geometric tree may be generated by a combination of two or more of an octree, a quadtree, or a binary tree.

[0020] In one implementation, the LOD generator can be configured to: configure one or more child nodes that are split from the occupied node at the current depth in the geometry tree as neighbors; and select a point from among the points belonging to the one or more child nodes configured as neighbors, wherein the configuration and the selection can be performed on each of the occupied nodes at the current depth to generate a LoD corresponding to the current depth.

[0021] In one implementation, the number of child nodes configured as the neighbor can vary depending on whether the tree type at the current depth is an octree, a quadtree, or a binary tree.

[0022] In one implementation, the LOD generator may select the point based on the Morton code order of the points belonging to the one or more child nodes configured as the neighbors.

[0023] The method for receiving point cloud data according to the embodiments may include the following steps: receiving geometric information, attribute information, and signaling information; decoding the geometric information based on the signaling information and the geometric tree; decoding the attribute information based on the signaling information and the geometric tree; and rendering the point cloud data reconstructed based on the decoded geometric information and the decoded attribute information.

[0024] In one implementation, the decoded geometric information may include the positions of points in the reconstructed point cloud data, and the decoded attribute information may include the attribute values ​​of points in the reconstructed point cloud data.

[0025] In one implementation, the step of decoding the attribute information may include: generating a level of detail (LoD) based on the geometry tree; and reconstructing the attribute information based on the LoD.

[0026] In one implementation, the geometric tree may be generated by a combination of two or more of an octree, a quadtree, or a binary tree.

[0027] In one implementation, the step of generating the LOD may include: configuring one or more child nodes that are split from the occupying node at the current depth in the geometry tree as neighbors; and selecting a point from the points belonging to the one or more child nodes configured as neighbors, wherein configuring the neighbors and selecting the points may be performed on each of the occupying nodes at the current depth to generate an LOD corresponding to the current depth.

[0028] In one implementation, the number of child nodes configured as the neighbor can vary depending on whether the tree type at the current depth is an octree, a quadtree, or a binary tree.

[0029] In one implementation, the selection of the point may be performed based on the Morton code order of the points belonging to the one or more child nodes configured as the neighbors.

[0030] An apparatus for receiving point cloud data according to an embodiment may include: a receiver configured to receive geometric information, attribute information, and signaling information; a geometry decoder configured to decode the geometric information based on the signaling information and a geometry tree; an attribute decoder configured to decode the attribute information based on the signaling information and the geometry tree; and a renderer configured to render the point cloud data reconstructed based on the decoded geometric information and the decoded attribute information.

[0031] In one implementation, the decoded geometric information may include the positions of points in the reconstructed point cloud data, and the decoded attribute information may include the attribute values ​​of points in the reconstructed point cloud data.

[0032] In one implementation, the attribute decoder may include: a Level of Detail (LoD) generator configured to generate LoDs based on the geometry tree; and a reconstructor configured to reconstruct the attribute information based on the LoDs.

[0033] In one implementation, the geometric tree may be generated by a combination of two or more of an octree, a quadtree, or a binary tree.

[0034] In one implementation, the LOD generator may be configured to: configure one or more child nodes that are split from the occupied node at the current depth in the geometry tree as neighbors; and select a point from among the points belonging to the one or more child nodes configured as neighbors, wherein the configuration and the selection may be performed on each of the occupied nodes at the current depth to generate an LOD corresponding to the current depth.

[0035] In one implementation, the number of child nodes configured as the neighbor can vary depending on whether the tree type at the current depth is an octree, a quadtree, or a binary tree.

[0036] In one implementation, the LOD generator may select the point based on the Morton code order of the points belonging to the one or more child nodes configured as the neighbors.

[0037] Beneficial effects

[0038] The point cloud data transmission method, point cloud data transmission device, point cloud data reception method, and point cloud reception device according to the embodiments can provide high-quality point cloud services.

[0039] The point cloud data transmission method, point cloud data transmission device, point cloud data reception method, and point cloud reception device according to the embodiments can implement various video encoding and decoding methods.

[0040] The point cloud data transmission method, point cloud data transmission device, point cloud data reception method, and point cloud reception device according to the embodiments can provide general point cloud content such as autonomous driving services (or autonomous driving services).

[0041] The point cloud data transmission method, point cloud data transmission apparatus, point cloud data reception method, and point cloud data reception apparatus according to the embodiments can perform spatial adaptive segmentation of point cloud data to independently encode and decode point cloud data, thereby improving parallel processing and providing scalability.

[0042] The point cloud data transmission method, point cloud data transmission apparatus, point cloud data reception method, and point cloud data reception apparatus according to the embodiments can perform encoding and decoding by dividing point cloud data into units of tiles and / or slices, and thus signal the necessary data, thereby improving the encoding and decoding performance of point clouds.

[0043] The point cloud data transmission method, point cloud data transmission apparatus, point cloud data reception method, and point cloud data reception apparatus according to the embodiments can improve the attribute compression efficiency of content with asymmetric point distribution by configuring a scalable layer using an asymmetric neighbor node in scalable point cloud encoding using a LoD-based scalable layer.

[0044] According to the embodiments, the point cloud data transmitting apparatus, point cloud data transmitting method, point cloud data receiving apparatus, and point cloud data receiving method can generate a LoD for attribute compression using an OT / QT / BT-based geometric tree structure for geometric compression, thereby matching the number of points in the geometry with the number of points for the attributes configured for the corresponding depth in the scalable representation. According to the embodiments, an intermediate layer can be output based on the hierarchical structure of the point cloud data. This can be defined as a scalable representation.

[0045] The point cloud data transmitting apparatus, point cloud data transmitting method, point cloud data receiving apparatus, and point cloud data receiving method according to the embodiments can be limited to depth-generated LoDs that match the QT (OT in 3D space)-based LoD generation method in the geometric tree structure used for geometry compression. Therefore, point mismatch between geometry and attributes can be prevented in scalable representations.

[0046] The point cloud data transmitting apparatus, point cloud data transmitting method, point cloud data receiving apparatus, and point cloud data receiving method according to the embodiments can match points / nodes with attribute information in a one-to-one correspondence based on the depth in the geometric tree when performing QTBT-based geometric encoding for scalable encoded application fields. Therefore, scalable representation can be enabled for all depths or for certain matching depths. Attached Figure Description

[0047] The accompanying drawings are included to provide a further understanding of this disclosure and are incorporated in and constitute a part of this application. The drawings illustrate embodiments of this disclosure and, together with the description, serve to illustrate the principles of this disclosure.

[0048] Figure 1 An exemplary point cloud content provisioning system according to an implementation method is illustrated.

[0049] Figure 2 This is a block diagram illustrating the operation of providing point cloud content according to an implementation method.

[0050] Figure 3 An exemplary process for capturing point cloud video according to an implementation method is illustrated.

[0051] Figure 4 An exemplary block diagram of a point cloud video encoder according to an embodiment is shown.

[0052] Figure 5 An example of a voxel in 3D space according to an embodiment is shown.

[0053] Figure 6 An example of an octree and occupancy code according to an implementation method is shown.

[0054] Figure 7 An example of a neighboring node pattern according to an implementation method is shown.

[0055] Figure 8 An example of point configuration for point cloud content per LOD according to an implementation method is shown.

[0056] Figure 9 An example of point configuration for point cloud content per LOD according to an implementation method is shown.

[0057] Figure 10 An example block diagram of a point cloud video decoder according to an embodiment is shown.

[0058] Figure 11 An example of a point cloud video decoder according to an implementation method is shown.

[0059] Figure 12 The configuration of point cloud video encoding of the transmitting device according to the embodiment is illustrated.

[0060] Figure 13 The configuration of point cloud video decoding of the receiving device according to the embodiment is illustrated.

[0061] Figure 14 An exemplary structure for operation of a point cloud data method / apparatus according to an embodiment is illustrated.

[0062] Figure 15 This is a diagram illustrating an example of point cloud data composed of asymmetrically distributed points according to an implementation method.

[0063] Figure 16 This is a diagram illustrating an example of segmenting a cube based on an octree in 3D space.

[0064] Figure 17 (a) to Figure 17 (c) is a diagram illustrating an example of a cube segmented based on a quadtree in 3D space.

[0065] Figure 18 (a) to Figure 18 (c) is a diagram illustrating an example of a cube segmented based on a binary tree in 3D space.

[0066] Figure 19 (a) and Figure 19 (b) illustrates an example of segmenting a bounding box in 2D space.

[0067] Figure 20 This is a diagram illustrating an example of an attribute encoder generating a Level of Detail (LOD) based on an octree according to an implementation method.

[0068] Figure 21 This is a diagram illustrating an example of the Morton code order in a 3D cube according to an embodiment.

[0069] Figure 22 This is a diagram illustrating an example of an attribute selection process based on the LoD generated for nodes in a 2D plane according to an implementation method.

[0070] Figure 23 This is a diagram illustrating another example of the attribute selection process generated by the attribute encoder based on the LoD of a node in a 2D plane according to an implementation method.

[0071] Figure 24 (a) and Figure 24 (b) is a diagram illustrating an example of LoD generation based on a geometric tree structure used to segment point cloud data distributed in a 2D plane at each depth.

[0072] Figure 25 (a) and Figure 25(b) is a diagram illustrating other examples of LoD generation based on the geometric tree structure used to segment point cloud data distributed on a 2D plane at each depth.

[0073] Figure 26 This is a diagram illustrating an example of LoD generation at certain depths in a geometric tree structure used to segment point cloud data distributed in a 2D plane.

[0074] Figure 27 This is another example of LoD generation at certain depths in a geometric tree structure used to segment point cloud data distributed in a 2D plane.

[0075] Figure 28 (a) and Figure 28 (b) is a diagram illustrating an example of a configuration for skipping LoD at certain depths in a geometric tree structure for segmenting point cloud data distributed in a 2D plane, according to the conditions described above.

[0076] Figure 29 This is another example of a configuration for skipping LoD at certain depths in a geometric tree structure for segmenting point cloud data distributed in a 2D plane, based on the conditions described above.

[0077] Figure 30 This is another example of LoD generation based on a geometric tree structure used to segment point cloud data distributed in a 2D plane at each depth.

[0078] Figure 31 An exemplary bitstream structure for sending / receiving point cloud data according to an embodiment is illustrated.

[0079] Figure 32 An exemplary bitstream structure for sending / receiving point cloud data according to an embodiment is illustrated.

[0080] Figure 33 The connection relationships between elements in a bitstream of point cloud data according to an embodiment are illustrated.

[0081] Figure 34 An implementation of the syntax structure of the sequence parameter set according to this disclosure is shown.

[0082] Figure 35 An example of the syntax structure of the geometric parameter set according to the implementation is shown.

[0083] Figure 36 An example of the syntax structure of the attribute parameter set according to the implementation is shown.

[0084] Figure 37An example of a syntax structure for an attribute parameter set that includes information related to LoD-based attribute compression, according to an implementation method, is shown.

[0085] Figure 38 An exemplary syntax structure for geometry_slice_bitstream() according to an implementation is shown.

[0086] Figure 39 An exemplary syntax structure for a geometry slice header according to an implementation is shown.

[0087] Figure 40 An exemplary syntax structure for geometric slice data according to an implementation method is shown.

[0088] Figure 41 An exemplary syntax structure for attribute_slice_bitstream() according to an implementation is shown.

[0089] Figure 42 An exemplary syntax structure for an attribute slice header according to an implementation is shown.

[0090] Figure 43 An exemplary syntax structure for a set of attribute parameters, including information related to LoD-based attribute compression, is shown according to an embodiment.

[0091] Figure 44 An exemplary syntax structure for attribute slice data according to an implementation method is shown.

[0092] Figure 45 Another example of a point cloud transmitting apparatus according to an embodiment is shown.

[0093] Figure 46 This is an exemplary detailed block diagram illustrating an attribute encoder according to an implementation method.

[0094] Figure 47 This is a flowchart illustrating an exemplary LoD generation method according to an implementation method.

[0095] Figure 48 Another example of a point cloud receiving device according to an embodiment is illustrated.

[0096] Figure 49 This is an exemplary detailed block diagram illustrating an attribute decoder according to an implementation method.

[0097] Figure 50 This is a flowchart illustrating a method for sending point cloud data according to an implementation method.

[0098] Figure 51 This is a flowchart illustrating a method for receiving point cloud data according to an embodiment. Detailed Implementation

[0099] The following description will now be given in detail with reference to the accompanying drawings and exemplary embodiments disclosed herein. For the sake of brevity with reference to the drawings, the same reference numerals may be used for the same or equivalent parts, and their descriptions will not be repeated. It should be noted that the following examples are for illustrative purposes only and do not limit the scope of this disclosure. What can be readily inferred by one of the art to which this disclosure pertains from the detailed description and examples of this disclosure will be interpreted as being within the scope of this disclosure.

[0100] The detailed description in this specification should be construed as illustrative rather than restrictive in all respects. The scope of this disclosure should be determined by the appended claims and their legal equivalents, and all changes falling within the meaning and scope of the appended claims are intended to be covered herein.

[0101] Now, reference will be made in detail to preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The following detailed description, given with reference to the accompanying drawings, is intended to explain exemplary embodiments of the present disclosure, and not to show the only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details. While most of the terms used in this specification have been selected from commonly used terms widely in the art, the applicant has arbitrarily chosen some terms, and their meanings will be explained in detail in the following description as needed. Therefore, the present disclosure should be understood based on the literal meaning of the terms rather than their simple names or meanings. Furthermore, the following drawings and detailed description should not be construed as limiting to the specifically described embodiments, but should be construed as including equivalents or alternatives to the embodiments described in the drawings and detailed description.

[0102] Figure 1 An exemplary point cloud content delivery system according to an implementation is shown.

[0103] Figure 1 The point cloud content providing system illustrated herein may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 are capable of wired or wireless communication to transmit and receive point cloud data.

[0104] The point cloud data transmission device 10000 according to an embodiment can protect and process point cloud video (or point cloud content) and transmit the point cloud video (or point cloud content). According to an embodiment, the transmission device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device, and / or a server. According to an embodiment, the transmission device 10000 may include devices configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers.

[0105] The transmitting device 10000 according to the embodiment includes a point cloud video acquisition unit 10001, a point cloud video encoder 10002 and / or a transmitter (or communication module) 10003.

[0106] The point cloud video acquisition unit 10001 according to an embodiment acquires point cloud video through processing procedures such as capture, synthesis, or generation. Point cloud video is point cloud content represented by a point cloud as a set of points in 3D space, and may be referred to as point cloud video data. The point cloud video according to an embodiment may include one or more frames. A frame represents a still image / picture. Therefore, point cloud video may include point cloud images / frames / pictures, and may be referred to as point cloud images, frames, or pictures.

[0107] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 can encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to an embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next-generation coding. The point cloud compression coding according to an embodiment is not limited to the above embodiments. The point cloud video encoder 10002 can output a bitstream containing the encoded point cloud video data. The bitstream may contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0108] According to an embodiment, transmitter 10003 transmits a bitstream containing encoded point cloud video data. The bitstream, according to an embodiment, is encapsulated in a file or segment (e.g., a streaming segment) and transmitted over various networks such as broadcast networks and / or broadband networks. Although not shown in the figures, transmitting device 10000 may include an encapsulator (or encapsulation module) configured to perform encapsulation operations. According to an embodiment, the encapsulator may be included in transmitter 10003. According to an embodiment, the file or segment can be transmitted over a network to receiving device 10004 or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). Transmitter 10003 according to an embodiment is capable of wired / wireless communication with receiving device 10004 (or receiver 10005) via networks such as 4G, 5G, and 6G. Additionally, the transmitter can perform necessary data processing operations depending on the network system (e.g., a 4G, 5G, or 6G communication network system). Transmitting device 10000 can transmit encapsulated data on demand.

[0109] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include devices, robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)).

[0110] According to an embodiment, receiver 10005 receives a bitstream containing point cloud video data or a file / segment encapsulated with the bitstream from a network or storage medium. Receiver 10005 can perform necessary data processing according to the network system (e.g., 4G, 5G, 6G, etc. communication network systems). According to an embodiment, receiver 10005 can decapsulate the received file / segment and output the bitstream. According to an embodiment, receiver 10005 may include a decapsulator (or decapsulator module) configured to perform a decapsulation operation. The decapsulator may be implemented as a separate element (or component or module) from receiver 10005.

[0111] The point cloud video decoder 10006 decodes a bitstream containing point cloud video data. The point cloud video decoder 10006 can decode point cloud video data according to a method used to encode the point cloud video data (e.g., the inverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode point cloud video data by performing point cloud decompression encoding, which is the inverse process of point cloud compression. Point cloud decompression encoding includes G-PCC encoding.

[0112] Renderer 10007 renders the decoded point cloud video data. According to one embodiment, renderer 10007 can render the decoded point cloud data according to the viewport. Renderer 10007 can output point cloud content by rendering not only the point cloud video data but also the audio data. According to one embodiment, renderer 10007 may include a display configured to display the point cloud content. According to one embodiment, the display may be implemented as a separate device or component, rather than being included in renderer 10007.

[0113] The arrows indicated by the dashed lines in the diagram represent the transmission paths of the feedback information acquired by the receiving device 10004. The feedback information reflects the interactivity of a user consuming the point cloud content and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is the content of a service requiring user interaction (e.g., autonomous driving service, etc.), the feedback information can be provided to the content sender (e.g., the sending device 10000) and / or the service provider. Depending on the implementation, the feedback information may be used in both the receiving device 10004 and the sending device 10000, or it may not be provided.

[0114] According to the embodiment, head orientation information can represent information about the position, orientation, angle, and movement of the user's head. The receiving device 10004 according to the embodiment can calculate viewport information based on the head orientation information. Viewport information is information related to the area of ​​the point cloud video that the user is viewing (i.e., the area the user is currently viewing). In other words, viewport information is information about the area the user is currently viewing in the point cloud video. In other words, the viewport or viewport area can represent the area the user is viewing in the point cloud video. The viewpoint is the point the user is viewing in the point cloud video and can represent the center point of the viewport area. That is, the viewport is the area centered on the viewpoint, and the size and shape of this area can be determined by the field of view (FOV). Therefore, the receiving device 10004 can extract viewport information based on the vertical or horizontal FOV supported by the device and the head orientation information. Furthermore, the receiving device 10004 can perform gaze analysis, etc., based on the head orientation information and / or viewport information to determine how the user consumes the point cloud video, the area the user is gazing at in the point cloud video, and the gaze duration. According to one embodiment, the receiving device 10004 can send feedback information, including gaze analysis results, to the transmitting device 10000. According to another embodiment, a device such as a VR / XR / AR / MR display can extract the viewport region based on the user's head position / orientation and a vertical or horizontal field of view (FOV) supported by the device. According to another embodiment, head orientation information and viewport information can be referred to as feedback information, signaling information, or metadata.

[0115] Feedback information, according to the embodiment, can be obtained during rendering and / or display processing. The feedback information can be acquired by one or more sensors included in the receiving device 10004. According to the embodiment, the feedback information can be acquired by the renderer 10007 or by a separate external component (or device, part, etc.). Figure 1 The dashed lines in the diagram represent the processing of feedback information obtained by renderer 10007. This feedback information can be sent not only to the sending side but also consumed by the receiving side. In other words, the point cloud content providing system can process (encode / decode / render) point cloud data based on the feedback information. For example, point cloud video decoder 10006 and renderer 10007 can prioritize decoding and rendering only the point cloud video of the area currently being viewed by the user, based on the feedback information (i.e., head orientation information and / or viewport information).

[0116] The receiving device 10004 can send feedback information to the transmitting device 10000. The transmitting device 10000 (or the point cloud video encoder 10002) can perform encoding operations based on the feedback information. Therefore, the point cloud content providing system can efficiently process necessary data (e.g., point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide point cloud content to the user.

[0117] According to the implementation, the transmitting device 10000 may be referred to as an encoder, transmitting device, transmitter, transmitting system, etc., and the receiving device 10004 may be referred to as a decoder, receiving device, receiver, receiving system, etc.

[0118] (Through a series of processes including acquisition / encoding / sending / decoding / rendering) according to the implementation method Figure 1 The point cloud data processed in the point cloud content provision system can be referred to as point cloud content data or point cloud video data. According to the implementation, point cloud content data can be used as a concept encompassing metadata or signaling information related to the point cloud data.

[0119] Figure 1 The components of the point cloud content providing system illustrated herein can be implemented by hardware, software, processors, and / or combinations thereof.

[0120] Figure 2 This is a block diagram illustrating the operation of providing point cloud content according to an implementation method.

[0121] Figure 2 The block diagram shows Figure 1 The operation of the point cloud content providing system described herein. As mentioned above, the point cloud content providing system can process point cloud data based on point cloud compression encoding (e.g., G-PCC).

[0122] A point cloud content providing system according to an embodiment (e.g., point cloud sending device 10000 or point cloud video acquisition unit 10001) can acquire point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system used to represent 3D space. The point cloud video according to an embodiment may include a Ply (polygon file format or Stanford triangle format) file. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. The Ply file contains point cloud data such as point geometry and / or attributes. The geometry includes the position of the points. The position of each point may be represented by parameters (e.g., values ​​of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include the attributes of the points (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc., of each point). A point has one or more attributes. For example, a point may have an attribute as color or two attributes as color and reflectivity. According to the implementation, the geometric structure can be referred to as location, geometric information, geometric data, etc., and the attribute can be referred to as attribute, attribute information, attribute data, etc. The point cloud content providing system (e.g., point cloud sending device 10000 or point cloud video acquisition unit 10001) can acquire point cloud data based on information related to the acquisition and processing of point cloud video (e.g., depth information, color information, etc.).

[0123] A point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) according to an embodiment can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression encoding. As described above, point cloud data can include the geometric structure and attributes of points. Therefore, the point cloud content providing system can perform geometric encoding to encode the geometric structure and output a geometric bitstream. The point cloud content providing system can perform attribute encoding to encode the attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system can perform attribute encoding based on geometric encoding. The geometric bitstream and attribute bitstream according to an embodiment can be multiplexed and output as a single bitstream. The bitstream according to an embodiment may also contain signaling information related to geometric encoding and attribute encoding.

[0124] A point cloud content providing system according to an embodiment (e.g., transmitting device 10000 or transmitter 10003) can transmit encoded point cloud data (20002). For example... Figure 1As illustrated, the encoded point cloud data can be represented by a geometric bitstream and an attribute bitstream. Additionally, the encoded point cloud data can be transmitted as a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometric and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit the bitstream as a file or segment.

[0125] The point cloud content providing system (e.g., receiving device 10004 or receiver 10005) according to the embodiment can receive a bitstream containing encoded point cloud data. Additionally, the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) can demultiplex the bitstream.

[0126] A point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode encoded point cloud data (e.g., geometric bitstream, attribute bitstream) transmitted in a bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode point cloud video data based on signaling information related to the encoding of point cloud video data contained in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the geometric bitstream to reconstruct the location (geometric structure) of points. The point cloud content providing system can reconstruct the attributes of points by decoding the attribute bitstream based on the reconstructed geometric structure. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can reconstruct point cloud video based on the location of the reconstructed geometric structure and the decoded attributes.

[0127] A point cloud content providing system (e.g., receiving device 10004 or renderer 10007) according to an embodiment can render decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can use various rendering methods to render the geometry and attributes decoded through the decoding process. Points in the point cloud content can be rendered as vertices with a certain thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to a user through a display (e.g., a VR / AR display, a conventional display, etc.).

[0128] The point cloud content providing system (e.g., receiving device 10004) according to an embodiment can protect feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. The feedback information and operation and reference of the point cloud content providing system according to an embodiment... Figure 1The feedback information and operations described are the same, so a detailed description of them is omitted.

[0129] Figure 3 An exemplary process for capturing point cloud video according to an implementation method is illustrated.

[0130] Figure 3 Examples of references are provided. Figures 1 to 2 The described point cloud content provides an example of point cloud video capture processing for the system.

[0131] Point cloud content includes point cloud videos (images and / or videos) representing objects and / or environments located in various 3D spaces (e.g., 3D spaces representing real environments, 3D spaces representing virtual environments, etc.). Therefore, the point cloud content providing system according to embodiments can capture point cloud videos using one or more cameras (e.g., infrared cameras capable of protecting depth information, RGB cameras capable of extracting color information corresponding to the depth information, etc.), projectors (e.g., infrared pattern projectors for protecting depth information), LiDRA, etc. The point cloud content providing system according to embodiments can extract the shape of the geometry composed of points in 3D space from the depth information and extract the attributes of each point from the color information to protect the point cloud data. Images and / or videos according to embodiments can be captured based on at least one of inward-oriented and outward-oriented techniques.

[0132] Figure 3 The left side illustrates inward-facing technology. Inward-facing technology refers to the technique of capturing images of a central object using one or more cameras (or camera sensors) positioned around it. Inward-facing technology can be used to generate point cloud content that provides users with 360-degree images of key objects (e.g., VR / AR content that provides users with 360-degree images of objects such as characters, players, objects, or actors).

[0133] Figure 3 The right side illustrates outward-facing techniques. Outward-facing techniques refer to techniques that capture images of the environment of a central object, rather than the central object itself, using one or more cameras (or camera sensors) positioned around it. Outward-facing techniques can be used to generate point cloud content that provides the surrounding environment from a user's perspective (e.g., content representing the external environment that can be provided to users of autonomous vehicles).

[0134] like Figure 3As shown, point cloud content can be generated based on the capture operations of one or more cameras. In this case, the coordinate system is different in each camera; therefore, the point cloud content providing system can calibrate one or more cameras to set the global coordinate system before the capture operation. Alternatively, the point cloud content providing system can generate point cloud content by compositing arbitrary images and / or videos with images and / or videos captured using the aforementioned capture techniques. The point cloud content providing system cannot perform the following when generating point cloud content representing virtual space: Figure 3 The capture operations described herein. The point cloud content providing system according to an embodiment can perform post-processing on the captured images and / or videos. In other words, the point cloud content providing system can remove unwanted areas (e.g., background), identify spaces to which the captured images and / or videos are connected, and perform a space-hole filling operation when space holes exist.

[0135] A point cloud content delivery system can generate point cloud content by performing coordinate transformations on points in a point cloud video protected by each camera. The system can perform coordinate transformations on points based on the coordinates of each camera position. Therefore, the system can generate content representing a wide range or point cloud content with high-density points.

[0136] Figure 4 An exemplary point cloud video encoder according to an implementation method is illustrated.

[0137] Figure 4 It shows Figure 1 An example of a point cloud video encoder 10002. The point cloud video encoder reconstructs and encodes point cloud data (e.g., point locations and / or attributes) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) based on network conditions or applications. When the total size of the point cloud content is large (e.g., 60Gbps of point cloud content for 30fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on a maximum target bitrate to provide the point cloud content according to network conditions, etc.

[0138] For reference Figures 1 to 2 The point cloud video encoder described herein can perform geometric encoding and attribute encoding. Geometric encoding is performed before attribute encoding.

[0139] The point cloud video encoder according to the embodiment includes a coordinate transformation unit 40000, a quantization unit 40001, an octree analysis unit 40002, a surface approximation analysis unit 40003, an arithmetic encoder 40004, a geometric reconstruction unit 40005, a color transformation unit 40006, an attribute transformation unit 40007, a RAHT unit 40008, a LOD generator 40009, a lift transformation unit 40010, a coefficient quantization unit 40011, and / or an arithmetic encoder 40012.

[0140] The coordinate transformation unit 40000, quantization unit 40001, octree analysis unit 40002, surface approximation analysis unit 40003, arithmetic encoder 40004, and geometric reconstruction unit 40005 can perform geometric encoding. Geometric encoding according to the implementation may include octree geometric encoding, direct encoding, trisoup geometric encoding, and entropy encoding. Direct encoding and trisoup geometric encoding are applied selectively or in combination. Geometric encoding is not limited to the examples described above.

[0141] As shown in the figure, the coordinate transformation unit 40000 according to the embodiment receives the position and transforms it into coordinates. For example, the position can be transformed into position information in three-dimensional space (e.g., three-dimensional space represented by the XYZ coordinate system). The position information in three-dimensional space according to the embodiment can be referred to as geometric information.

[0142] The quantization unit 40001 according to the embodiment quantizes geometric information. For example, the quantization unit 40001 may quantize points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). The quantization unit 40001 performs the following quantization operation: multiplying the difference between the position value of each point and the minimum position value by a preset quantization scaling value, and then finding the nearest integer value by rounding the value obtained by multiplication. Thus, one or more points may have the same quantized position (or position value). The quantization unit 40001 according to the embodiment performs voxelization based on the quantized position to reconstruct the quantized points. Voxelization means the smallest unit representing position information in 3D space. Points in the point cloud content (or 3D point cloud video) according to the embodiment may be included in one or more voxels. The term voxel, a compound word of volume and pixel, refers to the 3D cubic space generated when the 3D space is divided into units (unit = 1.0) based on axes representing the 3D space (e.g., X-axis, Y-axis, and Z-axis). The quantization unit 40001 can match groups of points in 3D space to voxels. According to one embodiment, a voxel may include only one point. According to another embodiment, a voxel may include one or more points. To represent a voxel as a point, the position of the voxel's center point can be set based on the positions of one or more points included in the voxel. In this case, attributes of all positions included in a voxel can be combined and assigned to that voxel.

[0143] According to the implementation method, the octree analysis unit 40002 performs octree geometric encoding (or octree encoding) to represent voxels in an octree structure. The octree structure represents points based on the octree structure and voxel matching.

[0144] The surface approximation analysis unit 40003 according to the embodiment can perform octree analysis and approximation. The octree analysis and approximation according to the embodiment analyzes a region containing multiple points to efficiently provide octree and voxelization processing.

[0145] The arithmetic encoder 40004 according to the embodiment performs entropy encoding on an octree and / or an approximate octree. For example, the encoding scheme includes arithmetic encoding. As a result of the encoding, a geometric bitstream is generated.

[0146] The attribute encoding is performed by a color transformation unit 40006, an attribute transformation unit 40007, a RAHT unit 40008, a LOD generator 40009, a boosting transformation unit 40010, a coefficient quantization unit 40011, and / or an arithmetic encoder 40012. As described above, a point can have one or more attributes. The attribute encoding according to the embodiment is also applied to the attributes that a point has. However, when an attribute (e.g., color) includes one or more elements, the attribute encoding is applied independently to each element. The attribute encoding according to the embodiment includes color transformation encoding, attribute transformation encoding, region adaptive hierarchical transformation (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) encoding, and interpolation-based hierarchical nearest neighbor prediction encoding with an update / boosting step (boosting transformation). Depending on the point cloud content, the above-described RAHT encoding, prediction transformation encoding, and boosting transformation encoding can be selectively used, or a combination of one or more encoding schemes can be used. The attribute encoding according to the embodiment is not limited to the examples described above.

[0147] The color conversion unit 40006 according to the embodiment performs color conversion encoding of the color values ​​(or textures) included in the conversion attributes. For example, the color conversion unit 40006 can convert the format of color information (e.g., from RGB to YCbCr). The operation of the color conversion unit 40006 according to the embodiment can be optionally applied according to the color values ​​included in the attributes.

[0148] According to the implementation method, the geometric reconstruction unit 40005 reconstructs (decompresses) the octree and / or approximate octree. The geometric reconstruction unit 40005 reconstructs the octree / voxel based on the results of analyzing the distribution of points. The reconstructed octree / voxel can be referred to as the reconstructed geometry (recovered geometry).

[0149] The attribute transformation unit 40007, according to the embodiment, performs attribute transformation to transform attributes based on the location and / or reconstructed geometry where geometric encoding has not been performed. As described above, since attributes depend on the geometry, the attribute transformation unit 40007 can transform attributes based on reconstructed geometric information. For example, based on the position values ​​of points included in a voxel, the attribute transformation unit 40007 can transform the attributes of points at that location. As described above, when the position of the voxel center is set based on the positions of one or more points included in the voxel, the attribute transformation unit 40007 transforms the attributes of said one or more points. When performing trisoup geometric encoding, the attribute transformation unit 40007 can transform attributes based on the trisoup geometric encoding.

[0150] The attribute transformation unit 40007 can perform attribute transformation by calculating the average of the attributes or attribute values ​​(e.g., color or reflectivity of each point) of neighboring points within a specific position / radius from the center (or position value) of each voxel. The attribute transformation unit 40007 can apply weights based on the distance from the center to each point when calculating the average. Therefore, each voxel has a position and a calculated attribute (or attribute value).

[0151] The attribute transformation unit 40007 can search for neighboring points existing within a specific location / radius from the center of each voxel based on a KD-tree or Morton code. A KD-tree is a binary search tree and supports point data structures that can be managed based on location to enable fast nearest neighbor search (NNS). The Morton code is generated by representing the coordinates (e.g., (x, y, z)) of the 3D location of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the location of a point are (5, 9, 1), the bit values ​​are (0101, 1001, 0001). The bit values ​​are mixed according to the bit indices in the order of z, y, and x to produce 010001000111. This value is represented as the decimal number 1095. That is, the Morton code value for a point with coordinates (5, 9, 1) is 1095. The attribute transformation unit 40007 can sort the points based on the Morton code values ​​and perform NNS through depth-first traversal. After an attribute transformation operation, if an NNS is required in another transformation process used for attribute encoding, use a KD tree or Morton code.

[0152] As shown in the figure, the transformed attributes are input to RAHT unit 40008 and / or LOD generator 40009.

[0153] According to the implementation, the RAHT unit 40008 performs RAHT encoding for predicting attribute information based on the reconstructed geometric information. For example, the RAHT unit 40008 can predict the attribute information of higher-level nodes in the octree based on the attribute information associated with lower-level nodes in the octree.

[0154] The LOD generator 40009 according to the embodiment generates the Level of Detail (LOD). The LOD according to the embodiment represents the level of detail of the point cloud content. A decrease in the LOD value indicates a decrease in the level of detail of the point cloud content. An increase in the LOD value indicates an increase in the level of detail of the point cloud content. Points can be classified according to their LOD.

[0155] The lifting transformation unit 40010 according to the embodiment performs lifting transformation coding to transform the attributes of the point cloud based on weights. As described above, lifting transformation coding may be optionally applied.

[0156] According to the implementation method, the coefficient quantization unit 40011 quantizes the attribute after attribute encoding based on the coefficient.

[0157] According to the implementation method, the arithmetic encoder 40012 encodes the quantized attributes based on arithmetic encoding.

[0158] Although not shown in the figure, Figure 4 The elements of the point cloud video encoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud content providing device. The one or more processors can perform the above-described... Figure 4 At least one of the operations and / or functions of the elements of the point cloud video encoder. Additionally, one or more processors can operate on or execute a set of software programs and / or instructions to perform... Figure 4 The operation and / or functionality of the elements of the point cloud video encoder. One or more memories according to the embodiment may include high-speed random access memory, or include non-volatile memory (e.g., one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices).

[0159] Figure 5 An example of a voxel according to an embodiment is shown.

[0160] Figure 5 This illustrates voxels in 3D space represented by a coordinate system consisting of three axes: the X-axis, Y-axis, and Z-axis. (See reference...) Figure 4 As described, a point cloud video encoder (e.g., quantization unit 40001) can perform voxelization. A voxel refers to the 3D cubic space generated when the 3D space is divided into units (unit = 1.0) based on axes representing the 3D space (e.g., X-axis, Y-axis, and Z-axis). Figure 5 An example of a voxel generated by an octree structure is shown, in which there are two poles (0, 0, 0) and (2). d ,2 d ,2 d The cubic axis-aligned bounding box, as defined, is recursively subdivided. A voxel comprises at least one point. The spatial coordinates of a voxel can be estimated based on its positional relationship to a group of voxels. As mentioned above, voxels possess properties similar to pixels in a 2D image / video (such as color or reflectivity). Details and references for voxels are provided. Figure 4 The details described are the same, so the description of it is omitted.

[0161] Figure 6 An example of an octree and occupancy code according to an implementation is shown.

[0162] For reference Figures 1 to 4The described point cloud content providing system (point cloud video encoder 10002) or the octree analysis unit 40002 of the point cloud video encoder performs octree geometric coding (or octree coding) based on an octree structure to efficiently manage the regions and / or locations of voxels.

[0163] Figure 6 The upper part shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by axes of a coordinate system (e.g., X-axis, Y-axis, and Z-axis). The octree structure is recursively subdivided into two poles (0, 0, 0) and (2...). d ,2 d ,2 d It is created by defining a bounding box aligned to the cubic axis. Here, 2 d This can be set to the value of the minimum bounding box that constitutes all points surrounding the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in Equation 1. In Equation 1, (x int n ,y int n ,z int n ) indicates the position (or position value) of the quantization point.

[0164] [Formula 1]

[0165]

[0166] like Figure 6 As shown in the upper center, the entire 3D space can be divided into eight spaces according to partitions. Each partitioned space is represented by a cube with six faces. (See diagram below.) Figure 6 As shown in the upper right corner, each of the eight spaces is further divided based on the axes of the coordinate system (e.g., the X, Y, and Z axes). Thus, each space is divided into eight smaller spaces. These smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0167] Figure 6 The lower part shows the octree occupancy code. The occupancy code of the octree is generated to indicate whether each of the eight partitioned spaces resulting from partitioning a space contains at least one point. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of the partitioned space, and the child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to the child node contains at least one point, the node is assigned a value of 1. When the space corresponding to the child node does not contain a point (the space is empty), the node is assigned a value of 0. Since... Figure 6The occupancy code shown is 00100001, indicating that the spaces corresponding to the third and eighth child nodes out of eight each contain at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and each child node is represented by an 8-bit occupancy code. The figure shows the occupancy code for the third child node as 10000111, and the occupancy code for the eighth child node as 01001111. A point cloud video encoder (e.g., an arithmetic encoder 40004) according to an embodiment can perform entropy encoding on the occupancy code. To improve compression efficiency, the point cloud video encoder can perform intra / inter-frame encoding on the occupancy code. A receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to an embodiment reconstructs the octree based on the occupancy code.

[0168] A point cloud video encoder (e.g., octree analysis unit 40002) according to an implementation can perform voxelization and octree encoding to store the locations of points. However, points are not always uniformly distributed in 3D space, and therefore there will be specific regions where fewer points are present. Therefore, performing voxelization over the entire 3D space is inefficient. For example, when a specific region contains fewer points, it is unnecessary to perform voxelization in that specific region.

[0169] Therefore, for the specific region mentioned above (or nodes other than the leaf nodes of the octree), the point cloud video encoder according to the embodiment can skip voxelization and perform direct encoding to directly encode the positions of points included in the specific region. The coordinates of the directly encoded points according to the embodiment are called the Direct Encoding Mode (DCM). The point cloud video encoder according to the embodiment can also perform trisoup geometric encoding based on a surface model to reconstruct the positions of points in the specific region (or node) based on voxels. Trisoup geometric encoding is a geometric encoding that represents an object as a series of triangular meshes. Therefore, the point cloud video decoder can generate a point cloud from the mesh surface. Trisoup geometric encoding and direct encoding according to the embodiment can be selectively performed. In addition, trisoup geometric encoding and direct encoding according to the embodiment can be performed in combination with octree geometric encoding (or octree coding).

[0170] To perform direct encoding, the option to apply direct encoding using direct mode should be enabled. The node to be directly encoded is not a leaf node, and there should be fewer than a threshold number of points within that node. Furthermore, the total number of points to be directly encoded should not exceed a preset threshold. When the above conditions are met, the point cloud video encoder (or arithmetic encoder 40004) according to the implementation can perform entropy encoding on the point positions (or position values).

[0171] A point cloud video encoder according to an embodiment (e.g., surface approximation analysis unit 40003) can determine a specific level of an octree (a level less than the depth d of the octree) and can perform trisoup geometric encoding using a surface model starting from that level to reconstruct the position of points in the region of a node based on voxels (trisoup mode). The point cloud video encoder according to an embodiment can specify the level at which trisoup geometric encoding will be applied. For example, when a specific level is equal to the depth of the octree, the point cloud video encoder does not operate in trisoup mode. In other words, the point cloud video encoder according to an embodiment can operate in trisoup mode only when the specified level is less than the depth value of the octree. The 3D cubic region of a node at the specified level according to an embodiment is called a block. A block may include one or more voxels. A block or voxel may correspond to a brick. The geometry is represented as a surface within each block. A surface according to an embodiment may intersect each edge of a block at most once.

[0172] A block has 12 edges, therefore there are at least 12 intersections within a block. Each intersection is called a vertex (or apex point). A vertex is detected along an edge when there is at least one occupied voxel adjacent to that edge in all blocks sharing that edge. An occupied voxel, according to the implementation, refers to a voxel containing a point. The position of a vertex detected along an edge is the average position of the edges of all voxels adjacent to that edge in all blocks sharing that edge.

[0173] Once a vertex is detected, the point cloud video encoder according to the embodiment can perform entropy encoding on the starting point (x, y, z) of the edge, the direction vector (Δx, Δy, Δz) of the edge, and the vertex position value (relative position value within the edge). When trisoup geometric encoding is applied, the point cloud video encoder according to the embodiment (e.g., geometric reconstruction unit 40005) can generate the restored geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization.

[0174] Vertices at the edges of a block define the surface passing through the block. The surface, according to the implementation, is a non-planar polygon. In the triangle reconstruction process, the surface represented by triangles is reconstructed based on the starting point of the edge, the direction vector of the edge, and the position values ​​of the vertices. According to Equation 2, the triangle reconstruction process is performed by: ① calculating the centroid value of each vertex, ② subtracting the centroid value from each vertex value, and ③ estimating the sum of squares of the values ​​obtained through the subtraction.

[0175] [Equation 2]

[0176]

[0177] Then, the minimum value of the sum is estimated, and projection is performed according to the axis with the minimum value. For example, when element x is minimum, each vertex is projected onto the x-axis relative to the center of the block and onto the (y,z) plane. When the value obtained by projecting onto the (y,z) plane is (ai,bi), the value of θ is estimated by atan2(bi,ai), and the vertices are sorted according to the value of θ. Table 1 below shows the vertex combinations for creating triangles based on the number of vertices. The vertices are sorted from 1 to n. Table 1 below shows that for four vertices, two triangles can be constructed based on the combinations of vertices. The first triangle can be composed of vertices 1, 2, and 3 from the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 from the sorted vertices.

[0178] Table 1. Triangles formed by vertices sorted in order of 1, ..., n [Table 1]

[0179]

[0180] Upsampling is performed to add points along the edges of the triangle at the center and voxelization is then performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called thinned vertices. A point cloud video encoder according to an embodiment can voxelize the thinned vertices. In addition, the point cloud video encoder can perform attribute encoding based on the voxelized locations (or location values).

[0181] Figure 7 An example of a neighbor node pattern according to an implementation is illustrated. To improve the compression efficiency of point cloud video, the point cloud video encoder according to an implementation can perform entropy coding based on context-adaptive arithmetic coding.

[0182] Point cloud video encoders can use context-adaptive arithmetic coding for entropy coding to enhance the compression efficiency of point cloud videos.

[0183] For reference Figures 1 to 6 Described, Figure 1 The point cloud content providing system or point cloud video encoder 10002 or Figure 4 The point cloud video encoder or arithmetic encoder 40004 can immediately perform entropy coding on the occupancy code. Alternatively, the point cloud content providing system or point cloud video encoder can perform entropy coding (intra-frame coding) based on the occupancy code of the current node and the occupancy of neighboring nodes, or perform entropy coding (inter-frame coding) based on the occupancy code of the previous frame. According to the embodiment, a frame represents a collection of simultaneously generated point cloud videos. The compression efficiency of the intra-frame coding / inter-frame coding according to the embodiment can depend on the number of referenced neighboring nodes. As the number of bits increases, the operation becomes more complex, but the coding can be biased to one side, thereby increasing compression efficiency. For example, when given a 3-bit context, 2...3 = There are 8 methods to perform the encoding. The division of space for encoding affects the complexity of the implementation. Therefore, an appropriate level of compression efficiency and complexity must be achieved.

[0184] Figure 7 This illustrates the process of obtaining an occupancy pattern based on the occupancy of neighboring nodes. A point cloud video encoder according to an implementation determines the occupancy of neighboring nodes for each node in an octree and obtains a neighboring pattern value. This neighboring node pattern is then used to infer the node's occupancy pattern. Figure 7 The upper part of the diagram shows the cube corresponding to each node (the cube in the middle) and six cubes sharing at least one face with the cube (neighboring nodes). The nodes shown in the diagram are nodes at the same depth. The numbers shown in the diagram represent the weights associated with the six nodes (1, 2, 4, 8, 16, and 32). Weights are assigned sequentially based on the position of the neighboring nodes.

[0185] Figure 7 The lower part shows the neighbor node pattern values. The neighbor node pattern value is the sum of the values ​​multiplied by the weights of occupied neighbor nodes (neighbor nodes with points). Therefore, the neighbor node pattern value ranges from 0 to 63. When the neighbor node pattern value is 0, it indicates that there are no nodes with points among the node's neighbors (unoccupied nodes). When the neighbor node pattern value is 63, it indicates that all neighbor nodes are occupied nodes. As shown in the figure, since the neighbor nodes assigned weights 1, 2, 4, and 8 are occupied nodes, the neighbor node pattern value is 15, which is the sum of 1, 2, 4, and 8. The point cloud video encoder can perform encoding based on the neighbor node pattern value (e.g., 64 encodings can be performed when the neighbor node pattern value is 63). According to implementations, the point cloud video encoder can reduce encoding complexity by changing the neighbor node pattern value (e.g., based on a table that changes 64 to 10 or 6).

[0186] Figure 8 An example of point configuration in each LOD according to the implementation method is shown.

[0187] For reference Figures 1 to 7 This describes the reconstruction (decompression) of the encoded geometry before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation may include changing the placement of directly encoded points (e.g., placing directly encoded points in front of the point cloud data). When trisoup geometry encoding is applied, geometry reconstruction is performed through triangle reconstruction, upsampling, and voxelization. Since attributes depend on the geometry, attribute encoding is performed based on the reconstructed geometry.

[0188] Point cloud video encoders (e.g., LOD generator 40009) can classify (or reorganize) points using LOD. Figure 8 The point cloud content corresponding to LOD is shown. Figure 8 The leftmost image in the image represents the original point cloud content. Figure 8 The second image from the left shows the distribution of points in the lowest LOD, and Figure 8 The rightmost image in the table represents the distribution of points in the highest LOD. That is, points in the lowest LOD are sparsely distributed, while points in the highest LOD are densely distributed. In other words, as LOD increases... Figure 8 As the arrow at the bottom points upwards, the space (or distance) between the points narrows.

[0189] Figure 9 An example of point configuration for each LOD according to the implementation method is shown.

[0190] For reference Figures 1 to 8 The described point cloud content delivery system or point cloud video encoder (e.g., Figure 1 Point cloud video encoder 10002, Figure 4 A point cloud video encoder or LOD generator (40009) can generate LODs. LODs are generated by reorganizing points into a set of refinement levels based on a set LOD distance value (or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud video encoder but also by the point cloud video decoder.

[0191] Figure 9 The upper part shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 9 In this context, the original order represents the order of points P0 to P9 before LOD generation. Figure 9 In this context, LOD-based order represents the order in which points are generated according to their LOD values. Points are reorganized using LOD. Furthermore, higher LOD values ​​include points belonging to lower LOD values. For example... Figure 9 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0192] For reference Figure 4 The point cloud video encoder described herein can selectively or in combination perform LOD-based predictive transform coding, LOD-based lifting transform coding, and RAHT transform coding.

[0193] The point cloud video encoder according to the implementation can generate predictors for points to perform LOD-based predictive transform coding to set the predictive attributes (or predictive attribute values) for each point. That is, N predictors can be generated for N points. The predictors according to the implementation can calculate weights (= 1 / distance) based on the LOD value of each point, indexed information related to neighboring points existing within a set distance for each LOD, and the distance to the neighboring points.

[0194] According to the implementation, the predicted attribute (or attribute value) is set to the average of the values ​​obtained by multiplying the attributes (or attribute values) of neighboring points (e.g., color, reflectivity, etc.) set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud video encoder (e.g., coefficient quantization unit 40011) according to the implementation can quantize and inverse quantize the residual (which may be referred to as residual attribute, residual attribute value, attribute prediction residual value, or prediction error attribute value, etc.) of each point obtained by subtracting the predicted attribute (or attribute value) of each point from the attribute (i.e., the original attribute value) of each point. The quantization processing performed on the residual attribute values ​​in the transmitting device is configured as shown in Table 2. The inverse quantization processing performed on the residual attribute values ​​in the receiving device is configured as shown in Table 3.

[0195] [Table 2]

[0196] int PCCQuantization(int value,int quantStep){ if(value>=0){ return floor(value / quantStep+1.0 / 3.0); }else{ return-floor(-value / quantStep+1.0 / 3.0); } }

[0197] [Table 3]

[0198] int PCCInverseQuantization(int value,int quantStep){ if(quantStep==0){ return value; }else{ return value * quantStep; } }

[0199] When the predictor for each point has neighboring points, the point cloud video encoder (e.g., arithmetic encoder 40012) according to the implementation can perform entropy encoding on the quantized and inverse quantized residual attribute values ​​as described above.

[0200] 1) Create an array QuantizationWeight(QW) to store the weight value for each point. All elements of QW are initialized to 1.0. The QW value of the predictor index of the neighboring nodes registered in the predictor is multiplied by the predictor weight of the current point, and the resulting values ​​are summed.

[0201] 2) Improved prediction processing: Subtract the value obtained by multiplying the attribute value of the point by the weight from the existing attribute value to calculate the predicted attribute value.

[0202] 3) Create a temporary array called updateweight, and update and initialize the temporary array to zero.

[0203] 4) The weights calculated by multiplying the weights computed for all predictors by the weights stored in QW corresponding to the predictor indices are cumulatively added to the update weight array as the indices of neighboring nodes. The values ​​obtained by multiplying the attribute values ​​of the neighboring node indices by the calculated weights are cumulatively added to the update array.

[0204] 5) Improved update processing: Divide the attribute values ​​of the update array for all predictors by the weight values ​​of the update weight array of the predictor index, and add the existing attribute values ​​to the values ​​obtained by division.

[0205] 6) The predicted attribute is calculated by multiplying the attribute value updated by the boost update process by the weight (stored in the QW) updated by the boost prediction process for all predictors. The predicted attribute value is quantized by the point cloud video encoder (e.g., coefficient quantization unit 40011) according to the implementation. In addition, the point cloud video encoder (e.g., arithmetic encoder 40012) performs entropy encoding on the quantized attribute value.

[0206] A point cloud video encoder according to an embodiment (e.g., RAHT unit 40008) can perform RAHT transform coding, where attributes associated with lower-level nodes in an octree are used to predict attributes of higher-level nodes. RAHT transform coding is an example of intra-frame attribute coding performed via backward scanning of an octree. The point cloud video encoder according to an embodiment scans the entire region from voxels and repeats a merging process in each step, merging voxels into larger blocks, until the root node is reached. The merging process according to an embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed in a higher mode directly above empty nodes.

[0207] Equation 3 below represents the RAHT transformation matrix. In Equation 3, This represents the average attribute value of the voxel at level l. It can be based on... and To calculate and The weight is and

[0208] [Formula 3]

[0209]

[0210] here, It is a low-pass value and is used in the next higher level of merge processing. This represents the high-pass coefficient. The high-pass coefficient in each step is quantized and undergoes entropy encoding (e.g., via an arithmetic encoder 40012). Weights are calculated as follows: As shown in equation 4 and Calculate the root node.

[0211] [Formula 4]

[0212]

[0213] The value of gDC is also quantized and entropy encoded, just like the high-pass coefficient.

[0214] Figure 10 An example of a point cloud video decoder according to an implementation method is shown.

[0215] Figure 10 The point cloud video decoder shown in the example is Figure 1 The example of the point cloud video decoder 10006 described in [the document], and can perform [operations] with [other functions]. Figure 1 The point cloud video decoder 10006 illustrated in the figure operates in the same or similar manner. As shown in the figure, the point cloud video decoder can receive geometric bitstreams and attribute bitstreams contained in one or more bitstreams. The point cloud video decoder includes a geometry decoder and an attribute decoder. The geometry decoder performs geometry decoding on the geometry bitstream and outputs the decoded geometric structure. The attribute decoder performs attribute decoding on the attribute bitstream based on the decoded geometry and outputs the decoded attributes. The decoded geometry and decoded attributes are used to reconstruct the point cloud content (the decoded point cloud).

[0216] Figure 11 An example of a point cloud video decoder according to an implementation method is shown.

[0217] Figure 11 The point cloud video decoder shown in the example is Figure 10 The example shown is a point cloud video decoder, which can be executed as... Figures 1 to 9 The example above illustrates the decoding operation, which is the inverse of the encoding operation of a point cloud video encoder.

[0218] For reference Figure 1 and Figure 10 The point cloud video decoder described herein can perform geometric decoding and attribute decoding. Geometric decoding is performed before attribute decoding.

[0219] The point cloud video decoder according to the embodiment includes an arithmetic decoder (arithmetic decoding) 11000, an octree synthesizer (synthesized octree) 11001, a surface approximation synthesizer (synthesized surface approximation) 11002, a geometric reconstruction unit (reconstructed geometry) 11003, an inverse coordinate transformer (inverse coordinate transformation) 11004, an arithmetic decoder (arithmetic decoding) 11005, an inverse quantization unit (inverse quantization) 11006, a RAHT transformer 11007, an LOD generator (generated LOD) 11008, an inverse lifter (inverse lift) 11009, and / or an inverse color transformation unit (inverse color transformation) 11010.

[0220] Arithmetic decoder 11000, octree synthesizer 11001, surface approximation synthesizer 11002, geometric reconstruction unit 11003, and coordinate inverse transformer 11004 can perform geometric decoding. Geometric decoding according to embodiments may include direct decoding and trisoup geometric decoding. Direct decoding and trisoup geometric decoding are selectively applied. Geometric decoding is not limited to the examples described above and is provided for reference only. Figures 1 to 9 The inverse processing of the described geometric encoding is performed.

[0221] The arithmetic decoder 11000 according to the embodiment decodes the received geometric bitstream based on arithmetic coding. The operation of the arithmetic decoder 11000 corresponds to the inverse processing of the arithmetic encoder 40004.

[0222] The octree synthesizer 11001 according to the embodiment can generate an octree by obtaining occupancy codes from the decoded geometric bitstream (or information about the geometric structure protected as a decoding result). See reference... Figures 1 to 9 Configure the code in detail.

[0223] When applying trisoup geometry encoding, the surface approximation synthesizer 11002 according to the implementation can synthesize the surface based on the decoded geometry and / or the generated octree.

[0224] According to the embodiment, the geometry reconstruction unit 11003 can regenerate the geometry based on the surface and / or the decoded geometry. See reference... Figures 1 to 9 As described, direct encoding and Trisoup geometric encoding are selectively applied. Therefore, geometric reconstruction unit 11003 directly imports and adds positional information about points where direct encoding has been applied. When Trisoup geometric encoding is applied, geometric reconstruction unit 11003 can reconstruct the geometry by performing reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) by geometric reconstruction unit 40005. Details and References Figure 6The details described are the same, therefore their description is omitted. The reconstructed geometry may include point cloud images or frames that do not contain attributes.

[0225] According to the implementation, the inverse coordinate transformer 11004 can obtain the position of a point by transforming the coordinates based on the reconstructed geometry.

[0226] The arithmetic decoder 11005, inverse quantization unit 11006, RAHT transformer 11007, LOD generator 11008, inverse booster 11009, and / or inverse color transformation unit 11010 can perform reference... Figure 10 The attribute decoding described herein includes Region Adaptive Hierarchical Transformation (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) decoding, and interpolation-based hierarchical nearest neighbor prediction decoding with an update / lifting step (lifting transformation). The above three decoding schemes may be used selectively, or a combination of one or more decoding schemes may be used. The attribute decoding according to the embodiments is not limited to the examples described above.

[0227] According to the embodiment, the arithmetic decoder 11005 decodes the attribute bitstream by arithmetic encoding.

[0228] The inverse quantization unit 11006 according to the embodiment performs inverse quantization on the information about the decoded attribute bitstream or attribute that is protected as a decoding result, and outputs the inverse quantized attribute (or attribute value). Inverse quantization can be selectively applied based on the attribute encoding of the point cloud video encoder.

[0229] According to the implementation, the RAHT transformer 11007, LOD generator 11008, and / or inverse booster 11009 can process the reconstructed geometry and the inversely quantized attributes. As described above, the RAHT transformer 11007, LOD generator 11008, and / or inverse booster 11009 can selectively perform decoding operations corresponding to the encoding of the point cloud video encoder.

[0230] The inverse color transformation unit 11010 according to the embodiment performs inverse transformation encoding to inversely transform the color values ​​(or textures) included in the decoded attributes. The operation of the inverse color transformation unit 11010 can be selectively executed based on the operation of the color transformation unit 40006 of the point cloud video encoder.

[0231] Although not shown in the figure, Figure 11 The elements of the point cloud video decoder can be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits, which are configured to communicate with one or more memories included in the point cloud content providing device. The one or more processors can perform the above-described... Figure 11The point cloud video decoder has at least one or more of the operation and / or functions of its components. Additionally, one or more processors can operate on or execute a set of software programs and / or instructions to perform... Figure 11 The operation and / or functions of the components of the point cloud video decoder.

[0232] Figure 12 An example of a transmitting device according to an embodiment is shown.

[0233] Figure 12 The transmitting device shown is Figure 1 The transmitting device 10000 (or Figure 4 Example of a point cloud video encoder. Figure 12 The transmitting device illustrated in the example can perform and reference Figures 1 to 9 The described point cloud video encoder operates and uses one or more of the same or similar operations and methods. The transmitting apparatus according to the embodiment may include a data input unit 12000, a quantization processor 12001, a voxelization processor 12002, an octree occupancy code generator 12003, a surface model processor 12004, an intra / inter-frame coding processor 12005, an arithmetic encoder 12006, a metadata processor 12007, a color transformation processor 12008, an attribute transformation processor 12009, a LOD / lifting / RAHT transformation processor 12010, an arithmetic encoder 12011, and / or a transmitting processor 12012.

[0234] According to the embodiment, the data input unit 12000 receives or acquires point cloud data. The data input unit 12000 can perform operations and / or acquisition methods similar to those of the point cloud video acquisition unit 10001 (or refer to...). Figure 2 The described acquisition process (20000) is the same as or similar to the operation and / or acquisition method.

[0235] The data input unit 12000, quantization processor 12001, voxelization processor 12002, octree occupancy code generator 12003, surface model processor 12004, intra / inter-frame coding processor 12005, and arithmetic encoder 12006 perform geometric coding. Geometric coding according to the embodiment and reference... Figures 1 to 9 The geometric codes described are the same or similar, so a detailed description of them is omitted.

[0236] The quantization processor 12001 according to the embodiment quantizes geometric structures (e.g., point position values). The operation of the quantization processor 12001 and / or the quantization with reference... Figure 4 The operation and / or quantization of the described quantization unit 40001 are the same as or similar. Details and references Figures 1 to 9 The details described are the same.

[0237] The voxelization processor 12002 according to the embodiment performs voxelization on the quantized position values ​​of points. The voxelization processor 12002 can perform operations similar to those described above. Figure 4 The operation and / or voxelization process of the described quantization unit 40001 are the same as or similar to the operation and / or process. Details and references Figures 1 to 9 The details described are the same.

[0238] The octree occupancy code generator 12003 according to the implementation method performs octree encoding based on the voxelized positions of points in the octree structure. The octree occupancy code generator 12003 can generate occupancy codes. The octree occupancy code generator 12003 can execute and reference... Figure 4 and Figure 6 The operations and / or methods described are the same as or similar to those of the point cloud video encoder (or octree analysis unit 40002). Details and references Figures 1 to 9 The details described are the same.

[0239] According to the implementation, the surface model processor 12004 can perform trisoup geometric encoding based on a surface model to reconstruct the positions of points in a specific region (or node) based on voxels. The surface model processor 12004 can perform operations related to reference... Figure 4 The operation and / or method of the described point cloud video encoder (e.g., surface approximation analysis unit 40003) are the same as or similar to those described. Details and references Figures 1 to 9 The details described are the same.

[0240] The intra / inter-frame coding processor 12005 according to the embodiment can perform intra / inter-frame coding on point cloud data. The intra / inter-frame coding processor 12005 can perform operations similar to those described above. Figure 7 The described intra / inter-frame coding is the same or similar. Details and references Figure 7 The details described are the same. According to the implementation, the intra / inter-frame coding processor 12005 may be included in the arithmetic encoder 12006.

[0241] The arithmetic encoder 12006 according to the embodiment performs entropy encoding on an octree and / or an approximate octree of point cloud data. For example, the encoding scheme includes arithmetic encoding. The arithmetic encoder 12006 performs the same or similar operations and / or methods as the arithmetic encoder 40004.

[0242] The metadata processor 12007 according to the embodiment processes metadata (e.g., set values) about point cloud data and provides it to necessary processing procedures such as geometric encoding and / or attribute encoding. Additionally, the metadata processor 12007 according to the embodiment can generate and / or process signaling information related to geometric encoding and / or attribute encoding. The signaling information according to the embodiment can be encoded separately from geometric encoding and / or attribute encoding. The signaling information according to the embodiment can be interleaved.

[0243] Color transformation processor 12008, attribute transformation processor 12009, LOD / lifting / RAHT transformation processor 12010, and arithmetic encoder 12011 perform attribute encoding. Attribute encoding and reference according to the implementation method. Figures 1 to 9 The attribute codes described are the same or similar, so detailed descriptions of them are omitted.

[0244] According to the embodiment, the color transformation processor 12008 performs color transformation encoding to transform color values ​​included in the attributes. The color transformation processor 12008 can perform color transformation encoding based on a reconstructed geometry. The reconstructed geometry is related to a reference... Figures 1 to 9 The description is the same. Additionally, it performs the same as the reference. Figure 4 The operation and / or method of the color transformation unit 40006 described are the same as or similar to those described. Detailed description of it is omitted.

[0245] The attribute transformation processor 12009 according to the implementation performs attribute transformation to transform attributes based on the reconstructed geometry and / or locations where geometric encoding has not been performed. The attribute transformation processor 12009 performs and references... Figure 4 The operation and / or method of the attribute transformation unit 40007 described herein are the same as or similar to the operation and / or method. Detailed description thereof is omitted. The LOD / lifting / RAHT transformation processor 12010 according to the embodiment can encode the transformed attribute by any one or a combination of RAHT encoding, predictive transformation encoding, and lifting transformation encoding. The LOD / lifting / RAHT transformation processor 12010 performs and references... Figure 4 The operations of the described RAHT unit 40008, LOD generator 40009, and lift transform unit 40010 are at least one of the same or similar operations. Furthermore, the predictive transform coding, lift transform coding, and RAHT transform coding are similar to those of the reference... Figures 1 to 9 The descriptions are the same, so detailed descriptions of them are omitted.

[0246] The arithmetic encoder 12011 according to the embodiment can encode the encoded attributes based on arithmetic encoding. The arithmetic encoder 12011 performs the same or similar operations and / or methods as the arithmetic encoder 40012.

[0247] According to an embodiment, the transmitting processor 12012 can transmit each bitstream containing encoded geometry and / or encoded attributes and / or metadata (or metadata information), or transmit a bitstream configured with encoded geometry and / or encoded attributes and / or metadata. When the encoded geometry and / or encoded attributes and / or metadata according to an embodiment are configured as a bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to an embodiment may contain signaling information, including a sequence parameter set (SPS) for sequence-level signaling, a geometry parameter set (GPS) for signaling for geometry information encoding, an attribute parameter set (APS) for signaling for attribute information encoding, and a tile parameter set (TPS or tile list) and tile data for tile-level signaling. The tile data may include information about one or more tiles. A tile according to an embodiment may include a geometry bitstream Geom00 and one or more attribute bitstreams Attr00 and Attr10. The TPS (or tile list) according to an embodiment may include information about each of one or more tiles (e.g., height / size information and coordinate information about the bounding box). The geometry bitstream may include a header and a payload. The header of the geometry bitstream according to an embodiment may include a geom_parameter_set_id, a geom_tile_id, and a geom_slice_id included in the GPS, as well as information about the data contained in the payload. As described above, the metadata processor 12007 according to an embodiment may generate and / or process signaling information and send it to the transmit processor 12012. According to an embodiment, the elements for performing geometry encoding and the elements for performing attribute encoding may share data / information with each other, as indicated by the dashed lines. The transmit processor 12012 according to an embodiment may perform the same or similar operations and / or transmission methods as the transmitter 10003. Details and References Figure 1 and Figure 2 The details described are the same, so the description of it is omitted.

[0248] Figure 13 An example of a receiving device according to an embodiment is shown.

[0249] Figure 13 The receiving device illustrated in the example is Figure 1 The receiving device 10004 (or Figure 10 and Figure 11 An example of a point cloud video decoder. Figure 13 The receiving device illustrated in the example can perform the same operation as the reference. Figures 1 to 11 The operations and methods described for the point cloud video decoder are one or more of the same or similar operations and methods.

[0250] The receiving apparatus according to the embodiments may include a receiver 13000, a receiving processor 13001, an arithmetic decoder 13002, an octree reconstruction processor based on occupancy codes 13003, a surface model processor (triangle reconstruction, upsampling, voxelization) 13004, an inverse quantization processor 13005, a metadata parser 13006, an arithmetic decoder 13007, an inverse quantization processor 13008, an LOD / lifting / RAHT inverse transform processor 13009, a color inverse transform processor 13010, and / or a renderer 13011. Each element for decoding according to the embodiments may perform the inverse processing of the operation of the corresponding element for encoding according to the embodiments.

[0251] Receiver 13000, according to an embodiment, receives point cloud data. Receiver 13000 can perform operations related to... Figure 1 The operation and / or receiving method of the receiver 10005 are the same as or similar to those of the receiver. Detailed description of it is omitted.

[0252] According to the embodiment, the receiving processor 13001 can acquire geometric bitstreams and / or attribute bitstreams from received data. The receiving processor 13001 may be included in the receiver 13000.

[0253] Arithmetic decoder 13002, octree reconstruction processor based on occupancy code 13003, surface model processor 13004, and inverse quantization processor 13005 can perform geometric decoding. Geometric decoding and reference according to the implementation method. Figures 1 to 10 The described geometric decodings are the same or similar, so a detailed description of them is omitted.

[0254] The arithmetic decoder 13002 according to the embodiment can decode a geometric bitstream based on arithmetic coding. The arithmetic decoder 13002 performs the same or similar operations and / or encoding as the arithmetic decoder 11000.

[0255] According to the embodiment, the octree reconstruction processor 13003 based on occupancy codes can reconstruct an octree by obtaining occupancy codes from the decoded geometric bitstream (or information about the geometric structure obtained as a decoding result). The octree reconstruction processor 13003 based on occupancy codes performs the same or similar operations and / or methods as the octree synthesizer 11001 and / or the octree generation method. When applying trisoup geometric encoding, the surface model processor 13004 according to the embodiment can perform trisoup geometric decoding and related geometric reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on surface model methods. The surface model processor 13004 performs the same or similar operations as the surface approximation synthesizer 11002 and / or the geometric reconstruction unit 11003.

[0256] The inverse quantization processor 13005 according to the implementation method can perform inverse quantization on the decoded geometry.

[0257] The metadata parser 13006 according to the implementation can parse metadata contained in received point cloud data, such as set values. The metadata parser 13006 can transmit metadata for geometry decoding and / or attribute decoding. Metadata and reference Figure 12 The metadata described is the same, so a detailed description of it is omitted.

[0258] The arithmetic decoder 13007, inverse quantization processor 13008, prediction / boost / RAHT inverse transform processor 13009, and color inverse transform processor 13010 perform attribute decoding. Attribute decoding and reference Figures 1 to 10 The properties described are decoded in the same or similar ways, so detailed descriptions of them are omitted.

[0259] The arithmetic decoder 13007 according to the embodiment can decode the attribute bitstream via arithmetic coding. The arithmetic decoder 13007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 13007 performs the same or similar operations and / or encoding as the arithmetic decoder 11005.

[0260] The inverse quantization processor 13008 according to the embodiment can perform inverse quantization on the decoded attribute bitstream. The inverse quantization processor 13008 performs the same or similar operations and / or methods as the inverse quantization unit 11006 and / or the inverse quantization method.

[0261] The prediction / boosting / RAHT inverse transform processor 13009 according to the embodiment can process the reconstructed geometry and inversely quantized attributes. The prediction / boosting / RAHT inverse transform processor 13009 performs one or more operations and / or decodings that are the same as or similar to those of the RAHT transformer 11007, LOD generator 11008, and / or inverse booster 11009. The color inverse transform processor 13010 according to the embodiment performs inverse transform encoding to inversely transform the color values ​​(or textures) included in the decoded attributes. The color inverse transform processor 13010 performs operations and / or inverse transform encodings that are the same as or similar to those of the inverse color transform unit 11010. The renderer 13011 according to the embodiment can render point cloud data.

[0262] Figure 14 An exemplary structure, according to an embodiment, can be connected to a method / apparatus for sending and receiving point cloud data during operation.

[0263] Figure 14 The structure represents a configuration in which at least one of the following components—server 17600, robot 17100, autonomous vehicle 17200, XR device 17300, smartphone 17400, home appliance 17500, and / or head-mounted display (HMD) 17700—is connected to cloud network 17000. Robot 17100, autonomous vehicle 17200, XR device 17300, smartphone 17400, or home appliance 17500 are referred to as devices. Additionally, XR device 17300 may correspond to a point cloud compressed data (PCC) device according to an embodiment, or may be connected to a PCC device during operation.

[0264] Cloud Network 17000 can refer to a network that forms part of or exists within a cloud computing infrastructure. Here, Cloud Network 17000 can be configured using a 3G, 4G, LTE, or 5G network.

[0265] Server 17600 can be connected via cloud network 17000 to at least one of robot 17100, autonomous vehicle 17200, XR device 17300, smartphone 17400, home appliance 17500 and / or HMD 17700, and can assist at least a portion of the processing of connected devices 17100 to 17700.

[0266] HMD 17700 represents one type of implementation of an XR device and / or PCC device according to an embodiment. An HMD-type device according to an embodiment includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0267] In the following, various embodiments of apparatuses 17100 to 17500 to which the above technology is applied will be described. According to the above embodiments, Figure 14 the apparatuses 17100 to 17500 illustrated in [the above] can be connected / linked to a point cloud data transmitting apparatus and a receiver during operation.

[0268] <PCC+XR>

[0269] The XR / PCC apparatus 17300 can adopt PCC technology and / or XR (AR+VR) technology and can be implemented as a head-mounted display (HMD), a head-up display (HUD) installed in a vehicle, a television set, a mobile phone, a smartphone, a computer, a wearable device, a household appliance, a digital signage, a vehicle, a stationary robot, or a mobile robot.

[0270] The XR / PCC apparatus 17300 can analyze 3D point cloud data or image data obtained through various sensors or from an external apparatus and generate position data and attribute data regarding 3D points. Thereby, the XR / PCC apparatus 17300 can obtain information regarding the surrounding space or real objects and render and output XR objects. For example, the XR / PCC apparatus 17300 can match an XR object including auxiliary information regarding an identified object with the identified object and output the matched XR object.

[0271] <PCC+Autopilot+XR>

[0272] The autonomous vehicle 17200 can be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0273] The autonomous vehicle 17200 applying XR / PCC technology can represent an autonomous vehicle provided with an apparatus for providing an XR image or an autonomous vehicle that is a control / interaction target in an XR image. Specifically, the autonomous vehicle 17200 that is a control / interaction target in an XR image can be distinguished from the XR apparatus 17300 and can be connected to the XR apparatus 1730 during operation.

[0274] The autonomous vehicle 17200 having an apparatus for providing an XR / PCC image can obtain sensor information from a sensor including a camera and output the generated XR / PCC image based on the obtained sensor information. For example, the autonomous vehicle 17200 can have a HUD and output an XR / PCC image thereto, thereby providing an XR / PCC object corresponding to a real object or an object existing on a screen to an occupant.

[0275] When an XR / PCC object is output to a HUD, at least a portion of the XR / PCC object can be output to overlap with the actual object pointed to by the occupant's eyes. Conversely, when an XR / PCC object is output to a display installed inside an autonomous vehicle, at least a portion of the XR / PCC object can be output to overlap with objects on the screen. For example, the autonomous vehicle 17200 can output XR / PCC objects corresponding to objects such as roads, other vehicles, traffic lights, traffic signs, two-wheeled vehicles, pedestrians, and buildings.

[0276] Virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or point cloud compression (PCC) technologies according to the implementation methods are applicable to various devices.

[0277] In other words, VR technology is a technology that only provides CG images of real-world objects, backgrounds, etc. AR technology, on the other hand, refers to the technology of displaying virtually created CG images on top of images of real objects. MR technology is similar to AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology differs from AR technology in that AR technology explicitly distinguishes between real objects and virtual objects created as CG images, using virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with the same characteristics as real objects. More specifically, an example of MR technology application is holographic services.

[0278] Recently, VR, AR, and MR technologies have sometimes been referred to as Extended Reality (XR) technologies without being clearly distinguished from each other. Therefore, embodiments of this disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are suitable for such technologies.

[0279] The PCC method / apparatus according to the implementation method can be applied to vehicles that provide autonomous driving services.

[0280] Vehicles providing autonomous driving services connect to the PCC device for wired / wireless communication.

[0281] When the point cloud compressed data (PCC) transmitting / receiving device according to the embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services that can be provided along with autonomous driving services and transmit it to the vehicle. When the PCC transmitting / receiving device is installed in the vehicle, it can receive / process content data related to AR / VR / PCC services based on user input signals input through a user interface device and provide it to the user. The vehicle or user interface device according to the embodiment can receive user input signals. User input signals according to the embodiment may include signals indicating autonomous driving services.

[0282] As mentioned above, point cloud data can consist of a set of points, and each point can have geometric information and attribute information. The geometric information is the three-dimensional position information (xyz) of each point, and the attribute information is the color (RGB, YUV, etc.) and / or the reflectance value of the point.

[0283] The point cloud data encoding process includes compressing geometric information and compressing attribute information based on the geometric information reconstructed from the positional information obtained through compression (reconstructed geometry = decoded geometry). Additionally, the point cloud data decoding process includes: receiving the encoded geometric bitstream and attribute bitstream; decoding the geometric information; and decoding the attribute information based on the geometric information reconstructed through decoding. According to an implementation, the point cloud data encoding process can be performed by… Figure 1 Point cloud video encoder 10002, Figure 2 The code 20001 Figure 4 Point cloud video encoder, Figure 12 Point cloud video encoder or Figure 45 The geometric encoder 51003 and attribute encoder 51004 perform the decoding. The point cloud data decoding process can be performed by... Figure 1 Point cloud video decoder 10006 Figure 2 Decoding 20003 Figure 11 Point cloud video decoder, Figure 13 Point cloud video decoder or Figure 48 The geometry decoder 61003 and attribute decoder 61004 perform the encoding. According to the implementation, the encoding of the geometric information described below can be performed by... Figure 45 The geometric encoder 51003 performs the encoding, and the attribute information can be encoded by... Figure 45 The attribute encoder 51004 is executed. This will be described in detail later. Figure 45 and Figure 48 .

[0284] According to the implementation, the geometric encoder 51003 can use an octree-based, quadtree-based (hereinafter referred to as QT), or binary tree-based (hereinafter referred to as BT) compression method to compress geometric information. In this disclosure, for simplicity, the octree, quadtree, or binary tree used for geometric compression is referred to as a geometric tree. That is, the geometric tree can be one or more of an octree, quadtree, or binary tree.

[0285] According to the implementation, in order to improve the geometric compression efficiency of asymmetrically distributed points, the geometric encoder 51003 can compress geometric information based on a quadtree (hereinafter referred to as QT) or a binary tree (hereinafter referred to as BT).

[0286] In other words, for the eight node positions that make up the octree, when the points are offset towards a plane or a line segment, the number of bits used for geometric compression can be reduced by using QT or BT.

[0287] Figure 15 This is a diagram illustrating an example of point cloud data composed of asymmetrically distributed points according to an implementation method.

[0288] According to the implementation, the geometric encoder 51003 can segment based on one or more of OT, QT, and BT, such as Figure 15 The bounding boxes of the point cloud data shown (which can be used interchangeably with B below) are used to compress geometric information.

[0289] Figure 16 This is a diagram illustrating an example of segmenting a cube based on OT in 3D space.

[0290] In an implementation, when the points in the point cloud data are uniformly distributed, an OT-based segmentation method that divides a cube (i.e., a 3D cube) into eight equal cubes can be used for compression of geometric information about the point cloud data. In this method, the process of dividing a node into eight cubes is repeated by splitting each side of an occupied node in half. Whether each node is occupied can be indicated by sequentially using one bit as 0 or 1. That is, when a cube is divided into eight cubes, eight bits can be used to indicate whether the eight child nodes are occupied. For example, in... Figure 16 In this approach, when nodes 0, 3, 4, and 5 out of the eight nodes are occupied, the occupancy of the corresponding node can be indicated as 10011100. That is, when performing the OT-based segmentation method, an occupancy code is expressed for each of the eight child nodes. In other words, when segmenting a 3D cube based on OT, eight cube nodes are generated, and the occupancy code is represented as an 8-bit code. In the implementation, the eight cube nodes can be referred to as eight child nodes.

[0291] Figure 17 (a) to Figure 17 (c) is a diagram illustrating an example of a QT-segmented cube in 3D space.

[0292] Figure 18 (a) to Figure 18 (c) is a diagram illustrating an example of a BT-divided cube in 3D space.

[0293] In an implementation, when the point cloud data has an uneven point distribution, the number of bits used to indicate the occupied bits can be reduced by using QT and / or BT according to the characteristics of the point distribution.

[0294] According to the implementation method, in the case of QT, the 3D cube can be divided along the xy-axis, yz-axis, or xz-axis, such as... Figure 17 (a) to Figure 17 As shown in (c). Therefore, when segmenting a 3D cube based on QT, four cuboid nodes are generated, and the occupancy code is represented as a 4-bit code. In this case, the occupancy code can be reduced by 4 bits from the occupancy code used for occupancy code based on octtree. In the implementation, the four cuboid nodes can be referred to as four child nodes. That is, a parent node can have four child nodes. For example, when the 3D cube is segmented along the xy axis based on QT and nodes 0 and 6 of the four nodes are occupied, the occupancy of each node can be indicated by 1001, indicating that the segmentation is based on QT in the xy direction.

[0295] According to the implementation method, in the case of BT, the 3D cube can be divided along the x-axis, y-axis, or z-axis, such as... Figure 18 (a) to Figure 18 As shown in (c). Therefore, when segmenting a 3D cube based on BT, two cuboid nodes are generated, and the occupancy code is represented as a 2-bit code. In this case, the occupancy code can be reduced by 6 bits from the occupancy code used for OT-based segmentation. In the implementation, the two cuboid nodes can be referred to as two child nodes. That is, a parent node can have two child nodes.

[0296] According to the implementation method, the order of OT, QT and BT of point cloud data can be determined based on parameters K and M.

[0297] According to the implementation, parameter K can represent the number of times QTBT is performed when the 3D bounding box is started to be segmented, and parameter M can represent the number of OTs performed immediately before the segmentation leaf node level.

[0298] According to the implementation method, for a device with a size of 2 dx 2 dy and 2 dzThe bounding box of the point cloud, parameter K can have values ​​in the range from 0 to max(dx,dy,dz)-min(dx,dy,dz). Parameter M can have values ​​in the range from 0 to min(dx,dy,dz).

[0299] K(0≤K≤max(dx,dy,dz)-min(dx,dy,dz))

[0300] In other words, parameter K represents the maximum number of QTs and BTs before OT (the maximum number of QTs and BTs before OT).

[0301] M(0≤M≤min(dx,dy,dz))

[0302] In other words, parameter M represents the minimum size of QT and BT. When the X-axis size (dx), Y-axis size (dy), and Z-axis size (dz) are less than or equal to M, QT and BT may not be executed (if dx, dy, dz ≤ M, then QT and BT will not be executed).

[0303] Additionally, when the X-axis dimensions (dx), Y-axis dimensions (dy), and Z-axis dimensions (dz) are different from each other (when dx ≠ dy ≠ dz), BT is performed before QT. M = 0 means that OT is performed at least 0 or more times immediately before the split leaf node level. For example, when M = 0, OT may not be performed immediately before the split leaf node level, but OT may be performed a predetermined number of times, such as once or twice.

[0304] Here, min(), med(), and max() are functions used to find the minimum, median, and maximum values, and dx, dy, and dz are log2 values ​​of the size of each of the x, y, and z axes for child nodes at a specific depth.

[0305] According to the implementation method, the number of OT / QT / BT operations can be determined based on the size of the bounding box as described below.

[0306] N(OT) = min(dx, dy, dz)

[0307] N(QT)=med(dx,dy,dz)-min(dx,dy,dz)

[0308] N(BT)=max(dx,dy,dz)-med(dx,dy,dz)

[0309] For example, when performing a QT / BT-based partition, the total number of partition executions is N(OT) + N(QT) + N(BT) = max(dx,dy,dz), which is equal to the total number of OT-based partition executions max(dx,dy,dz). The order of OT / QT / BT is determined by parameters K and M.

[0310] Figure 19 (a) and Figure 19 (b) shows an example of segmenting a bounding box in 2D space. Figure 19 (a) and Figure 19 In (b), the x marker indicates the location of a point within the bounding box.

[0311] Figure 19 (a) shows an example of the segmentation order of point cloud data based solely on the distribution of QT within a 16×4 bounding box in 2D space, and Figure 19 (b) shows an example of the segmentation order of point cloud data distributed in a 16×4 bounding box in 2D space based on QT and BT.

[0312] Figure 19 (a) and Figure 19 The example in (b) can be generalized to the segmentation order in 3D space based on the use of OT, QT, and BT. In this case, QT becomes OT in 3D space, and BT becomes either QT or BT in 3D space.

[0313] exist Figure 19 (a) and Figure 19 In example (b), the split is performed four times until the 1×1 leaf node.

[0314] When segmentation is performed solely based on QT (OT in 3D space), such as Figure 19 As shown in (a), a segmentation is performed, assuming a 16×4 bounding box with a size of 16×16. In the figure, the segmentation progresses to the right, and at each stage, the occupied nodes are divided by solid or dashed lines according to the QT (OT in 3D space). The unit node size at each stage can be represented as 16×4, 8×8, 4×4, 2×2, or 1×1. That is, in Figure 19 In (a), the dashed or solid lines in the bounding box of each stage indicate the nodes at each depth when the segmentation is performed based on QT.

[0315] When Figure 19 As shown in (b), when performing segmentation based on a combination of QT (OT in 3D space) and BT (QT in 3D space), a 16×4 asymmetric bounding box can be used, QT can be performed twice, and BT can be performed twice until a 1×1 leaf node is reached. As an example, in Figure 19In case (b), parameters K and M are set to K=0 and M=0, and the partitioning is performed in this order according to QT, QT, BT, and BT. In this case, the unit node sizes of the corresponding stages can be represented as 16×4, 8×2, 4×1, 2×1, and 1×1.

[0316] The compression of attribute information can be performed by the attribute encoder 51004 based on the level of detail (LoD). In this case, the attribute encoder 51004 can generate the LoD using various methods. In one implementation, the attribute encoder 51004 can generate the LoD based on an octree.

[0317] According to the implementation, LOD indicates the level of detail of point cloud data (or point cloud content). A decrease in the LOD value means a degradation in the detail of the point cloud content, while an increase in the LOD value means an enhancement in the detail of the point cloud content. That is, LOD is configured such that the density of points increases as LOD increases. The points of the geometry reconstructed by the geometry encoder 51003 (that is, the reconstructed locations) can be classified by the attribute encoder 51004 according to the LOD.

[0318] In implementation, when using predictive transform coding and lifting transform coding techniques, points can be grouped by dividing them into Levels of Detail (LODs).

[0319] This operation can be called the LOD generation process, and groups with different LODs can be called a set of LODs. l Here, l represents LOD and is an integer starting from 0. The detail of the point cloud content is lowest at LOD0, and the detail of the point cloud content increases as l increases.

[0320] Figure 20 This is a diagram illustrating an example of LOD generation by an attribute encoder 51004 based on an octree according to an implementation method. Figure 20 In this context, a point can be a point in the captured point cloud or a geometric point reconstructed by the geometric encoder 51003.

[0321] According to the implementation method, when generating a Level of Detail (LOD) based on an octree, each depth level of the octree can be matched with each LOD, such as... Figure 20 As shown. In other words, the octree-based LOD generation method generates LODs based on the following principle: as the depth of the octree structure increases (in the direction from root to leaf), the detail representing the point cloud data increases. According to the implementation, the octree-based LOD configuration can proceed from the root node to the leaf node, or from the leaf node to the root node.

[0322] This example illustrates the arrangement of points at each depth level based on Morton code order, with the first child node selected as the reference or reserved node for the parent node. Reference numeral 50001 indicates an example of a point belonging to LOD Nx. In other words, points belonging to LOD Nx correspond to the number of occupied nodes at the corresponding depth level.

[0323] Figure 21 This is a diagram illustrating an example of the Morton code order in a 3D cube according to an embodiment.

[0324] In other words, a Morton code for each point in the point cloud is generated based on the x, y, and z position values ​​of each point. Once the Morton codes for the points in the point cloud are generated through this process, the points in the point cloud can be sorted according to the order of the Morton codes. Depending on the implementation, the points in the point cloud can be sorted in ascending order of the Morton codes. The order of the points sorted in ascending order of the Morton codes can be called the Morton order.

[0325] According to the implementation, spatial scalability can be supported when the attribute encoder 51004 generates a Level of Detail (LOD) based on the octree described above. Utilizing spatial scalability, when the source point cloud is dense, a lower-resolution point cloud can be accessed as a thumbnail with less decoder complexity and / or less bandwidth. Geometric spatial scalability is provided by adjusting the depth level of the octree during geometric encoding / decoding, through a process of encoding or decoding occupied bits up to a selected depth level. Furthermore, in attribute encoding / decoding, spatial scalability of attributes can be provided through a process of generating an LOD from a selected depth level of the octree and configuring the points for which the attributes will be encoded / decoded.

[0326] In other words, in scalable attribute encoding, the resolution can increase based on the depth of the octree-based geometric encoding. In other words, as the octree is segmented, the detail increases. In this case, scalable representation can be enabled by matching attributes with occupying nodes at each depth (or depth level). To do this, during scalable attribute encoding, octree-based LoD generation is performed, and selection is performed based on cell nodes rather than distance. In this case, child nodes with the same node as the parent node can be defined as neighbors (or neighbor search ranges), and an attribute can be selected within a neighbor node (or point) and matched with the parent node. For example, the attribute of the first child node out of eight child nodes can be selected as the attribute of the parent node.

[0327] Figure 22 This is a diagram illustrating an example of an attribute selection process based on LoD generated for nodes (or points) in a 2D plane, according to an implementation method.

[0328] exist Figure 22 In the diagram, the 'x' marker indicates the location of a point within a 16×4 bounding box. The dashed or solid lines within the bounding box of each stage represent nodes at each depth based on QT-based segmentation at each stage. That is, in... Figure 22 In the middle, the segment moves forward to the right, and the size of the unit node in each stage can be represented as 16×4, 8×8, 4×4, 2×2 or 1×1.

[0329] When based on, for example Figure 22 When performing LoD generation on the segmented tree structure shown, the operation can proceed from the leaf to the root, or from the root to the leaf as needed. The generated LoD can be configured such that the LoD increases in the direction from the root to the leaf. Additionally, the circles at each stage can represent the points that constitute each LoD, and the max LoD (e.g., LoD4) can contain all points. As another example, in Figure 22 In the process of generating LoD2 based on the points that constitute LoD3, the number of points constituting LoD2 is 4, because the number of occupied nodes at this depth level is 4.

[0330] As described above, when using an octree-based geometry compression method, scalable encoding is supported by configuring attributes corresponding to the octree layer structure. In this case, the ratio of the width, depth, and height of the unit nodes used at the depth of the geometry information tree is the same. Moreover, the geometric and attribute information included at a specific octree depth are matched to each other in a one-to-one correspondence. In other words, once the geometry encoder 51003 performs octree-based geometry compression and the attribute encoder 51004 generates a LoD based on the octree and performs attribute compression, the number of points in the geometry is the same as the number of points in the attributes. Therefore, when the point cloud after partial octree decoding is provided to the attribute decoder 61004 on the receiving side, the attribute decoder 61004 can accurately perform scalable attribute decoding from the provided depth level.

[0331] Figure 23 This is a diagram illustrating another example of the attribute selection process generated by the attribute encoder 51004 based on the LoD for a node (or point) in a 2D plane, according to an embodiment.

[0332] exist Figure 23 In the diagram, the 'x' marker indicates the location of a point within a 16×4 bounding box. The dashed or solid lines within the bounding box at each stage represent nodes at each depth based on either QT-based or BT-based segmentation at that stage. That is, Figure 23The diagram illustrates the case where K=1 and M=1, and the segmentation proceeds to the right according to BT, QT, BT, and QT in this order. In other words, when performing BT-based segmentation based on parameters K=1 and M=1, the initial segmentation is performed based on BT, and the segmentation is performed based on QT during leaf node generation. Therefore, the bounding box is segmented according to BT, QT, BT, and QT in this order. In this case, the cell node size at this stage can be represented as 16×4, 8×4, 4×2, 2×2, or 1×1.

[0333] Even when based on, for example Figure 23 When performing LoD generation on the segmented geometric tree structure shown, LoD generation can proceed from the leaf to the root, or, as needed, from the root to the leaf. Furthermore, the generated LoDs can be configured to increase in the direction from the root to the leaf. In this case, the LoDs can be configured with representative points of the nodes at each stage. As an example, circles at each stage can represent the points constituting each LoD, and a max LoD (e.g., LOD 4) can include all points. As another example, in Figure 23 In the process of generating LoD2 based on the points that constitute LoD3, the number of points constituting LoD2 is 5, because the number of occupied nodes at this depth level is 5.

[0334] When BT (BT or QT in 3D space) is used as... Figure 23 When performing geometric encoding as shown, LoD can be configured to have the same characteristics as described above. Figure 22 The number of points in the examples varies depending on the specific depth level of the points. This is because the selected points can be varied according to the neighbor search range setting in the LoD generation. For example, in an octree structure, the neighbor search range (i.e., the number of child nodes) can be 8. In a quadtree structure, the neighbor search range can be 4. In a binary tree structure, the neighbor search range can be 2.

[0335] According to the implementation method, in such Figure 22 In the QT-based LoD configuration shown, the number of points belonging to LoD2 can be 4. On the other hand, in cases such as Figure 23 In the QTBT-based LoD configuration shown, the number of points belonging to LoD2 can be 5. Therefore, when the geometry encoder 51003 compresses geometric information by generating a geometry tree based on OT / QT / BT, but the attribute encoder 51004 configures LoD based on OT, a mismatch of points between the geometry and attributes can occur at a certain depth (e.g., LoD2). In other words, when performing scalable rendering using LoD2, the number of points in the geometry can differ from the number of points in the attributes configured for LOD. Therefore, incorrect attributes can match the geometry, or points with mismatched attributes can exist.

[0336] Therefore, when OT / QT / BT is used to segment bounding boxes to account for asymmetric point distributions in the compression of geometric information, the cell nodes used at each depth of the geometry tree can have different ratios of width, depth, and height. For example, in Figure 23 In this context, the unit nodes at the corresponding stages of the 2D plane are 16×4, 8×4, 4×2, 2×2, and 1×1. In this case, when the layer structure of geometric encoding is not considered in scalable attribute encoding, the scalable point cloud representation may be inaccurate due to the lack of a one-to-one match between the geometric and attribute information of any scalable layer. In other words, when the geometric encoder 51003 compresses geometric information by generating a geometry tree based on OT / QT / BT and the attribute encoder 51004 compresses attribute information by generating a LoD based on OT, the number of points used for geometry may differ from the number of points used for attributes. In this case, the scalable point cloud representation at the corresponding depth may be inaccurate.

[0337] To address this discrepancy, asymmetric neighbor nodes can be used in scalable point cloud encoding to configure a scalable LoD-based layer. This increases the attribute compression efficiency for content with asymmetric point distributions. In other words, while the geometry encoder 51003 performs geometry compression by generating a geometry tree based on OT / QT / BT, the attribute encoder 51004 performs attribute compression by generating a LoD based on OT / QT / BT. Therefore, at each depth, the number of points in the geometry is consistent with the number of points in the attributes.

[0338] This disclosure provides a compression method based on asymmetric node segmentation for scalable point cloud encoding. When using OT / QT / BT-based geometric compression methods, the asymmetric node segmentation-based compression method described in this disclosure can be used as a scalable attribute compression method. The methods proposed in this disclosure can be used not only for scalable point cloud encoding but also for adaptively providing point cloud services in low-latency environments. Furthermore, the prediction-based attribute compression method proposed in this disclosure can be used independently of the geometric compression method and can be used for both geometric compression and attribute compression based on the methods proposed in this disclosure. The LoD generation method proposed in this disclosure can be used not only in scalable encoding applications but also for general point cloud compression.

[0339] Next, scalable attribute encoding methods that consider asymmetric point distributions will be described, including LoD generation methods that consider cuboid nodes and attribute encoding methods that consider node variations based on layers when using OT / QT / BT-based geometric compression, or constraints and signaling for using OT-based scalable attribute compression.

[0340] Implementation Method 1

[0341] When performing geometric encoding by generating a geometry tree based on OT / QT / BT, asymmetric (or cuboid) nodes can be used at each depth of the geometry tree. In this case, scalability is supported depth-by-depth in the geometry tree when scalable encoding is taken into account. Implementation 1 is an example of attribute encoding performed by generating a LoD based on the geometry tree structure. In the embodiments of this disclosure, asymmetric nodes of the geometry tree used for scalable encoding are taken into account when generating the LoD.

[0342] According to the implementation method, firstly, in order to consider the distribution characteristics of points distributed in a specific direction, as an example, neighbor search is performed based on non-cube nodes. According to the implementation method, when the size of the node at each depth is 2 in the geometric tree structure used in the geometric encoding... dx 2 dy and 2 dz At this point, dx, dy, and dz can have different values. However, in the implementation, when (k) is assumed to be the k-th depth, relationships of dx(k) >= dx(k+1), dy(k) >= dy(k+1), and dz(k) >= dz(k+1) can be established. As an example, in octree-based LoD generation, neighbor search can be performed based on cube nodes. In this case, only the case of dx = dy = dz is considered.

[0343] Additionally, criteria for selecting points of the LoD (LoD) for a given neighbor search range at a corresponding depth can be configured. The given neighbor search range at depth depends on whether the operation is based on OT, QT, or BT. For example, the neighbor search range (i.e., the number of child nodes) for an OT-based operation can be 8. The neighbor search range for a QT-based operation can be 4. The neighbor search range for a BT-based operation can be 2. In one implementation, the first point sorted according to the Morton code order (i.e., the first occupied node among the occupied nodes sorted according to the Morton code order) can be selected within the neighbor search range. In another implementation, points at different locations can be selected based on depth. For example, when selecting the first point among the points sorted according to the Morton code order within the neighbor search range for the k-th LoD, the last point among the points sorted according to the Morton code order within the neighbor search range can be selected for the (k-1)-th LoD. In another implementation, a point located at a close distance from the node center can be selected. In another implementation, information identifying the point selection method used at depth (e.g., LoD_sampling_method) can be signaled via signaling information. Signaling information can be at least a set of attribute parameters or an attribute slice header. In this disclosure, for simplicity, information used to identify the point selection method used at depth (e.g., LoD_sampling_method) may be referred to as information used to identify the LoD sampling method.

[0344] Figure 24 (a) and Figure 24 (b) is a diagram illustrating an example of LoD generation based on a geometric tree structure used to segment point cloud data distributed in a 2D plane at each depth.

[0345] Figure 24 (a) and Figure 24 (b) illustrates an example where QT / BT is used to segment bounding boxes when the geometry encoder 51003 performs geometry encoding and the geometry tree is configured in this order based on parameters set to K=0 and M=0 according to QT, QT, BT, and BT. According to an implementation, the attribute encoder 51004 can be based on having Figure 24 LoD generation is performed on a geometric tree structure with non-square nodes. In this case, asymmetric nodes at each depth of the geometric tree can be considered to generate the LoD. For example, when LoD4 is assumed to be the max LoD, LoD4 includes all points of the bounding box. Additionally, for lower LoD (smaller LoD number) configurations, neighbors can be configured among the points constituting the higher LoD (larger LoD) using node cells of the same size as the node at the depth corresponding to the lower LoD, and specific points can be selected from these neighbors. Thus, a lower LoD can be configured. Figure 24In this context, the node size at the corresponding depth can be represented as 16×4, 8×2, 4×1, 2×1, and 1×1. For example, LoD3 can be configured by configuring neighbors (neighbor search range) with nodes (BT) of size 2×1 among the points constituting LoD4 and selecting specific points among the configured neighbors. LoD2 can be configured by configuring neighbors with nodes (BT) of size 4×1 among the points constituting LoD3 and selecting specific points among the configured neighbors. LoD1 can be configured by configuring neighbors with nodes (QT) of size 8×2 among the points constituting LoD2 and selecting specific points among the configured neighbors. LoD0 can be configured by configuring the four points contained in LoD1 as neighbors and selecting specific points from the four configured neighbors.

[0346] Figure 24 (a) illustrates an example of selecting the first point in terms of Morton code during the point selection process for each depth. That is, for LoD3 to LoD0, when selecting a specific point from among the points configured as neighbors, the first point is selected from the neighbor points ordered according to the Morton code.

[0347] Figure 24 Figure (b) illustrates an example of selecting the first or last point in terms of Morton code based on depth during the point selection process. More specifically, the figure illustrates an example of alternately selecting the first and last points in terms of Morton code at consecutive depths during the point selection process. In other words, for LoD3, the first point can be selected from points in the neighbor search range ordered by Morton code. For LoD2, the last point can be selected from points in the neighbor search range ordered by Morton code. For LoD1, the first point can be selected from points in the neighbor search range ordered by Morton code. For LoD0, the last point can be selected from points in the neighbor search range ordered by Morton code.

[0348] and Figure 24 Compared to (a), Figure 24 The point distribution in (b) can be configured such that points with lower LOD are closer to the center of the node. Figure 24 Points 50003 and 50005 in (a) are the first points selected based on Morton codes at the corresponding depths, and Figure 24 Points 50007 and 50009 in (b) are the last points selected based on Morton code at the corresponding depth.

[0349] Figure 25 (a) and Figure 25(b) is a diagram illustrating other examples of LoD generation based on the geometric tree structure used to segment point cloud data distributed on a 2D plane at each depth.

[0350] Figure 25 (a) and Figure 25 (b) illustrates an example where QT / BT is used to segment bounding boxes when the geometry encoder 51003 performs geometry encoding and the geometry tree is configured in this order based on parameters set to K=1 and M=0 according to BT, QT, QT, and BT. According to an implementation, the attribute encoder 51004 can be based on having Figure 25 LoD generation is performed on a geometric tree structure with non-square nodes. In this case, asymmetric nodes at each depth of the geometric tree can be considered to generate the LoD. For example, when LoD4 is assumed to be the max LoD, LoD4 includes all points of the bounding box. Additionally, for lower LoD (smaller LoD number) configurations, neighbors can be configured among the points constituting the higher LoD (larger LoD) using node cells of the same size as the node at the depth corresponding to the lower LoD, and specific points can be selected from these neighbors. Thus, a lower LoD can be configured. Figure 25 In this context, the node size at the corresponding depth can be represented as 16×4, 8×4, 4×2, 2×1, and 1×1. For example, LoD3 can be configured by configuring neighbors (neighbor search range) with 2×1 node (BT) cells among the points constituting LoD4 and selecting a specific point among the configured neighbors. LoD2 can be configured by configuring neighbors with 4×2 node (QT) cells among the points constituting LoD3 and selecting a specific point among the configured neighbors. LoD1 can be configured by configuring neighbors with 8×4 node (QT) cells among the points constituting LoD2 and selecting a specific point among the configured neighbors. LoD0 can be configured by configuring two points contained in LoD1 as neighbors and selecting a specific point from the two configured neighbors.

[0351] Figure 25 (a) illustrates an example of selecting the first point in terms of Morton code during the point selection process for each depth. That is, for LoD3 to LoD0, when selecting a specific point from among the points configured as neighbors, the first point is selected from the neighbor points ordered according to the Morton code.

[0352] Figure 25Figure (b) illustrates an example of selecting the first or last point in terms of Morton code based on depth during the point selection process. More specifically, the figure illustrates an example of alternately selecting the first and last points in terms of Morton code at consecutive depths during the point selection process. In other words, for LoD3, the first point can be selected from points in the neighbor search range ordered by Morton code. For LoD2, the last point can be selected from points in the neighbor search range ordered by Morton code. For LoD1, the first point can be selected from points in the neighbor search range ordered by Morton code. For LoD0, the last point can be selected from points in the neighbor search range ordered by Morton code.

[0353] As in Figure 24 (a) and Figure 24 In case (b), it can be seen that, with Figure 25 Compared to (a), Figure 25 The point distribution in (b) can be configured such that points with lower LOD are closer to the center of the node.

[0354] Implementation Method 2

[0355] When performing geometry encoding by generating a geometry tree based on OT / QT / BT, asymmetric (or cuboid) nodes can be used at each depth of the geometry tree. In this case, scalability is supported depth-by-depth in the geometry tree when considering scalable encoding. Implementation 2 is an example of adding constraints to take into account the characteristics of octree-based LoD generation in geometry encoding. That is, when ensuring cube nodes for a certain depth range from the leaf level (i.e., when the size of the cell nodes is the same in the xyz direction), scalability can be ensured for the corresponding depth range by allowing the attribute encoder 51004 to perform octree-based LoD generation. To this end, in this disclosure, attribute compression can be restricted to be performed only using LoDs generated when the depth near the leaf node is a symmetric node.

[0356] Figure 26 This is a diagram illustrating an example of LoD generation at certain depths in a geometric tree structure used to segment point cloud data distributed in a 2D plane. Figure 26 This example illustrates how QT / BT is used to segment bounding boxes when the geometry encoder 51003 performs geometry encoding and the geometry tree is configured in this order based on BT, QT, BT, and QT, according to parameters set to K=1 and M=1. Figure 26 In this context, it is assumed that two depth levels, including the leaf level, have cube nodes. In this case, as an implementation, the attribute encoder 51004 can generate LoD only at the two consecutive depth levels including the leaf level.

[0357] Figure 27 This is another example of LoD generation at certain depths of a geometric tree structure that segments point cloud data distributed in a 2D plane. Figure 27 This example illustrates how QT / BT is used to segment bounding boxes when the geometry encoder 51003 performs geometry encoding and the geometry tree is configured in this order based on BT, QT, BT, and QT, with parameters set to K=2 and M=0. Figure 27 In this context, it is assumed that the three depth levels, including the leaf level, have cube nodes. In this case, as an implementation, the attribute encoder 51004 can generate the LoD only at the three consecutive depth levels including the leaf level.

[0358] In this disclosure, point mismatch between geometry and attributes can be prevented in scalable rendering by imposing the following constraint: LoD should only be generated for depths that match the QT (OT in 3D space) based LoD generation method in the geometry used for geometry compression.

[0359] In implementation, signaling information can be used to indicate the maximum number of LODs (e.g., max_num_LoD) and / or the starting depth information of LODs (e.g., LoD_starting_depth). The signaling information can be at least a set of attribute parameters or an attribute slice header.

[0360] In other words, the number of LoDs used in scalable encoding can be notified to the attribute decoder 61004 of the receiving device by signaling information indicating the maximum number of LoDs (e.g., max_num_LoD), or the starting position of LoD0 can be notified to the attribute decoder 61004 of the receiving device by signaling LoD starting depth information (e.g., LoD_starting_depth). According to an implementation, the LoD starting depth information (e.g., LoD_starting_depth) can indicate the maximum LoD depth that can match octree-based LoD generation. That is, it can indicate the number of depths from the leaf level. As an example, in Figure 26 In this context, the number of depths indicated by the LoD starting depth information (e.g., LoD_starting_depth) is 2, which corresponds to the depth of LoD0. As an example, in... Figure 27 In this context, the number of depths that can be matched from the leaf level to the depth of the octree-based LoD generation can be 3, which corresponds to the depth of LoD0.

[0361] According to the implementation, the information indicating the maximum number of LODs (e.g., max_num_LoD) and the LoD starting depth information (e.g., LoD_starting_depth) may have the following values.

[0362] max_num_LoD=max_dimension-min_dimension-M

[0363] LoD_starting_depth=max_dimension-M

[0364] Additionally, a constraint can be added that K+M = max_dimension - min_dimension should hold when the geometry encoding is performed by the geometry encoder 51003.

[0365] According to the implementation method, max_dimension represents max(dx,dy,dz), and min_dimension represents min(dx,dy,dz).

[0366] Implementation Method 3

[0367] When performing geometry encoding by generating a geometry tree based on OT / QT / BT, asymmetric (or cuboid) nodes can be used at each depth of the geometry tree. When considering scalable encoding, scalability is supported depth-by-depth within the geometry tree. Implementation 3 is an example where LoD generation is skipped at the depth where a point mismatch occurs between the geometry and attributes of the geometry tree, or where the LoD generated at said depth is not used in attribute compression, and information indicating the skip (e.g., LoD_skip_flag) is signaled via signaling information. The signaling information can be at least a set of attribute parameters or an attribute slice header.

[0368] In one implementation, the generation of LoDs corresponding to the depths in the geometry tree where a point mismatch occurs between geometry and attributes can be skipped, and LoD skipping information (e.g., LoD_skip_flag) indicating the skipped LoDs can be signaled and communicated to the attribute decoder 61004 of the receiving device. In this implementation, considering asymmetric nodes at each depth, LoDs can be generated at depths other than the depth where the mismatch occurs. For example, a LoD skipping value of 1 (e.g., LoD_skip_flag) can indicate a point mismatch between geometry and attributes at a depth in the geometry tree, and / or that LoD generation has been skipped at that depth. A LoD skipping value of 0 (e.g., LoD_skip_flag) can indicate that the point matches between geometry and attributes at that depth, and / or that a LoD has been generated at that depth.

[0369] In another embodiment, the LoD generated at a depth in the geometry tree (where a point mismatch occurs between geometry and attribute) may not be used for attribute compression, and LoD skipping information indicating skipping (e.g., LoD_skip_flag) can be signaled and notified to the attribute decoder 61004 of the receiving device. In this case, in the embodiment, considering asymmetric nodes at each depth, attribute compression can be performed using LoDs generated at depths other than the depth where the mismatched geometry occurs. For example, LoD skipping information equal to 1 (e.g., LoD_skip_flag) may indicate a point mismatch between geometry and attribute at a depth in the geometry tree, and / or the LoD generated at that depth is not used for attribute compression. LoD skipping information equal to 0 (e.g., LoD_skip_flag) may indicate a point match between geometry and attribute at that depth, and / or the LoD generated at that depth is used for attribute compression.

[0370] In other words, when the value of the LoD skip information (e.g., LoD_skip_flag) is 1, the LoD corresponding to that depth may not be used for attribute compression. Alternatively, LoDs can be defined for depth, but may not be used for scalable rendering.

[0371] According to the implementation, the setting of LoD skip information (e.g., LoD_skip_flag) can meet the following conditions.

[0372] if dx_ot=dx&dy_ot=dy&dz_ot=dz LoD_skip_flag=0

[0373] else LoD_skip_flag = 1

[0374] Here, dx_ot, dy_ot, and dz_ot can represent the xyz axis node sizes in the bounding box, where the points are actually distributed when using OT-based segmentation, and dx, dy, and dz can represent the xyz axis node sizes when using QTBT-based segmentation.

[0375] In other words, the LoD skipping information (e.g., LoD_skip_flag) can only be set to 0 if the x-axis node size (dx_ot) of the segmentation using OT-based methods is the same as the x-axis node size (dx) in the bounding box of the segmentation using QTBT-based methods, the y-axis node size (dy_ot) of the segmentation using OT-based methods is the same as the y-axis node size (dy) of the segmentation using QTBT-based methods, and the z-axis node size (dz_ot) of the segmentation using OT-based methods is the same as the z-axis node size (dz) of the segmentation using OTBT-based methods. Furthermore, if the sizes are not the same on any of the x, y, or z axes, the LoD skipping information (e.g., LoD_skip_flag) can be set to 1.

[0376] In other words, when skipping LoD, a hierarchical structure is provided through scalable encoding, but LoD does not structurally match the geometry. Therefore, LoD skipping information (e.g., LoD_skip_flag) can be used to notify that the output (or LoD) is unsuitable for attribute compression.

[0377] Figure 28 (a) and Figure 28 (b) is a diagram illustrating an example of the LoD skipping settings at certain depths in a geometric tree structure used to segment point cloud data distributed in a 2D plane, according to the conditions described above. Figure 28 (a) and Figure 28 (b) illustrates an example of QT / BT being used to segment bounding boxes when the geometry encoder 51003 performs geometry encoding and the geometry tree is configured in this order based on BT, QT, BT, and QT according to parameters set to K=1 and M=1. Figure 28 (a) illustrates an example of setting LoD skipping for the third depth based on the root node, and Figure 28 Example (b) illustrates an example of setting LoD skipping for three depths based on the root node.

[0378] According to the implementation method, in Figure 28 In (a), LoD0 and LoD1 represent the case where a cuboid node is contained within a cube node. In this case, because they match the geometry, it is possible to set not to skip LoD.

[0379] Figure 29 This is another example of a setting that illustrates the LoD skipping at certain depths in a geometric tree structure used to segment point cloud data distributed in a 2D plane, based on the conditions described above. Figure 29This example illustrates how QT / BT is used to segment bounding boxes when the geometry encoder 51003 performs geometry encoding and the geometry tree is configured in this order based on BT, BT, QT, and QT, with parameters set to K=2 and M=0. Figure 29 In the example, LoD is set to skip for both depths.

[0380] Figure 30 This is another example of LoD generation based on a geometric tree structure used to segment point cloud data distributed in a 2D plane at each depth. Figure 30 An example is illustrated where QT / BT is used to segment bounding boxes when the geometry encoder 51003 performs geometry encoding and the geometry tree is configured in this order based on BT, BT, QT, and QT according to parameters set to K=2 and M=0. According to an implementation, the attribute encoder 51004 can be based on having Figure 30 The geometric tree structure of non-cube nodes is used to perform LoD generation.

[0381] Depending on the implementation, LoD skip information (e.g., LoD_skip_flag) can also be applied. Figure 30 The geometric tree structure shown. In other words, it can be seen that... Figure 30 Each depth of the geometry has the same node size as the node size in the QT-based LoD generation method. Therefore, when the attribute encoder 51004 pre-identifies this feature, it can signal the LoD skip information (e.g., LoD_skip_flag) so that the LoD skip information (e.g., LoD_skip_flag) is set to 0 for all LoDs.

[0382] According to the embodiments, the geometric encoder 51003 and / or attribute encoder 51004 of the transmitting device can perform geometric information compression and / or attribute information compression or perform scalable attribute encoding by combining one or more of the embodiments 1 to 3 described above. Additionally, the geometric decoder 61003 and / or attribute decoder 61004 of the receiving device can perform geometric information reconstruction and / or attribute information reconstruction or perform scalable attribute decoding by combining one or more of the embodiments 1 to 3 described above.

[0383] Next, signaling information for signaling information related to the aforementioned LoD-based attribute compression, and a bitstream structure including such signaling information, will be described. According to an embodiment, the information related to LoD-based attribute compression may include at least one of the following: information identifying the LoD sampling method (e.g., LoD_sampling_method), information indicating the maximum number of LODs (e.g., max_num_LoD), LoD starting depth information (e.g., LoD_starting_depth), or LOD skipping information (e.g., LoD_skip_flag). The information related to LoD-based attribute compression may be referred to as information related to LoD-based attribute encoding.

[0384] According to the configuration, when geometric information is encoded by a point cloud video encoder, a geometric bitstream is generated as a result. When attribute information is encoded, an attribute bitstream is generated as a result. The geometric bitstream and attribute bitstream according to the embodiment can be multiplexed and output as a single bitstream. The bitstream according to the embodiment may also include signaling information related to geometric encoding and attribute encoding. In the embodiment, the signaling information may include information related to LoD-based attribute compression.

[0385] Figure 31 An example of a bitstream structure for sending / receiving point cloud data according to an embodiment is shown.

[0386] When the geometric bitstream, attribute bitstream, and / or signaling bitstream (or signaling information) according to the embodiment are configured as a single bitstream (or G-PCC bitstream), the bitstream may include one or more sub-bitstreams. The bitstream according to the embodiment may include a sequence parameter set (SPS) for sequence-level signaling, a geometric parameter set (GPS) for signaling encoded with geometric information, one or more attribute parameter sets (APS) (APS0, APS1) for signaling encoded with attribute information, a tile parameter set (TPS or tile list) for tile-level signaling, and one or more tiles (tile 0 to tile n). That is, the bitstream of point cloud data according to the embodiment may include one or more tiles, and each tile may be a set of tiles including one or more tiles (tile 0 to tile n). The image list (or TPS) according to the embodiment may contain information about each of the one or more tiles (e.g., coordinate values ​​and height / size information about the tile bounding box). Each tile may include a geometric bitstream (Geom0) and one or more attribute bitstreams (Attr0 and Attr1). For example, the first slice (slice 0) may include a geometric bitstream (Geom0). 0 ) and one or more attribute bitstreams (Attr) 0Attr1 0 ).

[0387] The geometric bitstream in each slice can consist of a geometric slice header and geometric slice data. According to an implementation, the geometric slice header may include identification information of the parameter set included in the GPS (geom_parameter_set_id), tile identifier (geom_tile_id), slice identifier (geom_slice_id), and information about the data contained in the geometric slice data (geom_slice_data) (geomBoxOrigin, geom_box_log2_scale, geom_max_node_size_log2, geom_num_points). geomBoxOrigin is the origin information of the geometric box indicating the origin of the geometric slice data frame; geom_box_log2_scale is the log scale information indicating the geometric slice data frame; geom_max_node_size_log2 is the information indicating the size of the root geometric octree node; and geom_num_points is information related to the number of points in the geometric slice data frame. According to the implementation method, geom_slice_data may include geometric information (or geometric data) about the point cloud data in the corresponding slice.

[0388] Each attribute bitstream within a slice can consist of an attribute slice header (attr_slice_header) and attribute slice data (attr_slice_data). Depending on the implementation, the attr_slice_header may include information about the corresponding attribute slice data. The attribute slice data may contain attribute information (or attribute data) about the point cloud data in the corresponding slice. When multiple attribute bitstreams exist within a slice, each bitstream may contain different attribute information. For example, one attribute bitstream may contain attribute information corresponding to color, while another attribute stream may contain attribute information corresponding to reflectivity.

[0389] Figure 32 An exemplary bitstream structure for sending / receiving point cloud data according to an embodiment is illustrated.

[0390] Figure 33 The connection relationships between elements in a bitstream of point cloud data according to an embodiment are illustrated.

[0391] Figure 32 and Figure 33The bitstream structure of the point cloud data illustrated in the example can be represented as Figure 31 The bitstream structure of the point cloud data is shown in the figure.

[0392] According to an implementation, the SPS may include an identifier (seq_parameter_set_id) for identifying the SPS, and the GPS may include an identifier (geom_parameter_set_id) for identifying the GPS and an identifier (seq_parameter_set_id) indicating the active SPS to which the GPS belongs. The APS may include an identifier (attr_parameter_set_id) for identifying the APS and an identifier (seq_parameter_set_id) indicating the active SPS to which the APS belongs. According to an implementation, the geometric data may include a geometric slice header and geometric slice data. The geometric slice header may include an identifier (geom_parameter_set_id) of the active GPS referenced by the corresponding geometric slice. Furthermore, the geometric slice header may also include an identifier (geom_slice_id) for identifying the corresponding geometric slice and / or an identifier (geom_tile_id) for identifying the corresponding tile. The geometric slice data may include geometric information belonging to the corresponding slice. According to an implementation, the attribute data may include an attribute slice header and attribute slice data. The attribute slice header may include an identifier (attr_parameter_set_id) of the active APS referenced by the corresponding attribute slice and an identifier (geom_slice_id) for identifying the geometric slice associated with the attribute slice. The attribute slice data may include the attribute bitstream belonging to the corresponding slice.

[0393] In other words, the geometry slice references the GPS, and the GPS references the SPS. Additionally, the SPS lists the available attributes, assigns an identifier to each attribute, and identifies the decoding method. Based on the identifier, the attribute slices are mapped to the output attributes. The attribute slices depend on the preceding (decoded) geometry slices and the APS. The APS references the SPS.

[0394] According to the implementation method, parameters necessary for encoding the point cloud data can be newly defined in the parameter set and / or the corresponding tile header of the point cloud data. For example, when encoding attribute information, parameters can be added in the APS. When performing tile-based encoding, parameters can be added in the tile and / or tile header.

[0395] like Figures 31 to 33As shown, the bitstream of point cloud data provides tiles or slices, allowing the point cloud data to be segmented and processed by region. Depending on the implementation, corresponding regions of the bitstream can have different importance. Therefore, when point cloud data is segmented into tiles, different filters (encoding methods) and different filter units can be applied to each tile. When point cloud data is segmented into slices, different filters and different filter units can be applied to each slice.

[0396] When point cloud data is segmented and compressed, the transmitting and receiving devices according to the implementation can transmit and receive bit streams in a high-level syntax structure to selectively transmit attribute information in the segmented regions.

[0397] The transmitting device according to the embodiment can be based on Figures 31 to 33 The bitstream structure shown transmits point cloud data. Therefore, it provides a method to apply different encoding operations to important regions and use high-quality encoding methods. Furthermore, it supports efficient encoding and transmission based on the characteristics of point cloud data, and attribute values ​​can be provided according to user requirements.

[0398] The receiving device according to the embodiment can be based on Figures 31 to 33 The bitstream structure shown receives point cloud data. Therefore, different filtering (decoding) methods can be applied to specific regions (areas segmented into tiles or slices), rather than applying complex decoding (filtering) methods to the entire point cloud data. This provides the user with better image quality in important areas and ensures appropriate system latency.

[0399] Fields of terms used in the syntax of this disclosure, as described later, may have the same meaning as parameters or elements.

[0400] Figure 34 An implementation of the syntax structure of the Sequence Parameter Set (SPS) (seq_parameter_set()) according to this disclosure is shown. The SPS may contain sequence information about the bit stream of point cloud data.

[0401] The SPS, depending on the implementation, may include the following fields: main_profile_compatibility_flag, unique_point_positions_constraint_flag, level_idc, sps_seq_parameter_set_id, sps_bounding_box_present_flag, sps_source_scale_factor_numerator_minus1, sps_source_scale_factor_denominator_minus1, sps_num_attribute_sets, log2_max_frame_idx, axis_coding_order, sps_bypass_stream_enabled_flag, and sps_extension_flag.

[0402] The `main_profile_compatibility_flag` field indicates whether the bitstream conforms to the main profile. For example, a `main_profile_compatibility_flag` value of 1 indicates that the bitstream conforms to the main profile. Conversely, a `main_profile_compatibility_flag` value of 0 indicates that the bitstream conforms to a profile different from the main profile.

[0403] When `unique_point_positions_constraint_flag` equals 1, all output points can have unique positions in each point cloud frame referenced by the current SPS. When `unique_point_positions_constraint_flag` equals 0, two or more output points can have the same position in any point cloud frame referenced by the current SPS. For example, even when all points are unique within their respective slices, slices in a frame can overlap with other points. In this case, setting `unique_point_positions_constraint_flag` to 0 is recommended.

[0404] level_idc indicates the level to which the bitstream conforms.

[0405] sps_seq_parameter_set_id provides an identifier for the SPS referenced by other syntax elements.

[0406] The sps_bounding_box_present_flag field indicates whether the bounding box exists in the SPS. For example, a sps_bounding_box_present_flag of 1 indicates that the bounding box exists in the SPS, and a sps_bounding_box_present_flag of 0 indicates that the size of the bounding box is undefined.

[0407] According to the implementation method, when sps_bounding_box_present_flag equals 1, SPS may also include the fields sps_bounding_box_offset_x, sps_bounding_box_offset_y, sps_bounding_box_offset_z, sps_bounding_box_offset_log2_scale, sps_bounding_box_size_width, sps_bounding_box_size_height, and sps_bounding_box_size_depth.

[0408] sps_bounding_box_offset_x indicates the x-offset of the source bounding box in Cartesian coordinates. When there is no x-offset of the source bounding box, the value of sps_bounding_box_offset_x is 0.

[0409] sps_bounding_box_offset_y indicates the y-offset of the source bounding box in Cartesian coordinates. When there is no y-offset of the source bounding box, the value of sps_bounding_box_offset_y is 0.

[0410] sps_bounding_box_offset_z indicates the z-offset of the source bounding box in Cartesian coordinates. When there is no z-offset of the source bounding box, the value of sps_bounding_box_offset_z is 0.

[0411] sps_bounding_box_offset_log2_scale indicates the scaling factor used to scale the quantized x, y, and z source bounding box offsets.

[0412] sps_bounding_box_size_width indicates the width of the source bounding box in Cartesian coordinates. When the width of the source bounding box does not exist, the value of sps_bounding_box_size_width can be 1.

[0413] sps_bounding_box_size_height indicates the height of the source bounding box in Cartesian coordinates. When the height of the source bounding box does not exist, the value of sps_bounding_box_size_height can be 1.

[0414] sps_bounding_box_size_depth indicates the depth of the source bounding box in Cartesian coordinates. When no source bounding box depth exists, the value of sps_bounding_box_size_depth can be 1.

[0415] sps_source_scale_factor_numerator_minus1 plus 1 indicates the scale factor numerator of the source point cloud.

[0416] sps_source_scale_factor_denominator_minus1 plus 1 indicates the scale factor denominator of the source point cloud.

[0417] sps_num_attribute_sets indicates the number of encoded attributes in the bitstream.

[0418] The SPS according to the implementation includes an iterative statement that is repeated as many times as the value of the sps_num_attribute_sets field. In the implementation, i is initialized to 0 and incremented by 1 each time the iterative statement is executed. The iterative statement is repeated until the value of i becomes equal to the value of the sps_num_attitude_sets field. The iterative statement may include an attribute_dimension_minus1[i] field and an attribute_instance_id[i] field. The increment of attribute_dimension_minus1[i] indicates the number of components of the i-th attribute.

[0419] The attribute_instance_id[i] field specifies the instance ID of the i-th attribute.

[0420] According to the implementation method, when the value of the attribute_dimension_minus1[i] field is greater than 1, the iteration statement may also include the attribute_secondary_bitdepth_minus1[i] field, the attribute_cicp_colour_primaries[i] field, the attribute_cicp_transfer_characteristics[i] field, the attribute_cicp_matrix_coeffs[i] field, and the attribute_cicp_video_full_range_flag[i] field.

[0421] The attribute_secondary_bitdepth_minus1[i] increments by 1 to specify the bit depth of the secondary component of the i-th attribute signal.

[0422] attribute_cicp_colour_primaries[i] indicates the chromaticity coordinates of the primary color source of the i-th attribute.

[0423] attribute_cicp_transfer_characteristics[i] can be either a reference photoelectric transfer characteristic function indicating the color attribute as a function of the source input linear light intensity with a nominal real value range of 0 to 1, or the inverse of a reference electro-optical conversion characteristic function indicating the output linear light intensity.

[0424] attribute_cicp_matrix_coeffs[i] describes the matrix coefficients used to derive the luminance and chromaticity signals from the green, blue, and red or Y, Z, and X primary colors of the i-th attribute.

[0425] The attribute_cicp_video_full_range_flag[i] specifies the black level and range of the luminance and chrominance signals derived from the real-valued component signals of the i-th attribute, namely E'Y, E'PB, and E'PR or E'R, E'G, and E'B.

[0426] The `known_attribute_label_flag[i]` field indicates whether to signal the `known_attribute_label[i]` field or the `attribute_label_four_bytes[i]` field for the `i`th attribute. For example, when `known_attribute_label_flag[i]` equals 0, this indicates that the `known_attribute_label[i]` field should be signaled for the `i`th attribute. When `known_attribute_label_flag[i]` equals 1, this indicates that the `attribute_label_four_bytes[i]` field should be signaled for the `i`th attribute.

[0427] `known_attribute_label[i]` specifies the type of the i-th attribute. For example, `known_attribute_label[i]` equal to 0 can specify that the i-th attribute is color. `known_attribute_label[i]` equal to 1 can specify that the i-th attribute is reflectivity. `known_attribute_label[i]` equal to 2 can specify that the i-th attribute is frame index. Furthermore, `known_attribute_label[i]` equal to 4 can specify that the i-th attribute is transparency. `known_attribute_label[i]` equal to 5 can specify that the i-th attribute is normals.

[0428] The attribute_label_four_bytes[i] indicates the known attribute type with 4-byte codes.

[0429] According to the implementation, an attribute_label_four_bytes[i] equal to 0 can indicate that the i-th attribute is color. An attribute_label_four_bytes[i] equal to 1 can indicate that the i-th attribute is reflectivity. An attribute_label_four_bytes[i] equal to 2 can indicate that the i-th attribute is frame index. An attribute_label_four_bytes[i] equal to 4 can indicate that the i-th attribute is transparency. An attribute_label_four_bytes[i] equal to 5 can indicate that the i-th attribute is normal.

[0430] log2_max_frame_idx indicates the number of bits used to signal the syntax variable frame_idx.

[0431] axis_coding_order specifies the correspondence between the X, Y, and Z output axis labels and the three positional components in the reconstructed point cloud RecPic[pointidx][axis], where axis = 0··2.

[0432] A value of 1 for `sps_bypass_stream_enabled_flag` specifies that bypass encoding mode can be used when reading the bitstream. As another example, a value of 0 for `sps_bypass_stream_enabled_flag` specifies that bypass encoding mode is not used when reading the bitstream.

[0433] `sps_extension_flag` indicates whether the `sps_extension_data` syntax structure exists in the SPS syntax structure. For example, `sps_extension_present_flag` equal to 1 indicates that the `sps_extension_data` syntax structure exists in the SPS syntax structure. `sps_extension_present_flag` equal to 0 indicates that the syntax structure does not exist.

[0434] When the value of the sps_extension_flag field is 1, the SPS according to the implementation may also include the sps_extension_data_flag field.

[0435] sps_extension_data_flag can have any value.

[0436] Figure 35 An implementation of the syntax structure for GPS(geometry_parameter_set()) according to this disclosure is shown. GPS may include information about methods for encoding geometric information of point cloud data included in one or more slices.

[0437] According to the implementation method, GPS may include the following fields: gps_geom_parameter_set_id, gps_seq_parameter_set_id, gps_box_present_flag, unique_geometry_points_flag, geometry_planar_mode_flag, geometry_angular_mode_flag, neighbor_context_restriction_flag, inferred_direct_coding_mode_enabled_flag, bitwise_occupancy_coding_flag, adjacent_child_contextualization_enabled_flag, log2_neighbour_avail_boundary, log2_intra_pred_max_node_size, log2_trisoup_node_size, geom_scaling_enabled_flag, gps_implicit_geom_partition_flag, and gps_extension_flag.

[0438] The gps_geom_parameter_set_id field provides an identifier for the GPS that is referenced by other syntax elements.

[0439] The gps_seq_parameter_set_id field specifies the value of sps_seq_parameter_set_id for the active SPS.

[0440] The `gps_box_present_flag` field specifies whether to provide additional bounding box information in the geometry tile header referencing the current GPS. For example, a `gps_box_present_flag` field equal to 1 can specify that additional bounding box information is provided in the geometry tile header referencing the current GPS. Therefore, when the `gps_box_present_flag` field is equal to 1, the GPS can also include the `gps_gsh_box_log2_scale_present_flag` field.

[0441] The `gps_gsh_box_log2_scale_present_flag` field specifies whether to signal the `gps_gsh_box_log2_scale` field in each geometry slice header referencing the current GPS. For example, a `gps_gsh_box_log2_scale_present_flag` field equal to 1 can specify that the `gps_gsh_box_log2_scale` field is signaled in each geometry slice header referencing the current GPS. As another example, a `gps_gsh_box_log2_scale_present_flag` field equal to 0 can specify that the `gps_gsh_box_log2_scale` field is not signaled in each geometry slice header, but the common scale for all slices is signaled in the current GPS's `gps_gsh_box_log2_scale` field.

[0442] When the gps_gsh_box_log2_scale_present_flag field is equal to 0, GPS can also include the gps_gsh_box_log2_scale field.

[0443] The gps_gsh_box_log2_scale field indicates a common scaling factor for the origin of the bounding box of all slices referencing the current GPS.

[0444] The `unique_geometry_points_flag` indicates whether all output points have a unique location within one of all tiles referencing the current GPS. For example, a `unique_geometry_points_flag` of 1 indicates that all output points have a unique location within the tile referencing the current GPS. A `unique_geometry_points_flag` of 0 indicates that two or more output points may have the same location within the tile referencing the current GPS.

[0445] The `geometry_planar_mode_flag` field indicates whether planar coding mode is active. For example, a `geometry_planar_mode_flag` value of 1 indicates that planar coding mode is active, while a `geometry_planar_mode_flag` value of 0 indicates that planar coding mode is not active.

[0446] When the value of the geometry_planar_mode_flag field is 1 (that is, "true"), GPS may also include the geom_planar_mode_th_idcm field, the geom_planar_mode_th[1] field and the geom_planar_mode_th[2] field.

[0447] The geom_planar_mode_th_idcm field can specify the activation threshold value for the direct encoding mode.

[0448] The value of geom_planar_mode_th[i] for i in the range of 0…2 is valid for specifying the threshold value for activation along the i-th most probable direction used for the planar coding mode.

[0449] The `geometry_angular_mode_flag` indicates whether the angle encoding mode is active. For example, a `geometry_angular_mode_flag` field equal to 1 indicates that the angle encoding mode is active, while a `geometry_angular_mode_flag` field equal to 0 indicates that the angle encoding mode is not active.

[0450] When the value of the geometry_angular_mode_flag field is 1 (that is, true), GPS can also include the lidar_head_position[0] field, lidar_head_position[1] field, lidar_head_position[2] field, number_lasers field, planar_buffer_disabled field, implicit_qtbt_angular_max_node_min_dim_log2_to_split_z field, and implicit_qtbt_angular_max_diff_to_split_zfield field.

[0451] The lidar_head_position[0], lidar_head_position[1], and lidar_head_position[2] fields can specify the (X,Y,Z) coordinates of the lidar head in a coordinate system with internal axes.

[0452] number_lasers specifies the number of lasers used for the angle encoding mode.

[0453] The GPS according to the implementation includes an iterative statement that is repeated as many times as the value of the number_lasers field. In the implementation, i is initialized to 0 and incremented by 1 each time the iterative statement is executed. The iterative statement is repeated until the value of i becomes equal to the value of the number_lasers field. The iterative statement may include a laser_angle[i] field and a laser_correction[i] field.

[0454] laser_angle[i] specifies the elevation angle of the i-th laser relative to the tangent of the horizontal plane defined by the 0th and first internal axes.

[0455] laser_correction[i] specifies the correction of the i-th laser position relative to lidar_head_position[2] along the second inner axis.

[0456] A planar_buffer_disabled value of 1 indicates that using a buffer to track the nearest node is not used during the process of encoding the planar mode flag and planar position in planar mode. A planar_buffer_disabled value of 0 indicates that using a buffer to track the nearest node is permitted.

[0457] implicit_qtbt_angular_max_node_min_dim_log2_to_split_z specifies the log2 value of the node size. If the log2 value is lower than the node size, the horizontal split of the node is preferred over the vertical split.

[0458] implicit_qtbt_angular_max_diff_to_split_z specifies the log2 value of the maximum vertical to horizontal node size ratio allowed for a node.

[0459] A `neighbour_context_restriction_flag` equal to 0 indicates that the geometric node occupancy of the current node is encoded using the context determined by neighboring nodes located inside the current node's parent node. A `neighbour_context_restriction_flag` equal to 1 indicates that the geometric node occupancy of the current node is encoded using the context determined by neighboring nodes located inside or outside the current node's parent node.

[0460] The `inferred_direct_coding_mode_enabled_flag` field indicates whether the `direct_mode_flag` field exists in the geometry node syntax. For example, an `inferred_direct_coding_mode_enabled_flag` field equal to 1 indicates that the `direct_mode_flag` field can exist in the geometry node syntax. Conversely, an `inferred_direct_coding_mode_enabled_flag` field equal to 0 indicates that the `direct_mode_flag` field does not exist in the geometry node syntax.

[0461] The `bitwise_occupancy_coding_flag` field indicates whether geometry node occupancy is encoded using bitwise contextualization of the syntax element occupancy map. For example, a `bitwise_occupancy_coding_flag` field equal to 1 indicates that geometry node occupancy is encoded using bitwise contextualization of the syntax element `occupancy_map`. Conversely, a `bitwise_occupancy_coding_flag` field equal to 0 indicates that geometry node occupancy is encoded using dictionary-encoded syntax elements `occupancy_byte`.

[0462] The `adjacent_child_contextualization_enabled_flag` field indicates whether the adjacent child nodes of an adjacent octree node are used for bitwise contextualization. For example, a `adjacent_child_contextualization_enabled_flag` value of 1 indicates that the adjacent child nodes of an adjacent octree node are used for bitwise contextualization. Conversely, a `adjacent_child_contextualization_enabled_flag` value of 0 indicates that the child nodes of an adjacent octree node are not used for contextualization. The `log2_neighbour_avail_boundary` field specifies the value of the variable `NeighbAvailBoundary` used during decoding.

[0463] For example, when the `neighbour_context_restriction_flag` field equals 1, `NeighbAvailabilityMask` can be set to 1. Similarly, when the `neighbour_context_restriction_flag` field equals 0, `NeighbAvailabilityMask` can be set to 1. <log2_neighbour_avail_boundary。

[0464] The log2_intra_pred_max_node_size field specifies the size of the octree nodes that are eligible to be used for intra-frame prediction.

[0465] The log2_trisoup_node_size field specifies the TrisoupNodeSize variable as the size of the triangle node.

[0466] The `geom_scaling_enabled_flag` indicates whether scaling for the geometric position is applied during the geometry slicing decoding process. For example, a `geom_scaling_enabled_flag` of 1 specifies that scaling for the geometric position is applied during the geometry slicing decoding process. A `geom_scaling_enabled_flag` of 0 specifies that scaling is not required for the geometric position.

[0467] geom_base_qp indicates the base value of the geometric position quantization parameter.

[0468] `gps_implicit_geom_partition_flag` indicates whether implicit geometric partitioning is enabled for a sequence or slice. For example, a value of 1 specifies that implicit geometric partitioning is enabled for a sequence or slice. A value of 0 specifies that implicit geometric partitioning is disabled for a sequence or slice. When `gps_implicit_geom_partition_flag` is equal to 1, the following two fields are signaled: `gps_max_num_implicit_qtbt_prior_ot` and `gps_min_size_implicit_qtbt`.

[0469] `gps_max_num_implicit_qtbt_prior_ot` specifies the maximum number of implicit QT and BT partitions before the OT partition. Then, the variable `K` is initialized by `gps_max_num_implicit_qtbt_before_ot` as follows.

[0470] K=gps_max_num_implicit_qtbt_before_ot.

[0471] `gps_min_size_implicit_qtbt` specifies the minimum size for implicit QT and BT partitions. Then, the variable `M` is initialized by `gps_min_size_implicit_qtbt` as follows.

[0472] M=gps_min_size_implicit_qtbt

[0473] The `gps_extension_flag` indicates whether the `gps_extension_data` syntax structure exists in the GPS syntax structure. For example, a `gps_extension_flag` equal to 1 indicates that the `gps_extension_data` syntax structure exists in the GPS syntax. Conversely, a `gps_extension_flag` equal to 0 indicates that the `gps_extension_data` syntax structure does not exist in the GPS syntax.

[0474] When gps_extension_flag equals 1, the GPS according to the implementation method may also include the gps_extension_data_flag field.

[0475] The `gps_extension_data_flag` flag can have any value. Its presence and value do not affect the decoder's compliance with the configuration file.

[0476] Figure 36 An implementation of the syntax structure of the attribute parameter set (APS) (attribute_parameter_set()) according to this disclosure is shown. The APS according to the implementation may contain information about methods for encoding attribute information about point cloud data contained in one or more slices.

[0477] The APS according to the implementation may include the fields aps_attr_parameter_set_id, aps_seq_parameter_set_id, attr_coding_type, aps_attr_initial_qp, aps_attr_chroma_qp_offset, aps_slice_qp_delta_present_flag, and aps_extension_flag.

[0478] The aps_attr_parameter_set_id field provides an identifier for the APS referenced by other syntax elements.

[0479] The aps_seq_parameter_set_id field specifies the value of sps_seq_parameter_set_id used for the active SPS.

[0480] The attr_coding_type field indicates the encoding type of the attribute.

[0481] According to the implementation method, an attr_coding_type field equal to 0 can indicate predicted weight boosting as the encoding type. An attr_coding_type field equal to 1 can indicate RAHT as the encoding type. An attr_coding_type field equal to 2 can indicate fixed weight boosting.

[0482] The aps_attr_initial_qp field specifies the initial value of the variable SliceQp for each slice referencing APS.

[0483] The aps_attr_chroma_qp_offset field specifies the offset relative to the initial quantization parameters signaled by the syntax aps_attr_initial_qp.

[0484] The `aps_slice_qp_delta_present_flag` field specifies whether the `ash_attr_qp_delta_luma` and `ash_attr_qp_delta_chroma` syntax elements exist in the Attribute Slice Header (ASH). For example, an `aps_slice_qp_delta_present_flag` field equal to 1 indicates that the `ash_attr_qp_delta_luma` and `ash_attr_qp_delta_chroma` syntax elements exist in the ASH. Conversely, an `aps_slice_qp_delta_present_flag` field equal to 0 indicates that the `ash_attr_qp_delta_luma` and `ash_attr_qp_delta_chroma` syntax elements do not exist in the ASH.

[0485] When the value of the attr_coding_type field is 0 or 2 (that is, the encoding type is predictive weight boosting or fixed weight boosting), the APS according to the implementation may also include the lifting_num_pred_nearest_neighbours_minus1 field, the lifting_search_range_minus1 field, and the lifting_neighbour_bias[k] field.

[0486] Increasing `lifting_num_pred_nearest_neighbours` by 1 specifies the maximum number of nearest neighbors to be used for prediction. In some implementations, the value of `NumPredNearestNeighbours` is set to be equal to `lifting_num_pred_nearest_neighbours`.

[0487] Increasing 1 to lifting_search_range_minus1 specifies the search range used to determine the nearest neighbors to be used for prediction and to establish the distance-based hierarchy of details (LOD). The variable LiftingSearchRange, which specifies the search range, can be obtained by adding 1 to the value of the lifting_search_range_minus1 field (LiftingSearchRange = lifting_search_range_minus1 + 1).

[0488] The lifting_neighbour_bias[k] field specifies the bias used to weight the k-th component when calculating the Euclidean distance between two points as part of the nearest neighbor derivation process.

[0489] When the value of the attr_coding_type field is 2, that is, when the encoding type indicates a fixed weight boost, the APS according to the implementation may also include the lifting_scalability_enabled_flag field.

[0490] The `lifting_scalability_enabled_flag` field specifies whether the attribute decoding process allows the input of a pruned octree decoded result for the geometry. For example, a `lifting_scalability_enabled_flag` value of 1 indicates that the attribute decoding process allows the input of a pruned octree decoded result for the geometry. A `lifting_scalability_enabled_flag` value of 0 indicates that the attribute decoding process requires the input of a full octree decoded result for the geometry.

[0491] According to the implementation method, when the value of the lifting_scalability_enabled_flag field is false, the APS may also include the lifting_num_detail_levels_minus1 field.

[0492] The `lifting_num_detail_levels_minus1` field specifies the number of levels of detail used for attribute encoding. The variable `LevelDetailCount`, which specifies the number of Levels of Detail (LODs), can be obtained by incrementing the value of the `lifting_num_detail_levels_minus1` field by 1. (`LevelDetailCount = lifting_num_detail_levels_minus1 + 1`).

[0493] According to the implementation method, when the value of the lifting_num_detail_levels_minus1 field is greater than 1, the APS may also include the lifting_lod_regular_sampling_enabled_flag field.

[0494] The `lifting_lod_regular_sampling_enabled_flag` field specifies whether to construct the level of detail (LOD) using a regular sampling strategy. For example, a `lifting_lod_regular_sampling_enabled_flag` value of 1 indicates that the LOD is constructed using a regular sampling strategy. A `lifting_lod_regular_sampling_enabled_flag` value of 0 indicates that a distance-based sampling strategy is used instead.

[0495] According to the implementation, when the value of the `lifting_scalability_enabled_flag` field is false, the APS may also include an iterative statement that iterates as many times as the value of the `lifting_num_detail_levels_minus1` field. In the implementation, the index (idx) is initialized to 0 and incremented by 1 each time an iterative statement is executed, and the iterative statement is iterated until the index (idx) is greater than the value of the `lifting_num_detail_levels_minus1` field. When the value of the `lifting_lod_regular_sampling_enabled_flag` field is true (e.g., 1), the iterative statement may include the `lifting_sampling_period_minus2[idx]` field, and when the value of the `lifting_lod_regular_sampling_enabled_flag` field is false (e.g., 0), the iterative statement may include the `lifting_sampling_distance_squared_scale_minus1[idx]` field. Additionally, when the value of idx is not 0 (idx != 0), the lifting_sampling_distance_squared_offset[idx] field can also be included.

[0496] The lifting_sampling_period_minus2[idx] plus 2 specifies the sampling period of the detail level idx.

[0497] The lifting_sampling_distance_squared_scale_minu1[idx] increment by 1 specifies the scaling factor for the square of the sampling distance used to derive the level of detail idx.

[0498] The lifting_sampling_distance_squared_offset[idx] field specifies the offset of the squared sampling distance used to derive the level of detail idx.

[0499] When the value of the attr_coding_type field is 0, that is, when the encoding type is predicting weight increase, the APS according to the implementation method may also include the lifting_adaptive_prediction_threshold field, the lifting_intra_lod_prediction_num_layers field, the lifting_max_num_direct_predictors field, and the inter_component_prediction_enabled_flag field.

[0500] The `lifting_adaptive_prediction_threshold` field specifies the threshold for enabling adaptive prediction. According to the implementation, the variable `AdaptivePredictionThreshold`, which specifies the threshold used to switch the adaptive predictor selection mode, is set to the value of the `lifting_adaptive_prediction_threshold` field (`AdaptivePredictionThreshold = lifting_adaptive_prediction_threshold`).

[0501] The `lifting_intra_lod_prediction_num_layers` field specifies the number of LOD layers whose decoded points within the same LOD layer can be referenced to generate predictions for the target point. For example, a `lifting_intra_lod_prediction_num_layers` field equal to `LevelDetailCount` indicates that the target point can reference decoded points within the same LOD layer for all LOD layers. Conversely, a `lifting_intra_lod_prediction_num_layers` field equal to 0 indicates that the target point may not reference decoded points within the same LOD layer for any LOD layer. The `lifting_max_num_direct_predictors` field specifies the maximum number of predictors that will be used for direct prediction. The value of the `lifting_max_num_direct_predictors` field will range from 0 to `LevelDetailCount`.

[0502] The `inter_component_prediction_enabled_flag` field specifies whether the principal components of a multi-component attribute are used to predict the reconstructed values ​​of non-principal components. For example, if the `inter_component_prediction_enabled_flag` field equals 1, it specifies that the principal components of the multi-component attribute are used to predict the reconstructed values ​​of non-principal components. A `inter_component_prediction_enabled_flag` field equal to 0 specifies that all attribute components are reconstructed independently.

[0503] According to the implementation method, when the value of the attr_coding_type field is 1, that is, when the attribute encoding type is RAHT, the APS may also include the raht_prediction_enabled_flag field.

[0504] The `raht_prediction_enabled_flag` field specifies whether transform weight prediction from neighboring points is enabled during RAHT decoding. For example, a `raht_prediction_enabled_flag` value of 1 indicates that transform weight prediction from neighboring points is enabled during RAHT decoding. A `raht_prediction_enabled_flag` value of 0 indicates that transform weight prediction is disabled during RAHT decoding.

[0505] According to the implementation method, when the value of the raht_prediction_enabled_flag field is true, the APS may also include the raht_prediction_threshold 0 field and the raht_prediction_threshold 1 field.

[0506] The `raht_prediction_threshold0` field specifies the threshold at which to terminate the prediction of transformed weights from neighboring points.

[0507] The `raht_prediction_threshold1` field specifies the threshold for skipping the transformed weight predictions from neighboring points.

[0508] The `aps_extension_flag` field specifies whether the `aps_extension_data_flag` syntax structure exists in the APS syntax structure. For example, an `aps_extension_flag` value of 1 indicates that the `aps_extension_data` syntax structure exists in the APS syntax structure. Conversely, an `aps_extension_flag` value of 0 indicates that the `aps_extension_data` syntax structure does not exist in the APS syntax structure.

[0509] When the value of the aps_extension_flag field is 1, the APS according to the implementation may also include the aps_extension_data_flag field.

[0510] The `aps_extension_data_flag` field can have any value. Its presence and value do not affect the decoder's compliance with the configuration file.

[0511] The APS according to the implementation may also include information related to LoD-based attribute compression.

[0512] Figure 37 An implementation of the syntax structure of APS(attribute_parameter_set()) that includes information related to LoD-based attribute compression is shown.

[0513] exist Figure 37 In this context, information related to LoD-based attribute compression may include the constrained_QTBT_flag field, the num_LoD_minus1 field, and the LoD_sampling_method field.

[0514] The `constrained_QTBT_flag` field can indicate whether constraints exist in the tree structure generated for compressing geometric information and / or compressing attribute information via a point cloud video encoder. For example, a `constrained_QTBT_flag` equal to 1 can indicate the presence of constraints in the generated tree structure. Depending on the implementation, constraints can be conditions for scalable attribute encoding and / or can indicate that OT-based LoD generation can be used at a specific depth or a lower depth (see the description of Implementation 2 given above). For example, a value of `constrained_QTBT_flag` equal to 1 can be used to indicate that the attribute decoder 61004 of the receiving device can use an OT / QT / BT-based geometry tree to generate LoD. As another example, a value of `constrained_QTBT_flag` equal to 1 can be used to indicate that the attribute decoder 61004 of the receiving device can use a geometry tree for octree-based LoD generation at a certain depth (e.g., starting from a leaf node).

[0515] According to an implementation, when the value of the constrained_QTBT_flag field is 1, the information related to LoD-based attribute compression may further include a max_num_LoD field indicating the maximum number of LODs and a LoD_starting_depth field indicating the starting point of the LoD. According to an implementation, the max_num_LoD field may indicate the depth of the leaf levels in the geometry tree. Furthermore, the attribute decoder 61004 of the receiving device can decode attribute information by generating octree-based LoDs only for the corresponding depth in the geometry tree based on the constrained_QTBT_flag field, the max_num_LoD field, and the LoD_starting_depth field. A constrained_QTBT_flag field value of 0 can be used to indicate that the attribute decoder 61004 of the receiving device should use octree-based LoD generation.

[0516] Incrementing 1 by num_LoD_minus1 indicates the number of LoDs. Alternatively, it can be used in conjunction with the num_tree_depth_minus1 field to signal the depth in the geometry tree. To indicate the number of tree depths, the num_tree_depth_minus1 field can be included in information related to LoD-based attribute compression or geometry slice headers.

[0517] The `LoD_sampling_method` field indicates the sampling method used to select points within the neighbor search range (or configured neighbors) when generating the LoD. A `LoD_sampling_method` equal to 0 indicates that the first point can be selected from the points configured (or included) in the neighbors. In implementations, the first point means the first point among the points ordered in Morton code within the neighbor search range. A `LoD_sampling_method` equal to 1 indicates that the first and last points can be selected alternately for each depth. Here, the first point represents the first point among the points ordered in Morton code within the neighbor search range for that depth, and the last point represents the last point among the points ordered in Morton code within the neighbor search range for that depth. A `LoD_sampling_method` equal to 2 indicates that points with a small Euclidean distance relative to the neighbor center can be selected from the points within the neighbor search range. In other words, a `LoD_sampling_method` equal to 2 indicates that points close to the node center can be selected.

[0518] According to the implementation, when the value of the constrained_QTBT_flag field is true (e.g., 1), the information related to LoD-based attribute compression may also include the max_LoD_depth_for_octree_based_LoD_generation field.

[0519] When the `constrained_QTBT_flag` field has a value of 1, the `max_LoD_depth_for_octree_based_LoD_generation` field can indicate the maximum LoD depth that can match octree-based LoD generation. According to an implementation, the maximum LoD depth can indicate the depth magnitude from the leaf levels of the geometry tree. According to an implementation, the `max_LoD_depth_for_octree_based_LoD_generation` field can have the same meaning as the `LoD_starting_depth` field.

[0520] According to the implementation, the information related to LoD-based attribute compression may also include a loop that iterates as many times as the value of the num_LoD_minus1 field. In this case, in the implementation, i may be initialized to 0 and incremented by 1 each time the loop is executed. The loop may iterate until i reaches the value of the num_LoD_minus1 field. The loop may include the cubic_node_flag field, the node_size_x field, the node_size_y field, and the node_size_z field.

[0521] A cubic_node_flag of 1 indicates that the cell node used for neighbor search in LoD generation is a node with a cubic shape (i.e., a regular hexahedron) having the same length on the xyz side. A cubic_node_flag of 0 indicates that the cell node used for neighbor search in LoD generation is a cuboid (or cuboid or non-cubic shape) with different lengths on the xyz side.

[0522] When the value of the cubic_node_flag field is 0, information related to LoD-based attribute compression may also include the node_direction field.

[0523] The `node_direction` field indicates the directionality (i.e., the splitting direction) of a node with a cuboid shape. According to the implementation, values ​​from 0 to 5 can indicate the z, y, x, yz, xz, and xy directions of the node, respectively. For example, the size of a node on the xyz axis is represented as 2. dx 2 dy and 2 dz At that time, (k, k, 2k), (k, 2k, k), (2k, k, k), (k, 2k, 2k), (2k, k, 2k), and (2k, 2k, k) can be obtained. Here, k represents the cell size. Alternatively, the node_size_x, node_size_y, and node_size_z fields can be used to signal the size of the nodes on the x, y, and z axes.

[0524] The node_size_x, node_size_y, and node_size_z fields can indicate the size of the nodes on the x, y, and z axes.

[0525] According to the implementation, when the value of the constrained_QTBT_flag field is 1, the loop may also include the no_representation_flag field.

[0526] When performing octree-based LoD generation in attribute encoding, the `no_representation_flag` field can indicate whether the corresponding depth is suitable for scalable representation. Alternatively, the `no_representation_flag` field can be used to directly indicate that the number of points in the geometry at that depth is different from the number of points in the attribute. For example, a `no_representation_flag` field equal to 1 can indicate that the depth is not suitable for scalable representation. A `no_representation_flag` field equal to 0 can indicate that the depth is suitable for scalable representation, or that the number of points in the geometry at that depth is the same as the number of points in the attribute. As another example, the `no_representation_flag` field can indicate that the LoD was generated at the depth of the geometry tree where a point mismatch occurs between the geometry and the attribute. In this case, the LoD may not be used for attribute compression, and the attribute decoder 61004 of the receiving device may not use the LoD for attribute reconstruction based on this information. The `no_representation_flag` field can have the same meaning as the `LoD_skip_flag` field. Alternatively, the LoD_skip_flag field may also be included in information related to LoD-based attribute compression. Alternatively, the attribute decoder 61004 of the receiving device may determine whether to skip LoD based on information included in the loop.

[0527] According to the implementation method, Figure 37 Information related to LoD-based attribute compression can be included in Figure 36 In the APS, it can be in the if(LodParametersPresent) statement or the if(attr_coding_type==0) statement. This is merely an implementation method, and information related to LoD-based attribute compression can be included in Figure 36 At any location in the APS. Figure 38 An implementation of the syntax structure of the geometric slice bitstream () according to this disclosure is shown.

[0528] The geometry slice bitstream (geometry_slice_bitstream()) according to the implementation may include a geometry slice header (geometry_slice_header()) and geometry slice data (geometry_slice_data()). It can be referred to as a geometry slice. Furthermore, the attribute slice bitstream can be referred to as an attribute slice.

[0529] Figure 38 An exemplary syntax structure for geometry_slice_bitstream() according to an implementation is shown.

[0530] The geometry slice bitstream (geometry_slice_bitstream()) according to the implementation may include a geometry slice header (geometry_slice_header()) and geometry slice data (geometry_slice_data()).

[0531] Figure 39 An implementation of the syntax structure of the geometry_slice_header() according to this disclosure is shown.

[0532] According to the implementation, the bit stream transmitted by the transmitting device (or received by the receiving device) may contain one or more slices. Each slice may include a geometric slice and an attribute slice. The geometric slice includes a geometric slice header (GSH). The attribute slice includes an attribute slice header (ASH).

[0533] The geometry slice header (geometry_slice_header()) according to the implementation may include the gsh_geometry_parameter_set_id field, the gsh_tile_id field, the gsh_slice_id field, the frame_idx field, the gsh_num_points field, and the byte_alignment() field.

[0534] When the value of the gps_box_present_flag field included in GPS is true (e.g., 1) and the value of the gps_gsh_box_log2_scale_present_flag field is true (e.g., 1), the geometry_slice_header() according to the implementation may also include the gsh_box_log2_scale field, the gsh_box_origin_x field, the gsh_box_origin_y field, and the gsh_box_origin_z field.

[0535] gsh_geometry_parameter_set_id specifies the value of gps_geom_parameter_set_id for the active GPS.

[0536] The gsh_tile_id field specifies the value of the tile ID referenced by GSH.

[0537] gsh_slice_id specifies the ID of a slice referenced by other syntax elements.

[0538] The `frame_idx` field indicates the least significant bits of the concept frame number counter, which is `log2_max_frame_idx+1`. Consecutive slices with different values ​​of `frame_idx` form portions of different output point cloud frames. Consecutive slices with the same value of `frame_idx` (without intervening frame boundary marker data units) form portions of the same output point cloud frame.

[0539] The `gsh_num_points` field indicates the maximum number of encoded points in a slice. Depending on the implementation, bitstream consistency is required if `gsh_num_points` is greater than or equal to the number of decoded points in the slice.

[0540] The `gsh_box_log2_scale` field specifies the scaling factor for the origin of the bounding box of the slice.

[0541] The gsh_box_origin_x field specifies the x-value of the bounding box origin scaled according to the value of the gsh_box_log2_scale field.

[0542] The gsh_box_origin_y field specifies the y-value of the bounding box origin scaled according to the value of the gsh_box_log2_scale field.

[0543] The gsh_box_origin_z field specifies the z-value of the bounding box origin, scaled according to the value of the gsh_box_log2_scale field.

[0544] Here, the variables slice_origin_x, slice_origin_y, and slice_origin_z can be derived as follows.

[0545] When gps_gsh_box_log2_scale_present_flag equals 0, originScale is set to gsh_box_log2_scale.

[0546] When gps_gsh_box_log2_scale_present_flag equals 1, originScale is set to gps_gsh_box_log2_scale.

[0547] When gps_box_present_flag equals 0, the values ​​of the inferred variables slice_origin_x, slice_origin_y, and slice_origin_z are 0.

[0548] When gps_box_present_flag equals 1, the following equation will apply to the variables slice_origin_x, slice_origin_y, and slice_origin_z.

[0549] slice_origin_x=gsh_box_origin_x< <originScale

[0550] slice_origin_y=gsh_box_origin_y< <originScale

[0551] slice_origin_z=gsh_box_origin_z< <originScale

[0552] When the value of the `gps_implicit_geom_partition_flag` field is true (i.e., 0), the geometry slice header (`gsh_log2_max_nodesize_x`, `gsh_log2_max_nodesize_y_minus_x`, and `gsh_log2_max_nodesize_z_minus_y` fields may also be included. When the value of the `gps_implicit_geom_partition_flag` field is false (i.e., 1), the geometry slice header may also include the `gsh_log2_max_nodesize` field.

[0553] The `gsh_log2_max_nodesize_x` field specifies the bounding box size in the x-dimensional dimension, which is the following `MaxNodesize_XLog2` used during decoding.

[0554] MaxNodeSizeXLog2=gsh_log2_max_nodesize_x

[0555] MaxNodeSizeX = 1 <MaxNodeSizeXLog2

[0556] The `gsh_log2_max_nodesize_y_minus_x` field specifies the bounding box size in the y-axis, which is the `MaxNodesizeYLog2` used during decoding.

[0557] MaxNodeSizeYLog2=gsh_log2_max_nodesize_y_minus_x+MaxNodeSizeXLog2.

[0558] MaxNodeSizeY = 1 <MaxNodeSizeYLog2。

[0559] The `gsh_log2_max_nodesize_z_minus_y` field specifies the bounding box size in the z-axis, which is the `MaxNodesizeZLog2` used during decoding.

[0560] MaxNodeSizeZLog2=gsh_log2_max_nodesize_z_minus_y+MaxNodeSizeYLog2

[0561] MaxNodeSizeZ = 1 <MaxNodeSizeZLog2

[0562] When the value of the gps_implicit_geom_partition_flag field is 1, the gsh_log2_max_nodesize is obtained as follows.

[0563] gsh_log2_max_nodesize=max{MaxNodeSizeXLog2, MaxNodeSizeYLog2, MaxNodeSizeZLog2}

[0564] When gps_implicit_geom_partition_flag equals 0, the gsh_log2_max_nodesize field specifies the size of the root geometric octree node.

[0565] Here, the variables MaxNodeSize and MaxGeometryOctreeDepth are derived as follows.

[0566] MaxNodeSize = 1 <gsh_log2_max_nodesize

[0567] MaxGeometryOctreeDepth=gsh_log2_max_nodesize-log2_trisoup_node_size

[0568] When the value of the geom_scaling_enabled_flag field is true, the geometry_slice_header() according to the implementation may also include the geom_slice_qp_offset field and the geom_octree_qp_offsets_enabled_flag field.

[0569] The geom_slice_qp_offset field specifies the offset relative to the basic geometry quantization parameter geom_base_qp.

[0570] The `geom_octree_qp_offsets_enabled_flag` field specifies whether the `geom_octree_qp_ofsets_depth` field exists in the geometry tile header. For example, a `geom_octree_qp_offsets_enabled_flag` value of 1 indicates that the `geom_octree_qp_ofsets_depth` field exists in the geometry tile header. A `geom_octree_qp_offsets_enabled_flag` value of 0 indicates that the `geom_octree_qp_ofsets_depth` field does not exist.

[0571] The geom_octree_qp_offsets_depth field specifies the depth of the geometric octree.

[0572] Figure 40 An embodiment of the syntax structure of geometry slice data (geometry_slice_data()) according to this disclosure is shown. The geometry slice data (geometry_slice_data()) according to the embodiment can carry a geometry bitstream belonging to the corresponding slice.

[0573] According to the implementation, `geometry_slice_data()` may include a first iteration statement that repeats as many times as the value of `MaxGeometryoctreeDepth`. In the implementation, the depth is initialized to 0 and incremented by 1 each time the iteration statement is executed, and the first iteration statement is repeated until the depth becomes equal to `MaxGeometryoctreeDepth`. The first iteration statement may include a second loop statement that repeats as many times as the value of `NumNodesAtDepth`. In the implementation, `nodeidx` is initialized to 0 and incremented by 1 each time the iteration statement is executed. The second iteration statement is repeated until `nodeidx` becomes equal to `NumNodesAtDepth`. The second iteration statement may include `xN = NodeX[depth][nodeIdx]`, `yN = NodeY[depth][nodeIdx]`, `zN = NodeZ[depth][nodeIdx]`, and `geometry_node(depth,nodeIdx,xN,yN,zN)`. `MaxGeometryoctreeDepth` indicates the maximum depth of the geometric octree, and `NumNodesAtDepth[depth]` indicates the number of nodes to be decoded at the corresponding depth. The variables `NodeX[depth][nodeIdx]`, `NodeY[depth][nodeIdx]`, and `NodeZ[depth][nodeIdx]` indicate the x, y, and z coordinates of the `idx`-th node at a given depth in decoding order. The geometric bitstream of nodes at a given depth is sent via `geometry__node(depth, nodeIdx, xN, yN, zN)`.

[0574] When the value of the log2_trisoup_node_size field is greater than 0, the geometry slice data (geometry_slice_data()) according to the implementation method may also include geometry_slice_data(). That is, when the size of the triangle node is greater than 0, the geometry bitstream that has undergone trisoup geometry encoding is sent through geometry_trisoup_data().

[0575] Figure 41 An implementation of the syntax structure of attribute_slice_bitstream() according to this disclosure is shown.

[0576] The attribute slice bitstream (attribute_slice_bitstream()) according to the implementation method may include an attribute slice header (attribute_slice_header()) and attribute slice data (attribute_slice_data()).

[0577] Figure 42 An implementation of the syntax structure of the attribute slice header (attribute_slice_header()) according to this disclosure is shown.

[0578] The attribute slice header (attribute_slice_header()) according to the implementation may include the ash_attr_parameter_set_id field, the ash_attr_sps_attr_idx field, the ash_attr_geom_slice_id field, the ash_attr_layer_qp_delta_present_flag field, and the ash_attr_region_qp_delta_present_flag field.

[0579] When the value of the aps_slice_qp_delta_present_flag field of the APS is true (e.g., 1), the attribute slice header (attribute_slice_header()) according to the implementation may also include the ash_attr_qp_delta_luma field. When the value of the attribute_dimension_minus1[ash_attr_sps_attr_idx] field is greater than 0, the attribute slice header may also include the ash_attr_qp_delta_chroma field.

[0580] The ash_attr_parameter_set_id field specifies the value of the aps_attr_parameter_set_id field for the currently active APS.

[0581] The ash_attr_sps_attr_idx field specifies the set of attributes in the currently active SPS.

[0582] The ash_attr_geom_slice_id field specifies the value of the gsh_slice_id field in the current geometry slice header.

[0583] The ash_attr_qp_delta_luma field specifies the brightness increment quantization parameter qp derived from the initial slice qp in the active attribute parameter set.

[0584] The ash_attr_qp_delta_chroma field specifies the chroma increment qp derived from the initial slice qp in the active attribute parameter set.

[0585] The variables InitialSliceQpY and InitialSliceQpC are derived as follows.

[0586] InitialSliceQpY=aps_attrattr_initial_qp+ash_attr_qp_delta_luma

[0587] InitialSliceQpC=aps_attrattr_initial_qp+aps_attr_chroma_qp_offset+ash_attr_qp_delt a_chroma

[0588] The `ash_attr_layer_qp_delta_present_flag` field specifies whether the `ash_attr_layer_qp_delta_luma` and `ash_attr_layer_qp_delta_chroma` fields exist in the ASH for each layer. For example, a value of 1 indicates that the `ash_attr_layer_qp_delta_luma` and `ash_attr_layer_qp_delta_chroma` fields exist in the ASH. A value of 0 indicates that the fields do not exist.

[0589] When the value of the ash_attr_layer_qp_delta_present_flag field is true, ASH may also include the ash_attr_num_layer_qp_minus1 field.

[0590] Incrementing 1 by ash_attr_num_layer_qp_minus1 indicates the number of layers traversed by signaling the ash_attr_qp_delta_luma and ash_attr_qp_delta_chroma fields. When no signal is given to the ash_attr_num_layer_qp field, its value will be 0. According to an implementation, the number of layers, NumLayerQp (NumLayerQp = ash_attr_num_layer_qp_minus1 + 1), can be obtained by incrementing the value of the ash_attr_num_layer_qp_minus1 field by 1.

[0591] According to the implementation, when the value of the `ash_attr_layer_qp_delta_present_flag` field is true, the geometry slice header may include a loop that iterates as many times as the value of `NumLayerQp`. In this implementation, `i` may be initialized to 0 and incremented by 1 each time the loop is executed, and the loop iterates until the value of `i` reaches the value of `NumLayerQp`. This loop includes the `ash_attr_layer_qp_delta_luma[i]` field. Moreover, when the value of the `attribute_dimension_minus1[ash_attr_sps_attr_idx]` field is greater than 0, the loop may also include the `ash_attr_layer_qp_delta_chroma[i]` field.

[0592] The ash_attr_layer_qp_delta_luma field indicates the brightness increment quantization parameter qp from InitialSliceQpY in each layer.

[0593] The ash_attr_layer_qp_delta_chroma field indicates the chroma increment quantization parameter qp from InitialSliceQpC in each layer.

[0594] The variables SliceQpY[i] and SliceQpC[i] are derived as follows, where i = 0, ..., NumLayerQPNumQPLayer-1.

[0595] for(i=0; i <NumLayerQPNumQPLayer;i++){

[0596] SliceQpY[i]=InitialSliceQpY+ash_attr_layer_qp_delta_luma[i]

[0597] SliceQpC[i]=InitialSliceQpC+ash_attr_layer_qp_delta_chroma[i]

[0598] }

[0599] A flag of 1 for `ash_attr_region_qp_delta_present_flag` indicates that, according to the implementation, `ash_attr_region_qp_delta`, the origin of the region bounding box, and its size exist in the current attribute slice header (`attribute_slice_header()`). A flag of 0 for `ash_attr_region_qp_delta_present_flag` indicates that `ash_attr_region_qp_delta`, the origin of the region bounding box, and its size do not exist in the current attribute slice header.

[0600] In other words, when the value of the ash_attr_layer_qp_delta_present_flag field is 1, the attribute slice header can also include the ash_attr_qp_region_box_origin_x, ash_attr_qp_region_box_origin_y, ash_attr_qp_region_box_origin_z, ash_attr_qp_region_box_width, ash_attr_qp_region_box_height, ash_attr_qp_region_box_depth, and ash_attr_region_qp_delta fields.

[0601] The ash_attr_qp_region_box_origin_x field indicates the x-offset of the region bounding box relative to slice_origin_x.

[0602] The ash_attr_qp_region_box_origin_y field indicates the y-offset of the region bounding box relative to slice_origin_y.

[0603] The ash_attr_qp_region_box_origin_z field indicates the z-offset of the region bounding box relative to slice_origin_z.

[0604] The ash_attr_qp_region_box_size_width field indicates the width of the region's bounding box.

[0605] The ash_attr_qp_region_box_size_height field indicates the height of the region's bounding box.

[0606] The ash_attr_qp_region_box_size_depth field indicates the depth of the region bounding box.

[0607] The ash_attr_region_qp_delta field indicates the increment qp (delta qp) of SliceQpY[i] and SliceQpC[i] from the region specified by the ash_attr_qp_region_box field.

[0608] According to the implementation method, the variable RegionboxDeltaQp, which specifies the incremental quantization parameter of the region box, is set to be equal to the value of the ash_attr_region_qp_delta field (RegionboxDeltaQp = ash_attr_region_qp_delta).

[0609] The attribute slice header according to the implementation may also include information related to LoD-based attribute compression.

[0610] Figure 43 This is a diagram illustrating an implementation of a syntax structure for an attribute slice header (attribute_slice_header()) that includes information related to LoD-based attribute compression.

[0611] Figure 43Information related to LoD-based attribute compression may include the `constrained_QTBT_flag` field, the `num_LoD_minus1` field, and the `LoD_sampling_method` field. Depending on the value of the `constrained_QTBT_flag` field, the information related to LoD-based attribute compression may also include the `max_LoD_depth_for_octree_based_LoD_generation` field. Information related to LoD-based attribute compression may also include a loop that iterates as many times as the value of the `num_LoD_minus1` field. This loop may include the `cubic_node_flag` field, the `node_size_x` field, the `node_size_y` field, and the `node_size_z` field. Depending on the value of the `cubic_node_flag` field, the loop may also include the `node_direction` field, and depending on the value of the `constrained_QTBT_flag` field, it may also include the `no_representation_flag` field.

[0612] Included Figure 43 The fields in the information related to LoD-based attribute compression have the same characteristics as Figure 37 The fields in [the document] have the same meaning. Therefore, for details about the fields, please refer to [the document / reference]. Figure 37 A detailed description.

[0613] According to the implementation method, Figure 43 Information related to LoD-based attribute compression can be included in Figure 43 This is before the `if(aps_slice_qp_delta_present_flag)` statement in the attribute slice header. This is merely an implementation method, and information related to LoD-based attribute compression can be included. Figure 43 At any location in the APS.

[0614] Figure 44 This diagram illustrates an embodiment of the syntax structure of attribute slice data (attribute_slice_data()) according to this disclosure. The attribute slice data (attribute_slice_data()) according to this embodiment can carry an attribute bitstream belonging to the corresponding slice. The attribute slice data according to this embodiment may include attributes or attribute-related data that are some or all related to the point cloud.

[0615] exist Figure 44 In this context, the zerorun field specifies the number of zeros preceding predIndex or the residual.

[0616] The predIndex[i] field specifies the predictor sub-index used to decode the value of the i-th point of the attribute. The value of the predIndex[i] field is in the range from 0 to the value of the max_num_predictor field.

[0617] The operations described above can be performed using elements of a point cloud transmitting / receiving apparatus / method according to the embodiments described below. Each element according to the embodiments may correspond to hardware, software, a processor, and / or a combination thereof. Although this embodiment describes a method for compressing attribute information of point cloud data, the method described herein can be applied to geometric information compression and other compression methods.

[0618] Figure 45 Another example of a point cloud transmitting apparatus according to an embodiment is shown. Figure 45 The elements of the point cloud transmitting device shown can be implemented as hardware, software, processor and / or a combination thereof.

[0619] According to an implementation, the point cloud transmitting device may include a signaling processor 51002, a geometric encoder 51003, an attribute encoder 51004, and a transmitting processor 51005.

[0620] The geometry encoder 51003 and the attribute encoder 51004 can perform operations related to... Figure 1 Point cloud video encoder 10002, Figure 2 The code 20001 Figure 4 Point cloud video encoder and Figure 12 The point cloud video encoder describes some or all of the operations involved.

[0621] In one implementation, the geometric encoder 51003 generates a geometric tree by segmenting point cloud data consisting of points that are symmetrically or asymmetrically distributed based on OT or OT / QT / BT as described above, and compresses the geometric information of the point cloud data based on the generated geometric tree.

[0622] For details on generating geometric trees from point cloud data segmented by the geometric encoder 51003 based on OT or OT / QT / BT, refer to the above. Figures 16 to 30 The description.

[0623] The geometry encoder 51003 reconstructs geometric information based on the position changed by compression and outputs the reconstructed (or decoded) geometric information to the attribute encoder 51004.

[0624] The attribute encoder 51004 compresses attribute information based on location and / or reconstructed geometric information that has not yet undergone geometric encoding. The attribute encoder 51004 can perform encoded attribute information or perform scalable attribute encoding by combining one or more of embodiments 1 to 3.

[0625] The signaling processor 51002 can generate and / or process signaling information necessary for encoding / decoding / rendering geometric and attribute information, and provide it to the geometry encoder 51003, attribute encoder 51004, and / or transmitting processor 51005. Alternatively, the signaling processor 51002 can receive signaling information generated by the geometry encoder 51003, attribute encoder 51004, and / or transmitting processor 51005. The signaling processor 51002 can provide information (e.g., head orientation information and / or viewport information) fed back from the receiving device to the geometry encoder 51003, attribute encoder 51004, and / or transmitting processor 51005.

[0626] In this disclosure, signaling information can be notified and transmitted using signals in units of parameter sets (Sequence Parameter Set (SPS), Geometric Parameter Set (GPS), Attribute Parameter Set (APS), Patch Parameter Set (TPS) (or Patch List), etc.). Furthermore, it can be notified and transmitted using signals in coded units of each image (such as slices or patches).

[0627] According to the implementation, the signaling processor 51002 can at least signal the aforementioned LoD-based attribute compression related information to the APS or attribute slice head to provide the same information to the geometry encoder 51003, attribute encoder 51004 and / or sending processor 51005.

[0628] The method / apparatus according to the embodiments can use signals to notify relevant information to add / execute the operations of the embodiments. The signaling information according to the embodiments can be used by the transmitting and / or receiving devices.

[0629] The send processor 51005 can perform operations with Figure 12 The operation and / or transmission method of the transmitting processor 12012 are the same as or similar to the operation and / or transmission method, and the execution is the same as... Figure 1 The operation and / or transmission method of transmitter 1003 are the same as or similar to those of transmitter 1003. For details, refer to [reference needed]. Figure 1 or Figure 12 The description of [the object] will be omitted here.

[0630] The transmitting processor 51005 can transmit a geometry bitstream output from the geometry encoder 51003, an attribute bitstream output from the attribute encoder 51004, and a signaling bitstream output from the signaling processor 51002 by multiplexing the bitstreams into a single bitstream or by encapsulating the bitstreams in a file or fragment. In embodiments of this disclosure, the file may be in ISOBMFF file format.

[0631] According to the embodiments, files or fragments can be sent to a receiving device or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitting processor 51005 according to the embodiments can perform wired / wireless communication with the receiving device via a 4G, 5G, 6G, or other network. Furthermore, the transmitting processor 51005 can perform necessary data processing operations according to the network system (e.g., a 4G, 5G, or 6G communication network system). Moreover, the transmitting processor 51005 can transmit encapsulated data on demand.

[0632] Figure 46 This is an exemplary detailed block diagram of the attribute encoder 51004 according to an implementation method. Figure 46 The components of the attribute encoder shown can be implemented as hardware, software, processor, and / or a combination thereof.

[0633] According to an implementation, the attribute encoder 51004 may include a LoD generator 53001, an attribute compressor 53002, an attribute transformer 53003, and a quantizer 53004.

[0634] LoD generator 53001 generates LoDs based on geometric information of locations and / or reconstructed locations that have not yet undergone geometric encoding. LoD generator 53001 is based on... Figures 16 to 30 LoD is generated by one or a combination of two or more of the above embodiments 1 to 3.

[0635] In other words, the LoD generator 53001 generates LoDs based on the geometric tree structure described above, and performs subsampling by defining nodes at the corresponding geometric depth as neighbors (or neighbor search ranges) in the generated LoDs. Then, the geometric tree is constructed, and node size information based on the LoD layer is signaled using information related to LoD-based attribute compression, such as the `num_LoD_minus1` field, `cubic_node_flag` field, `node_direction` field, `node_size_x` field, `node_size_y` field, and `node_size_z` field. Additionally, when the LoD generator 53001 is configured to perform octree-based LoD generation using the geometric tree, relevant information can be signaled using information related to LoD-based attribute compression (e.g., the `constrained_QTBT_flag` field and the `max_LoD_depth_for_octree_based_LoD_generation` field). Furthermore, the LoD sampling method can be passed through information related to LoD-based attribute compression (e.g., the `LoD_sampling_method` field). For details regarding the fields num_LoD_minus1, cubic_node_flag, node_direction, node_size_x, node_size_y, node_size_z, constrained_QTBT_flag, max_LoD_depth_for_octree_based_LoD_generation, and LoD_sampling_method included in information related to LoD-based attribute compression, please refer to [link to relevant documentation]. Figure 37 The description.

[0636] The attribute compressor 53002 performs attribute prediction based on the LoD generated by the LoD generator 53001, and generates residual attribute information based on the predicted attribute information. The residual attribute information is transformed to the compressed domain by the attribute transformer 53003, and then quantized by the quantizer 53004. The quantized residual attribute information is then entropy-encoded and output as an attribute bitstream. The residual attribute information has the same meaning as the prediction error attribute information.

[0637] Figure 47 This is a flowchart illustrating an exemplary geometry tree-based LoD generation method according to an implementation method.

[0638] First, it is assumed that information related to LoD-based attribute compression (including information about the OT-based or OT / QT / BT-based geometric tree structure used by the geometry encoder 51003 to compress geometric information) is provided to the attribute encoder 51004 via the signaling processor 51002. Furthermore, it is assumed that the depth of the geometric tree is N.

[0639] In operation 55001, when point cloud data, including geometric tree information, LoD-based attribute compression related information, and depth information (N) are input, it is determined whether the value of the constrained_QTBT_flag field, which is included in the LoD-based attribute compression related information, is 1.

[0640] When the value of the constrained_QTBT_flag field is 1, the procedure proceeds to operation 55002. When the value of the constrained_QTBT_flag field is 0, the procedure proceeds to operation 55003.

[0641] When the `constrained_QTBT_flag` field has a value of 1, it indicates that constraints exist when generating the geometry tree structure. Here, constraints can be scalable attribute-encoded conditions and can indicate whether OT-based LoD generation can be used for a specific depth or a lower depth. Furthermore, a `constrained_QTBT_flag` value of 1 indicates that the receiving device can use OT-based LoD generation. A `constrained_QTBT_flag` value of 0 indicates that the receiving device should use geometry tree-based LoD generation.

[0642] In operation 55002, the cube nodes generated by OT-based segmentation are configured for neighbor search at the corresponding depth.

[0643] In operation 55003, the process proceeds to operation 55004 or operation 55005, depending on the values ​​of the cubic_node_flag and node_direction fields included in the information related to LoD-based attribute compression.

[0644] A `cubic_node_flag` equal to 1 indicates that the cell node used for neighbor search in LoD generation is a cube-shaped node with the same length on the x, y, and z sides. A `cubic_node_flag` equal to 0 indicates that the cell node used for neighbor search in LoD generation is a cuboid-shaped node with different lengths on the x, y, and z sides. The `node_direction` field indicates the orientation (i.e., the splitting direction) of the cuboid-shaped node. Alternatively, the size of the node along the x, y, and z axes can be indicated by the `node_size_x`, `node_size_y`, and `node_size_z` fields.

[0645] According to the implementation, the value of the node_direction field from 0 to 5 can indicate the z, y, x, yz, xz, and xy directions of the node, respectively. For example, when the size of a node on the xyz axis is represented as 2... dx 2 dy and 2 dz When we have k, we can obtain (k, k, 2k), (k, 2k, k), (2k, k, k), (k, 2k, 2k), (2k, k, 2k), and (2k, 2k, k). Here, k represents the cell size.

[0646] Therefore, when the value of the `cubic_node_flag` field is 1, the process proceeds to operation 55002 for OT-based LoD generation. When the value of the `cubic_node_flag` field is 0 and the value of the `node_direction` field is one of 3, 4, and 5, the process proceeds to operation 55004 for QT-based LoD generation. Additionally, when the value of the `cubic_node_flag` field is 0 and the value of the `node_direction` field is one of 0, 1, and 2, the process proceeds to operation 55005 for BT-based LoD generation.

[0647] In operation 55004, cuboid nodes generated by QT-based segmentation are configured for neighbor search at the corresponding depth.

[0648] In operation 55005, cuboid nodes generated by BT-based segmentation are configured for neighbor search at the corresponding depth.

[0649] In operation 55006, based on the value of the LoD_sampling_method field included in the LoD-based attribute compression information, a point belonging to LoD is selected from one of the cube nodes or cuboid nodes provided in operations 55002, 55004, and 55005.

[0650] For example, in operation 55006, when the value of the LoD_sampling_method field is 0, the first point can be selected. When the value is 1, the first and last points in the LoD can be selected alternately. When the value is 2, points with a small Euclidean distance relative to the neighbor center can be selected.

[0651] In operation 55007, the current depth is checked. If there is remaining depth for LoD generation, the depth variable N is decreased by 1 (operation 55008), and the process proceeds to operation 55001. When LoD generation is completed for all depths, LoD generation is terminated.

[0652] Therefore, when the value of the constrained_QTBT_flag field is 0 (or 1 depending on the choice of the receiving device), the shape of the node changes to a cube or cuboid based on the depth of the geometric tree. Once the node shape is determined (neighbor search) via signaling related to information associated with LoD-based attribute compression, a point corresponding to the LoD is selected for each node. This operation can be performed using a method signaled in the LOD_sampling_method. By executing this process up to the root level or a depth corresponding to N-max_LoD_depth_fr_octree_based_LoD_generation, LoDs can be generated.

[0653] Figure 48 Another example of a point cloud receiving device according to an embodiment is illustrated. Figure 48 The components of the point cloud receiving device shown can be implemented as hardware, software, a processor, and / or a combination thereof.

[0654] According to an implementation, the point cloud receiving device may include a receiving processor 61001, a signaling processor 61002, a geometry decoder 61003, an attribute decoder 61004, and a post-processor 61005.

[0655] The receiving processor 61001 according to the embodiment can receive a bit stream, or it can receive a geometric bit stream, an attribute bit stream, and a signaling bit stream respectively. When a file and / or fragment is received, the receiving processor 61001 can decapsulate the received file and / or fragment and output a bit stream.

[0656] When receiving (or decapsulating) a bitstream, the receiving processor 61001 according to the embodiment can demultiplex a geometric bitstream, an attribute bitstream, and / or a signaling bitstream from a bitstream, and output the demultiplexed signaling bitstream to a signaling processor 61002, the demultiplexed geometric bitstream to a geometric decoder 61003, and the demultiplexed attribute bitstream to an attribute decoder 61004.

[0657] When receiving (or decapsulating) the geometry bit stream, attribute bit stream, and / or signaling bit stream respectively, the receiving processor 61001 according to the embodiment can send the signaling bit stream to the signaling processor 61002, the geometry bit stream to the geometry decoder 61003, and the attribute bit stream to the attribute decoder 61004.

[0658] The signaling processor 61002 can parse and process information included in signaling information (e.g., SPS, GPS, APS, TPS, metadata, etc.) from the input signaling bitstream and provide it to the geometry decoder 61003, the attribute decoder 61004, and the post-processor 61005. In another embodiment, before decoding the corresponding slice data, the signaling processor 61002 can also parse the signaling information included in the geometry slice header and / or the attribute slice header.

[0659] According to the implementation, the signaling processor 61002 can at least parse and process the information related to LoD-based attribute compression that is signaled in the APS or attribute slice header, and provide it to the geometry decoder 61003, the attribute decoder 61004, and the post-processor 61005.

[0660] According to the implementation method, the geometry decoder 61003 can perform operations on the compressed geometry bitstream based on signaling information. Figure 45 The geometry is reconstructed by reversing the operation of the geometry encoder 51003. The geometry information reconstructed (or recovered) by the geometry decoder 61003 is provided to the attribute decoder 61004. The attribute decoder 61004 can then process the compressed attribute bitstream based on signaling information and the reconstructed geometry information. Figure 45 The property encoder 51004 reconstructs the property by reversing the operation.

[0661] According to the implementation, the post-processor 61005 can match the geometric information (i.e., position) reconstructed and output by the geometry decoder 61003 with the attribute information reconstructed and output by the attribute decoder 61004 to reconstruct and display / render point cloud data.

[0662] Figure 49 This is an exemplary detailed block diagram illustrating the attribute decoder 61004 according to an embodiment. Figure 49 The elements of the attribute decoder shown can be implemented as hardware, software, a processor, and / or a combination thereof.

[0663] According to the implementation, the attribute decoder 61004 may include a dequantizer 63001, an inverse transformer 63002, an LoD generator 63003, and an attribute reconstructor 63004.

[0664] According to the implementation, the attribute decoder 61004 reconstructs the attribute information by performing the inverse process of the attribute encoder 51004 of the transmitting device. In other words, the dequantizer 63001 can entropy decode the residual attribute information included in the attribute bitstream and then dequantize it. Then, the inverse transformer 63002 processes the dequantized residual attribute information by performing the inverse process of the attribute transformer 53003 and then outputs it to the attribute reconstructor 63004.

[0665] The attribute reconstructor 63004 performs attribute prediction based on the LoD generated by the LoD generator 63003, and reconstructs attribute information based on the predicted attribute information and the residual attribute information output from the inverse transformer 63002.

[0666] LoD generator 63003 generates LoDs in the same or similar manner as the LoD generation method of LoD generator 53001 used for transmitting devices. For example, LoD generator 63003 may be based on a reference... Figures 16 to 30 The described geometric tree structure generates a LoD.

[0667] According to the implementation, the LoD generator 63003 can use different types of LoD generation based on the value of the constrained_QTBT_flag field. For example, when the value of the constrained_QTBT_flag field is 1, the LoD generator 63003 can perform OT-based LoD generation or OT / QT / BT-based LoD generation. The degree to which the depth of the scalable representation is allowed can be determined by the max_LoD_depth_for_octree_based_LoD_generation field. For example, when the value of the constrained_QTBT_flag field is 0, the LoD generator 63003 can perform LoD-only generation based on a geometry tree generated by a combination of OT / QT / BT, and construct the geometry tree. The node size information according to the LoD layer can be determined based on the num_LoD_minus1, cubic_node_flag, node_direction, node_size_x, node_size_y, and node_size_z fields. Depending on the implementation, it can be determined whether a LoD is suitable for a scalable representation (i.e., whether the geometry layer matches the attribute layer) based on the no_representation_flag field. For example, a LoD can be used in a scalable representation only when the no_representation_flag field has a value of 0.

[0668] Therefore, when the transmitting device performs geometric compression by generating a geometric tree based on OT / QT / BT, attribute information can also be compressed by generating a LoD based on OT / QT / BT. Thus, the number of points in the geometry can be matched with the number of points in the attributes.

[0669] Figure 50 This is a flowchart illustrating a method for sending point cloud data according to an implementation method.

[0670] The method for transmitting point cloud data according to the embodiment may include: acquiring point cloud data (71001); encoding the point cloud data (71002); and transmitting the encoded point cloud data and signaling information (71003). In this case, the bit stream containing the encoded point cloud data and signaling information can be encapsulated into a file and transmitted.

[0671] In operation 71001, which acquires point cloud data, the following can be executed: Figure 1 The point cloud video acquisition unit 10001 performs some or all of the operations, or can execute Figure 12 The operation of the data input unit 12000 may be part or all of the operation.

[0672] In operation 71002, which encodes point cloud data, the following can be executed: Figure 1 Point cloud video encoder 10002, Figure 2 The code 20001 Figure 4 Point cloud video encoder, Figure 12 Point cloud video encoder, Figure 14 Point cloud encoding, Figure 15 Point cloud encoding and Figure 18 Some or all of the operations of the point cloud video encoder.

[0673] The operation 71002 for encoding point cloud data 71002 according to the embodiment may include encoding the geometric data (or geometric information) of the point cloud data and encoding the attribute data (or attribute information) of the point cloud data.

[0674] In the process of encoding geometric information, the acquired point cloud data is segmented based on OT or OT / QT / BT to compress the geometric information. For details on generating a geometric tree by segmenting point cloud data based on OT or OT / QT / BT during the geometric information encoding process, refer to [reference needed]. Figures 16 to 30The description of the geometric information is omitted here. In the operation of encoding geometric information, the geometric information is reconstructed based on the positions changed by compression, and the reconstructed (or decoded) geometric information is output to the operation of encoding attribute information. In the operation of encoding attribute information, the attribute information is compressed based on the reconstructed geometric information and / or positions where geometric encoding has not yet been performed. In the operation of encoding attribute information, the encoding of attribute information or scalable attribute encoding can be performed by one or a combination of two or more of the above embodiments 1 to 3. It can be based on... Figure 47 The flowchart executes the encoding of attribute information according to the implementation method.

[0675] In this disclosure, signaling information can be SPS, GPS, APS, TPS, metadata, etc., and the geometry slice header and / or attribute slice header can also be referred to as signaling information. Information related to LoD-based attribute compression required for compressing geometry and compressed attribute information can be signaled at least in the APS or attribute slice header. Information related to LoD-based attribute compression has already been described in detail above, therefore its description will be omitted.

[0676] Figure 51 This is a flowchart illustrating a method for receiving point cloud data according to an embodiment.

[0677] The method for receiving point cloud data according to the implementation method may include: receiving encoded point cloud data and signaling information (81001), decoding the point cloud data based on the signaling information (81002), and rendering the decoded point cloud data (81003).

[0678] The operation 81001 for receiving point cloud data and signaling information according to the implementation method can be performed by... Figure 1 Receiver 10005 Figure 2 Send 20002 or decode 20003 or Figure 13 The receiver 13000 or receiver processor 13001 shall execute this.

[0679] The operation 81002, which decodes point cloud data based on signaling information, may include decoding geometric information and decoding attribute information based on the decoded / reconstructed geometric and signaling information. The signaling information may be SPS, GPS, APS, TPS, metadata, etc., and geometric slice headers and / or attribute slice headers may also be referred to as signaling information. At least the APS or attribute slice header may include information related to LoD-based attribute compression. Information related to LoD-based attribute compression has already been described in detail above, therefore its description will be omitted.

[0680] In the operation of decoding geometric information according to the implementation method, the following can be performed: Figure 1Point cloud video decoder 10006 Figure 2 Decoding 20003 Figure 11 Point cloud video decoder, Figure 13 Point cloud video decoder and Figure 48 Some or all of the operations of the geometry decoder.

[0681] In the operation of decoding attribute information according to the implementation method, the following can be performed: Figure 1 Point cloud video decoder 10006 Figure 2 Decoding 20003 Figure 11 Point cloud video decoder, Figure 13 Point cloud video decoder and Figure 48 Some or all of the operations in the attribute decoder.

[0682] In the rendering operation 81003 according to the embodiment, point cloud data reconstructed based on decoded geometric information and decoded attribute information can be rendered according to various rendering methods. For example, points in the point cloud content can be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the vertex position, or circles centered at the vertex position. All or part of the rendered point cloud content area can be provided to the user through a display (e.g., a VR / AR display, a general-purpose display, etc.). The rendering operation 81003 of the point cloud data can be performed by… Figure 1 Renderer 10007 Figure 2 Rendering 20004 or Figure 13 The renderer 13011 is executed.

[0683] The geometry tree-based LoD generation method described in this disclosure is a method for partitioning point cloud data and can also be used for attribute encoding. Specifically, when performing QTBT-based geometry encoding in a scalable encoding application, attribute information can be matched with corresponding points / nodes in a one-to-one correspondence based on the depth of the geometry tree. Therefore, scalable representation can be enabled for all depths or certain matching depths.

[0684] This disclosure proposes a method for generating Level of Detail (LOD) data to improve compression efficiency when point cloud data has an asymmetric distribution. According to the embodiments described in this disclosure, scalable attribute encoding can be performed on asymmetric point cloud data. More specifically, asymmetric cuboid nodes can be considered to generate the LOD, and attribute encoding can be performed based on layer-based node variations considering QTBT-based geometric compression. Furthermore, the transmission / reception method / apparatus according to the embodiments can perform OT-based scalable attribute compression. Therefore, the compression efficiency of the point cloud data can be improved.

[0685] Each of the above parts, modules, or units can be a software, processor, or hardware component that executes a series of processes stored in memory (or storage units). Each step described in the above embodiments can be executed by a processor, software, or hardware component. Each module / block / unit described in the above embodiments can operate as a processor, software, or hardware. Furthermore, the methods proposed in the embodiments can be executed as code. This code can be written to a processor-readable storage medium, and thus be read by a processor provided by the device.

[0686] In this specification, when a part "comprises" or "includes" an element, it means that the part also includes or includes another element, unless otherwise stated. Additionally, the term "...module (or unit)" disclosed in this specification refers to a unit for performing at least one function or operation, and may be implemented by hardware, software, or a combination of hardware and software.

[0687] Although embodiments have been described with reference to each of the accompanying drawings for simplicity, new embodiments can be designed by incorporating the embodiments illustrated in the drawings. If a person skilled in the art designs a computer-readable recording medium containing a program for performing the embodiments described above, such recording medium may fall within the scope of the appended claims and their equivalents.

[0688] The apparatus and methods are not limited to the configurations and methods of the above embodiments. The above embodiments can be configured by selectively combining them, either wholly or partially, to enable various modifications.

[0689] Although preferred embodiments have been described with reference to the accompanying drawings, those skilled in the art will appreciate that various modifications and variations may be made to the embodiments without departing from the spirit or scope of this disclosure as set forth in the appended claims. Such modifications should not be understood as being independent of the technical concept or viewpoint of the embodiments.

[0690] Various components of the device according to the embodiments can be implemented by hardware, software, firmware, or a combination thereof. Various components in the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiments, the components according to the embodiments can be implemented as separate chips. According to the embodiments, at least one or more components of the device according to the embodiments can include one or more processors capable of executing one or more programs. The one or more programs can execute any one or more of the operations / methods according to the embodiments, or include instructions for executing them. Executable instructions for performing the methods / operations of the device according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transient CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiments can be used as a concept that covers not only volatile memory (e.g., RAM) but also non-volatile memory, flash memory, and PROM. Furthermore, it can also be implemented in the form of a carrier wave, such as transmission via the Internet. Additionally, the processor-readable recording medium can be distributed across a network-connected computer system, allowing processor-readable code to be stored and executed in a distributed manner. In this document, the terms “ / ” and “,” should be interpreted as indicating “and / or”. For example, the expression “A / B” can mean “A and / or B”. Additionally, “A, B” can mean “A and / or B”. Furthermore, “A / B / C” can mean “at least one of A, B and / or C”. “A, B, C” can mean “at least one of A, B and / or C”.

[0691] Furthermore, in this document, the term "or" should be interpreted as "and / or". For example, the expression "A or B" could mean 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in this document should be interpreted as "additionally or alternatively".

[0692] The various elements of the embodiments can be implemented by hardware, software, firmware, or a combination thereof. The various elements of the embodiments can be executed by a single chip, such as a single hardware circuit. According to the embodiments, the elements can be selectively executed by separate chips, respectively. According to the embodiments, at least one of the elements of the embodiments can be executed in one or more processors including instructions for performing the operations according to the embodiments.

[0693] Terms such as "first" and "second" can be used to describe various elements of the embodiments. However, the various components according to the embodiments should not be limited to the above terms. These terms are only used to distinguish one element from another. For example, a first user input signal can be referred to as a second user input signal. Similarly, a second user input signal can be referred to as a first user input signal. The use of these terms should be interpreted without departing from the scope of the various embodiments. Both a first user input signal and a second user input signal are user input signals, but they do not mean the same user input signal unless the context clearly indicates otherwise.

[0694] The terminology used to describe embodiments is used only for the purpose of describing particular embodiments and is not intended to limit the embodiments. As used in the description of embodiments and in the claims, the singular forms “a,” “an,” and “the” include plural indicators unless the context clearly specifies otherwise. The expression “and / or” is used to include all possible combinations of terms. Terms such as “comprising” or “having” are intended to indicate the presence of figures, numbers, steps, elements, and / or components, and should be understood not to exclude the possibility of the additional presence of figures, numbers, steps, elements, and / or components. As used herein, conditional expressions such as “if” and “when” are not limited to optional cases, but are intended to be interpreted as performing a related operation or interpreting a related definition based on a specific condition when that condition is met.

[0695] The mode of the present invention

[0696] As described above, the relevant content has been described in the best mode for implementing the method.

[0697] Industrial applicability

[0698] As described above, the embodiments can be applied in whole or in part to point cloud data transmitting / receiving apparatuses and systems. Those skilled in the art will appreciate that various modifications or variations can be made to the embodiments within their scope. Therefore, the embodiments are intended to cover modifications and variations of this disclosure, provided that they fall within the scope of the appended claims and their equivalents.

Claims

1. A method for encoding point cloud data, the method comprising the following steps: The geometric information of the position of points in the point cloud data is encoded by applying an octree; The attribute information, including the attribute values ​​of the points in the point cloud data, is encoded based on the octree. as well as Send encoded geometric information, encoded attribute information, and signaling information. The step of encoding the attribute information includes: The Level of Detail (LoD) is generated based on the octree. The octree is formed by depth. The depth in the context includes one or more occupied nodes. Wherein, at least one of the one or more occupying nodes includes two or more occupying child nodes. In this process, one of the two or more occupied child nodes is selected to generate the LoD corresponding to the depth. Specifically, a selection is performed for each of the at least one occupying node having two or more occupying child nodes to generate the LoD. Wherein, based on whether the LoD is odd or even, for each of the at least one occupied node, the selected occupied child node is either the first or the last node among two or more occupied child nodes. Wherein, the higher LoD includes points belonging to the lower LoD and points not belonging to the lower LoD, and Each LoD includes a point at at least one depth of the octree; and The attribute information is compressed based on the LoD.

2. The method according to claim 1, wherein, The two or more occupied child nodes with the same parent node are arranged according to the Morton code order.

3. An apparatus for encoding point cloud data, the apparatus comprising: A geometric encoder configured to encode geometric information including the positions of points in the point cloud data by applying an octree; as well as An attribute encoder, configured to encode attribute information including attribute values ​​of points in the point cloud data based on the octree; as well as A transmitter configured to transmit encoded geometric information, encoded attribute information, and signaling information. The attribute encoder includes: A LoD generator, configured to generate level-of-detail LoDs based on the octree. The octree is formed by depth. The depth in the context includes one or more occupied nodes. Wherein, at least one of the one or more occupying nodes includes two or more occupying child nodes. In this process, one of the two or more occupied child nodes is selected to generate the LoD corresponding to the depth. Specifically, a selection is performed for each of the at least one occupying node having two or more occupying child nodes to generate the LoD. Wherein, based on whether the LoD is odd or even, for each of the at least one occupied node, the selected occupied child node is either the first or the last node among two or more occupied child nodes. Wherein, the higher LoD includes points belonging to the lower LoD and points not belonging to the lower LoD, and Each LoD includes a point at at least one depth of the octree; and A compressor configured to compress the attribute information based on the LoD.

4. The apparatus according to claim 3, wherein, The two or more occupied child nodes with the same parent node are arranged according to the Morton code order.

5. A method for decoding point cloud data, the method comprising the following steps: Receive geometric information, attribute information, and signaling information; The geometric information is decoded based on the signaling information and the octree; as well as The attribute information is decoded based on the signaling information and the octree; The decoded geometric information includes the positions of points in the reconstructed point cloud data. The decoded attribute information includes the attribute values ​​of the points in the reconstructed point cloud data. The step of decoding the attribute information includes: The Level of Detail (LoD) is generated based on the octree. The octree is formed by depth. The depth in the context includes one or more occupied nodes. Wherein, at least one of the one or more occupying nodes includes two or more occupying child nodes. In this process, one of the two or more occupied child nodes is selected to generate the LoD corresponding to the depth. Specifically, a selection is performed for each of the at least one occupying node having two or more occupying child nodes to generate the LoD. Wherein, based on whether the LoD is odd or even, for each of the at least one occupied node, the selected occupied child node is either the first or the last node among two or more occupied child nodes. Wherein, the higher LoD includes points belonging to the lower LoD and points not belonging to the lower LoD, and Each LoD includes a point at at least one depth of the octree; and The attribute information is decoded based on the LoD.

6. The method according to claim 5, wherein, The two or more occupied child nodes with the same parent node are arranged according to the Morton code order.

7. An apparatus for decoding point cloud data, the apparatus comprising: A receiver configured to receive geometric information, attribute information, and signaling information; A geometry decoder configured to decode the geometric information based on the signaling information and an octree; as well as An attribute decoder, configured to decode the attribute information based on the signaling information and the octree; The decoded geometric information includes the positions of points in the reconstructed point cloud data. The decoded attribute information includes the attribute values ​​of the points in the reconstructed point cloud data. The attribute decoder includes: A level-of-detail (LoD) generator, configured to generate LoDs based on the octree. The octree is formed by depth. The depth in the context includes one or more occupied nodes. Wherein, at least one of the one or more occupying nodes includes two or more occupying child nodes. In this process, one of the two or more occupied child nodes is selected to generate the LoD corresponding to the depth. Specifically, a selection is performed for each of the at least one occupying node having two or more occupying child nodes to generate the LoD. Wherein, based on whether the LoD is odd or even, for each of the at least one occupied node, the selected occupied child node is either the first or the last node among two or more occupied child nodes. Wherein, the higher LoD includes points belonging to the lower LoD and points not belonging to the lower LoD, and Each LoD includes a point at at least one depth of the octree; and A decoder configured to decode the attribute information based on the LoD.

8. The apparatus according to claim 7, wherein, The two or more occupied child nodes with the same parent node are arranged according to the Morton code order.

Images