Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

The invention provides a point cloud data receiving method and device. A point cloud data receiving method according to an embodiment may comprise the steps of: receiving geometric data, attribute data, and signaling information; decoding the geometry data based on the signaling information; decoding the attribute data based on the signaling information and the decoded geometry data; and rendering point cloud data including the decoded geometry data and the decoded attribute data.

Description

Technical Field

[0001] Embodiments relate to methods and apparatus for processing point cloud content. Background Art

[0002] Point cloud content is represented by a point cloud—a collection of points belonging to a coordinate system representing a three-dimensional space (or volume). Point cloud content can express media configured in three dimensions and is used to provide a variety of services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), XR (scaled reality), and self-driving services. However, representing point cloud content requires tens to hundreds of thousands of points. Therefore, methods for efficiently processing large amounts of point data are needed.

[0003] In other words, high throughput is required to transmit and receive point cloud data. Therefore, in the process of transmitting and receiving point cloud data, encoding for compression and decoding for decompression are performed, and computational operations are complex and time-consuming due to the large amount of point cloud data. Summary of the Invention

[0004] Technical issues

[0005] An object of the present disclosure designed to solve the above-mentioned problems is to provide an apparatus and method for efficiently transmitting / receiving point clouds.

[0006] Another object of the present disclosure is to provide an apparatus and method for solving delay and encoding / decoding complexity.

[0007] Another object of the present disclosure is to provide an apparatus and method for providing an efficient and scalable service by performing LoD generation and neighbor search based on subgroups.

[0008] Another object of the present disclosure is to provide an apparatus and method for efficiently providing a scalable service by performing LoD generation and neighbor search on a subgroup basis in consideration of direct compression nodes.

[0009] The embodiments are not limited to the above-mentioned purposes, and the scope of the embodiments may cover other purposes that can be inferred by those skilled in the art based on the entire content of this disclosure.

[0010] Technical Solution

[0011] To achieve these objectives and other advantages and in accordance with the purposes of the present disclosure, as embodied and broadly described herein, a method for receiving point cloud data may include: receiving geometric data, attribute data, and signaling information; decoding the geometric data based on the signaling information; decoding the attribute data based on the signaling information and the decoded geometric data; and rendering the point cloud data including the decoded geometric data and the decoded attribute data.

[0012] According to an embodiment, decoding of attribute data may include: receiving attribute data based on a subgroup; storing the nodes of the current subgroup and the nodes of the parent subgroup of the current subgroup in a buffer based on signaling information, decoded geometric data and attribute data; generating at least one level of detail (LoD) of the current subgroup based on the nodes of the current subgroup and the nodes of the parent subgroup stored in the buffer; searching for nearest neighbors of the node to be subjected to attribute decoding in the current subgroup based on the at least one LoD of the current subgroup and the at least one LoD of the parent subgroup; and deriving attribute data based on the searched nearest neighbors.

[0013] According to an embodiment, the node of the parent subgroup used to generate at least one LoD for the current subgroup may not overlap with the node of the current subgroup.

[0014] According to an embodiment, the nodes of the parent subgroup used to generate at least one LoD of the current subgroup may include at least a node outside the boundary of the current subgroup or at least one direct compression (IDCM) node.

[0015] According to an embodiment, storage may include: storing the nodes of the current subgroup and the nodes of the parent subgroup in a buffer respectively; based on at least one of the nodes of the current subgroup being an IDCM node, storing the non-IDCM nodes of the current subgroup and at least one IDCM node separately in the buffer; and based on at least one of the nodes of the parent subgroup being an IDCM node, storing the non-IDCM nodes of the parent subgroup and at least one IDCM node separately in the buffer; wherein the nodes stored in the buffer can be identified by an index assigned to each of the nodes.

[0016] According to an embodiment, the storage may further include: based on at least one IDCM node of the current subgroup being included in multiple layers, separating at least one IDCM node of the current subgroup according to the layers and storing it in a buffer; and based on at least one IDCM node of the parent-child group being included in multiple layers, separating at least one IDCM node of the parent-child group according to the layers and storing it in a buffer.

[0017] According to an embodiment, the export of attribute data may include: based on the bottom layer of the parent-child group having a node with the same position as the node to be reconstructed in the top layer of the current child group, matching the attributes of the node in the parent-child group with the attributes of the node in the top layer of the current child group.

[0018] According to an embodiment, at least one IDCM node of the parent subgroup may be a node excluded from the top subsample of the current subgroup.

[0019] According to an embodiment, a device for receiving point cloud data includes: a receiver configured to receive geometric data, attribute data and signaling information; a geometry decoder configured to decode the geometric data based on the signaling information; an attribute decoder configured to decode the attribute data based on the signaling information and the decoded geometric data; and a renderer configured to render point cloud data including the decoded geometric data and the decoded attribute data.

[0020] According to an embodiment, an attribute decoder may include: a buffer configured to store nodes of a current subgroup and nodes of a parent subgroup of the current subgroup based on signaling information, decoded geometric data, and attribute data based on the subgroup; a level of detail (LoD) generator configured to generate at least one LoD of the current subgroup based on the nodes of the current subgroup and the nodes of the parent subgroup stored in the buffer; a neighbor searcher configured to search for nearest neighbors of nodes in the current subgroup to be subjected to attribute decoding based on at least one LoD of the current subgroup and at least one LoD of the parent subgroup; and an exporter configured to export attribute data based on the searched nearest neighbors.

[0021] According to an embodiment, the nodes of the parent subgroup used to generate at least one LoD of the current subgroup do not overlap with the nodes of the current subgroup.

[0022] According to an embodiment, the node of the parent subgroup used to generate at least one LoD of the current subgroup includes a node outside the boundary of the current subgroup or at least one direct compression (IDCM) node.

[0023] According to an embodiment, the buffer can be configured to store the nodes of the current subgroup and the nodes of the parent subgroup in separate storage areas; based on at least one of the nodes of the current subgroup being an IDCM node, store the non-IDCM nodes of the current subgroup and at least one IDCM node in separate storage areas; and based on at least one of the nodes of the parent subgroup being an IDCM node, store the non-IDCM nodes of the parent subgroup and at least one IDCM node in separate storage areas, wherein the nodes stored in the buffer can be identified by an index assigned to each of the nodes.

[0024] According to an embodiment, the buffer can be configured to: based on at least one IDCM node of the current sub-group being included in multiple layers, separate at least one IDCM node of the current sub-group according to the layers and store it; and based on at least one IDCM node of the parent-sub-group being included in multiple layers, separate at least one IDCM node of the parent-sub-group according to the layers and store it.

[0025] According to an embodiment, based on the bottom layer of the parent subgroup having a node at the same position as the node to be reconstructed in the top layer of the current subgroup, the exporter may match the attributes of the node in the parent subgroup with the attributes of the node in the top layer of the current subgroup.

[0026] Beneficial effects

[0027] The apparatus and method according to the embodiment may provide quality point cloud services.

[0028] The apparatus and method according to the embodiment may implement various video codec methods.

[0029] The apparatus and method according to the embodiment may provide general point cloud content such as autonomous driving services.

[0030] The apparatus and method according to the embodiment may perform spatially adaptive partitioning of point cloud data for independent encoding and decoding of the point cloud data, thereby improving parallel processing and providing scalability.

[0031] The apparatus and method according to the embodiment may improve encoding and decoding performance of a point cloud by partitioning point cloud data in units of tiles and / or slices and signaling necessary data therefor.

[0032] The apparatus and method according to the embodiment can divide and transmit compressed data of point cloud data according to predetermined criteria. In addition, when layered coding is used, the compressed data can be divided and transmitted according to the layer. This can increase the storage and transmission efficiency of the transmitting device.

[0033] When performing scalable transmission and / or scalable decoding, the apparatus and method according to the embodiments apply the same LoD generation and NN search performed by the attribute encoder in the transmitting apparatus to the attribute decoder in the receiving apparatus. Specifically, by performing the same position correction and NN search by the attribute encoder of the transmitting apparatus and the attribute decoder of the receiving apparatus, the neighbor candidates considered by the attribute encoder of the transmitting apparatus are the same as those considered by the attribute decoder of the receiving apparatus, and the order of the nodes is not changed. As a result, decoder prediction errors may not occur in operations such as predictive transforms that perform attribute prediction from neighboring nodes.

[0034] The apparatus and method according to the embodiment may improve the efficiency of scalable attribute coding by performing LoD generation and neighbor search in consideration of the presence of missing slices (or subgroups).

[0035] The apparatus and method according to the embodiment may improve the efficiency of scalable attribute decoding by performing LoD generation and neighbor search based on subgroups in attribute decoding.

[0036] The apparatus and method according to the embodiment may improve attribute decoding speed by performing LoD generation and neighbor search based on subgroups in consideration of direct compression nodes in attribute decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application. The accompanying drawings illustrate embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. In the drawings:

[0038] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown;

[0039] Figure 2 is a block diagram illustrating a point cloud content providing operation according to an embodiment;

[0040] Figure 3 illustrates an exemplary point cloud encoder according to an embodiment;

[0041] Figure 4 shows examples of octrees and occupancy codes according to an embodiment;

[0042] Figure 5 illustrates an example of point configuration in each LOD according to an embodiment;

[0043] Figure 6 illustrates an example of point configuration in each LOD according to an embodiment;

[0044] Figure 7 illustrates a point cloud decoder according to an embodiment;

[0045] Figure 8 FIGURE 1 illustrates an exemplary transmitting device according to an embodiment;

[0046] Figure 9 illustrates an exemplary receiving apparatus according to an embodiment;

[0047] Figure 10 An exemplary structure operable in conjunction with a method / apparatus for transmitting / receiving point cloud data according to an embodiment is illustrated;

[0048] Figure 11 and 12 A diagram illustrating a process for encoding, transmitting, and decoding point cloud data according to an embodiment;

[0049] Figure 13 illustrates a layer-based configuration of point cloud data according to an embodiment;

[0050] Figure 14 - (a) illustrates a bitstream structure for geometry data according to an embodiment, and Figure 14- (b) illustrates a bitstream structure for attribute data according to an embodiment;

[0051] Figure 15 is a diagram illustrating an example of a configuration of a bitstream for dividing a bitstream into layers for delivery according to an embodiment;

[0052] Figure 16 illustrates an exemplary bitstream ordering method used when a geometry bitstream and an attribute bitstream are multiplexed into a single bitstream according to an embodiment;

[0053] Figure 17 illustrates another exemplary bitstream ordering method used when a geometry bitstream and an attribute bitstream are multiplexed into a single bitstream according to an embodiment;

[0054] Figure 18 -(a) to Figure 18 - (c) illustrates an example of symmetric geometric property selection according to an embodiment;

[0055] Figure 19 -(a) to Figure 19 - (c) illustrates an example of asymmetric geometric property selection according to an embodiment;

[0056] Figure 20 -(a) to Figure 20 - (c) illustrates an example method of configuring a slice comprising point cloud data according to an embodiment;

[0057] Figure 21 -(a) and Figure 21 - (b) illustrates a geometric codec layer structure according to an embodiment;

[0058] Figure 22 illustrates a layer group and subgroup structure according to an embodiment;

[0059] Figure 23 -(a) to Figure 23 - (c) illustrates a representation of point cloud data based on layer groups according to an embodiment;

[0060] Figure 24 1. A point cloud data transmitting / receiving apparatus / method according to an embodiment is illustrated;

[0061] Figure 25 is a flowchart illustrating an exemplary LoD generation method according to an embodiment;

[0062] Figure 26 is a diagram illustrating an exemplary LoD generation method used when there is a missing subgroup according to an embodiment;

[0063] Figure 27 is a diagram illustrating an exemplary method of searching for nearest neighbors according to an embodiment;

[0064] Figure 28 is a diagram illustrating another exemplary method of searching for nearest neighbors according to an embodiment;

[0065] Figure 29 is a diagram illustrating yet another exemplary method of searching for nearest neighbors according to an embodiment;

[0066] Figure 30 Another exemplary point cloud transmitting device according to an embodiment is shown;

[0067] Figure 31 is a flowchart illustrating an exemplary method of encoding point cloud data according to an embodiment;

[0068] Figure 32 Another point cloud receiving device according to an embodiment is shown;

[0069] Figure 33 is a flowchart illustrating an exemplary method of decoding point cloud data according to an embodiment;

[0070] Figure 34 is a flowchart illustrating another embodiment of a point cloud data decoding method according to an embodiment;

[0071] Figure 35 illustrates an example of points included in each subgroup according to an embodiment;

[0072] Figure 36 illustrates an example of LoD generated by subsampling according to an embodiment;

[0073] Figure 37 illustrates an example of an input buffer according to an embodiment;

[0074] Figure 38 illustrates an example of a property inheritance process according to an embodiment;

[0075] Figure 39 illustrates an example bitstream structure for transmitting / receiving point cloud data according to an embodiment;

[0076] Figure 40 illustrates an example syntax structure of a sequence parameter set according to an embodiment;

[0077] Figure 41 illustrates an example syntax structure of a dependency attribute data unit header according to an embodiment;

[0078] Figure 42 Illustrate an example of providing services by compressing the geometry and attributes of point cloud data;

[0079] Figure 43illustrates another example of providing services by compressing the geometry and attributes of point cloud data according to an embodiment;

[0080] Figure 44 illustrates yet another example of providing a service by compressing the geometry and attributes of point cloud data according to an embodiment;

[0081] Figure 45 is a flowchart illustrating a method of transmitting point cloud data according to an embodiment;

[0082] Figure 46 is a flowchart illustrating a method of receiving point cloud data according to an embodiment. DETAILED DESCRIPTION

[0083] Reference will now be made in detail to preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The detailed description given below with reference to the accompanying drawings is intended to explain exemplary embodiments of the present disclosure and is not intended to illustrate the only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0084] Although most of the terms used in this disclosure have been selected from common terms widely used in the art, some terms have been arbitrarily selected by the applicant, and their meanings will be explained in detail as needed in the following description. Therefore, the present disclosure should be understood based on the original meaning of the terms rather than their simple names or meanings.

[0085] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0086] Figure 1 The illustrated point cloud content providing system may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 may be capable of wired or wireless communication to transmit and receive point cloud data.

[0087] According to an embodiment, the point cloud data transmitting device 10000 can obtain, process, and transmit point cloud video (or point cloud content). Depending on the embodiment, the transmitting device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device, and / or a server. Depending on the embodiment, the transmitting device 10000 may include a device configured to communicate with a base station and / or other wireless devices using a radio access technology (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0088] According to an embodiment, the transmitting device 10000 includes a point cloud video acquiring unit 10001 , a point cloud video encoder 10002 and / or a transmitter (or communication module) 10003 .

[0089] The point cloud video acquisition unit 10001 according to an embodiment acquires a point cloud video through a process such as capture, synthesis, or generation. A point cloud video is point cloud content represented by a point cloud. A point cloud is a collection of points located in a 3D space and may be referred to as point cloud video data, point cloud data, or the like. A point cloud video according to an embodiment may include one or more frames. A frame represents a still image or picture. Therefore, a point cloud video may include point cloud images, frames, or pictures and may be referred to as a point cloud image, frame, or picture.

[0090] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 may encode the point cloud video data based on a point cloud compression codec. The point cloud compression codec according to an embodiment may include a geometry-based point cloud compression (G-PCC) codec and / or a video-based point cloud compression (V-PCC) codec or a next-generation codec. The point cloud compression codec according to an embodiment is not limited to the above-described embodiment. The point cloud video encoder 10002 may output a bitstream containing the encoded point cloud video data. The bitstream may include not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0091] According to an embodiment, the transmitter 10003 transmits a bitstream containing encoded point cloud video data. According to an embodiment, the bitstream is encapsulated into files or segments (e.g., stream segments) and transmitted via various networks, such as broadcast networks and / or broadband networks. Although not shown in the figure, the transmitting device 10000 may include an encapsulator (or encapsulation module) configured to perform encapsulation operations. Depending on the embodiment, the encapsulator may be included in the transmitter 10003. Depending on the embodiment, the file or segment may be transmitted to the receiving device 10004 via a network or stored on a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). According to an embodiment, the transmitter 10003 is capable of wired / wireless communication with the receiving device 10004 (or receiver 10005) via 4G, 5G, 6G, and other networks. Furthermore, the transmitter may perform necessary data processing operations based on the network system (e.g., 4G, 5G, or 6G communication network system). The transmitting device 10000 can transmit the encapsulated data on demand.

[0092] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include a device configured to communicate with a base station and / or other wireless devices using a radio access technology (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0093] According to an embodiment, the receiver 10005 receives a bitstream containing point cloud video data, or a file / segment encapsulated with the bitstream, from a network or storage medium. The receiver 10005 may perform necessary data processing based on a network system (e.g., a 4G, 5G, or 6G communication network system). According to an embodiment, the receiver 10005 may decapsulate the received file / segment and output the bitstream. Depending on the embodiment, the receiver 10005 may include a decapsulator (or decapsulation module) configured to perform the decapsulation operation. The decapsulator may be implemented as a separate element (or component) from the receiver 10005.

[0094] The point cloud video decoder 10006 decodes the bitstream containing point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data based on the method by which the point cloud video data was encoded (e.g., by performing the inverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression codec (the inverse process of point cloud compression). The point cloud decompression codec includes the G-PCC codec.

[0095] The renderer 10007 renders the decoded point cloud video data. In one embodiment, the renderer 10007 may render the decoded point cloud video data according to a viewport, etc. The renderer 10007 may render not only the point cloud video data but also the audio data to output the point cloud content. Depending on the embodiment, the renderer 10007 may include a display configured to display the point cloud content. Depending on the embodiment, the display may be implemented as a separate device or component rather than being included in the renderer 10007.

[0096] The dashed arrows in the figure represent the transmission path of feedback information obtained by receiving device 10004. Feedback information reflects interactivity with the user consuming point cloud content and includes user-related information (e.g., header orientation information, viewport information, etc.). Specifically, when the point cloud content is used for a service that requires user interaction (e.g., self-driving services), feedback information can be provided to the content sender (e.g., sending device 10000) and / or the service provider. Depending on the embodiment, the feedback information may be used by receiving device 10004 and sending device 10000, or may not be provided.

[0097] According to an embodiment, the head orientation information may indicate information regarding the position, orientation, angle, and movement of the user's head. The receiving device 10004 according to an embodiment may calculate viewport information based on the head orientation information. Viewport information refers to the area of the point cloud video being viewed by the user (that is, the area the user is currently viewing). In other words, the viewport information refers to the area the user is currently viewing within the point cloud video. In other words, the viewport or viewport area may represent the area the user is viewing within the point cloud video. The viewpoint is the point the user is viewing within the point cloud video and may represent the center point of the viewport area. In other words, the viewport is the area centered around the viewpoint, and the size and shape of the area may be determined by the field of view (FOV). Therefore, the receiving device 10004 may extract viewport information based on the vertical or horizontal FOV supported by the device and the head orientation information. Furthermore, the receiving device 10004 may perform gaze analysis, etc., based on the head orientation information and / or viewport information, to determine how the user consumes the point cloud video, the area within the point cloud video the user is gazing at, and the duration of the gaze. According to an embodiment, the receiving device 10004 may transmit feedback information including gaze analysis results to the transmitting device 10000. According to an embodiment, a device such as a VR / XR / AR / MR display may extract a viewport area based on the position / orientation of the user's head and the vertical or horizontal FOV supported by the device. According to an embodiment, the head orientation information and viewport information may be referred to as feedback information, signaling information, or metadata.

[0098] According to an embodiment, feedback information may be obtained during the rendering and / or display process. The feedback information may be obtained by one or more sensors included in the receiving device 10004. According to an embodiment, the feedback information may be provided by the renderer 10007 or a separate external element (or device, component, etc.). Figure 1The dashed line in the figure represents the process of sending feedback information secured by the renderer 10007. This feedback information can not only be sent to the transmitting side but also consumed by the receiving side. That is, the point cloud content providing system can process (encode / decode / render) point cloud data based on this feedback information. For example, the point cloud video decoder 10006 and renderer 10007 can prioritize decoding and rendering point cloud video only for the area currently being viewed by the user based on this feedback information (i.e., header orientation information and / or viewport information).

[0099] Furthermore, the receiving device 10004 may transmit feedback information to the transmitting device 10000. The transmitting device 10000 (or the point cloud video data encoder 10002) may perform encoding operations based on the feedback information. Thus, the point cloud content providing system can efficiently process necessary data (e.g., point cloud data corresponding to the user's header position) based on the feedback information, rather than processing (encoding / decoding) the entire point cloud data, and provide the point cloud content to the user.

[0100] According to an embodiment, the transmitting device 10000 may be referred to as an encoder, a transmitting device, a transmitter, a transmitting system, etc., and the receiving device 10004 may be referred to as a decoder, a receiving device, a receiver, a receiving system, etc.

[0101] According to the embodiment Figure 1 Point cloud data processed in a point cloud content providing system (through a series of processes such as acquisition, encoding, transmission, decoding, and rendering) can be referred to as point cloud content data or point cloud video data. Depending on the embodiment, point cloud content data can be used as a concept that encompasses metadata or signaling information related to point cloud data.

[0102] Figure 1 The elements of the illustrated point cloud content providing system may be implemented by hardware, software, a processor, and / or a combination thereof.

[0103] Figure 2 is a block diagram illustrating a point cloud content providing operation according to an embodiment.

[0104] Figure 2 The block diagram shows Figure 1 As described above, the point cloud content providing system may process point cloud data based on a point cloud compression codec (eg, G-PCC).

[0105] According to an embodiment, a point cloud content providing system (e.g., the point cloud transmitting device 10000 or the point cloud video acquisition unit 10001) can acquire a point cloud video (20000). A point cloud video is represented by a point cloud belonging to a coordinate system used to express a 3D space. According to an embodiment, a point cloud video can include a PLY (Polygon file format or Stanford Triangle format) file. When a point cloud video has one or more frames, the acquired point cloud video can include one or more PLY files. A PLY file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of a point. The position of each point can be represented by parameters (e.g., X, Y, and Z axis values) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include point properties (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc. of each point). A point has one or more attributes. For example, a point can have a color attribute or both color and reflectivity attributes. Depending on the embodiment, geometry may be referred to as position, geometry information, geometry data, position information, position data, etc., and attribute may be referred to as attribute, attribute information, attribute data, etc. The point cloud content providing system (e.g., the point cloud transmitting device 10000 or the point cloud video acquiring unit 10001) may obtain point cloud data from information related to the point cloud video acquiring process (e.g., depth information, color information, etc.).

[0106] According to an embodiment, a point cloud content providing system (e.g., the transmitting device 10000 or the point cloud video encoder 10002) may encode point cloud data (20001). The point cloud content providing system may encode point cloud data based on point cloud compression codec. As described above, point cloud data may include geometric information and attribute information about points. Therefore, the point cloud content providing system may perform geometric encoding for encoding geometry and output a geometric bitstream. The point cloud content providing system may perform attribute encoding for encoding attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system may perform attribute encoding based on geometric encoding. The geometric bitstream and attribute bitstream according to an embodiment may be multiplexed and output as one bitstream. The bitstream according to an embodiment may also include signaling information related to geometric encoding and attribute encoding.

[0107] According to an embodiment, the point cloud content providing system (eg, the transmitting device 10000 or the transmitter 10003) may transmit the encoded point cloud data (20002). Figure 1 As shown, the encoded point cloud data can be represented by a geometry bitstream and an attribute bitstream. Furthermore, the encoded point cloud data can be transmitted in the form of a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit it in the form of a file or fragment.

[0108] According to an embodiment, a point cloud content providing system (eg, receiving device 10004 or receiver 10005) may receive a bitstream containing encoded point cloud data and demultiplex the bitstream.

[0109] A point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) may decode the encoded point cloud data (e.g., geometry bitstream, attribute bitstream) transmitted in a bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) may decode the point cloud video data based on signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) may decode the geometry bitstream to reconstruct the position (geometry) of the points. The point cloud content providing system may reconstruct the attributes of the points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) may reconstruct the point cloud video based on the position using the reconstructed geometry and the decoded attributes.

[0110] According to an embodiment, a point cloud content providing system (e.g., receiving device 10004 or renderer 10007) may render the decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) may use various rendering methods to render the geometry and attributes decoded by the decoding process. Points in the point cloud content may be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user via a display (e.g., a VR / AR display, a general display, etc.).

[0111] According to an embodiment, the point cloud content providing system (e.g., receiving device 10004) may obtain feedback information (20005). The point cloud content providing system may encode and / or decode point cloud data based on the feedback information. Figure 1 The feedback information and operations described are the same, so their detailed description is omitted.

[0112] Figure 3 An exemplary point cloud encoder according to an embodiment is shown.

[0113] Figure 3 Show Figure 1An example of a point cloud video encoder 10002 is provided. The point cloud encoder reconstructs and encodes point cloud data (e.g., point locations and / or attributes) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) based on network conditions or applications. When the total size of the point cloud content is large (e.g., 60 Gbps of point cloud content at 30 fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system may reconstruct the point cloud content based on a maximum target bitrate to provide the point cloud content according to network conditions, etc.

[0114] As reference Figure 1 and Figure 2 As described, the point cloud encoder can perform both geometry encoding and attribute encoding. Geometry encoding is performed before attribute encoding.

[0115] The point cloud video encoder according to an embodiment includes a coordinate transformer (Transforming Coordinates) 30000, a quantizer (Quantizing and Removing Points (Voxelization)) 30001, an octree analyzer (Analyzing Octree) 30002, a surface approximation analyzer (Analyzing Surface Approximation) 30003, an arithmetic encoder (Arithmetic Coding) 30004, a geometry reconstructor (Reconstructing Geometry) 30005, a color transformer (Transforming Color) 30006, an attribute transformer (Transforming Attributes) 30007, a RAHT transformer 30008, an LOD generator (Generated LOD) 30009, a lifting transformer (Lifting) 30010, a coefficient quantizer (Quantizing Coefficients) 30011, and / or an arithmetic encoder (Arithmetic Coding) 30012. Figure 3 In the point cloud encoder of FIG, the coordinate transformer 30000, the quantizer 30001, the octree analyzer 30002, the surface approximation analyzer 30003, the arithmetic encoder 30004, and the geometry reconstructor 30005 can be grouped together and referred to as a geometry encoder. The color transformer 30006, the attribute transformer 30007, the RAHT transformer 30008, the LOD generator 30009, the lifting transformer 30010, the coefficient quantizer 30011, and / or the arithmetic encoder 30012 can be grouped together and referred to as an attribute encoder.

[0116] The coordinate transformer 30000, quantizer 30001, octree analyzer 30002, surface approximation analyzer 30003, arithmetic encoder 30004, and geometry reconstructor 30005 may perform geometry coding. According to embodiments, geometry coding may include octree geometry coding, prediction tree geometry coding, direct coding, triplet geometry coding, and entropy coding. Direct coding and triplet geometry coding may be applied selectively or in combination. Geometric coding is not limited to the above examples.

[0117] As shown in the figure, the coordinate converter 30000 according to an embodiment receives a position and converts it into coordinates. For example, the position can be converted into position information in a three-dimensional space (e.g., a three-dimensional space represented by an XYZ coordinate system). According to an embodiment, the position information in the three-dimensional space can be referred to as geometric information.

[0118] The quantizer 30001 according to an embodiment quantizes geometry. For example, the quantizer 30001 may quantize points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). The quantizer 30001 performs a quantization operation by multiplying the difference between the minimum position value and the position value of each point by a preset quantization scale value, and then rounding the resulting value to the nearest integer. As a result, one or more points may have the same quantized position (or position value). The quantizer 30001 according to an embodiment performs voxelization based on the quantized positions to reconstruct the quantized points. Just like pixels (the smallest unit containing 2D image / video information), points in point cloud content (or 3D point cloud video) according to an embodiment may be included in one or more voxels. The term voxel, a composite of volume and pixel, refers to the 3D cubic space generated by dividing a 3D space into units (unit = 1.0) based on axes representing the 3D space (e.g., the X, Y, and Z axes). Quantizer 30001 can match point groups in 3D space to voxels. Depending on the embodiment, a voxel may include only one point. Depending on the embodiment, a voxel may include one or more points. To represent a voxel as a point, the position of the voxel's center can be set based on the positions of one or more points included in the voxel. In this case, the attributes of all positions included in a voxel can be combined and assigned to the voxel.

[0119] The octree analyzer 30002 according to an embodiment performs octree geometry encoding (or octree encoding) to present voxels in an octree structure. The octree structure represents points that are matched to voxels based on an octal tree structure.

[0120] The surface approximation analyzer 30003 according to an embodiment may analyze and approximate an octree. The octree analysis and approximation according to an embodiment is a process of analyzing a region including a plurality of points to efficiently provide an octree and voxelization.

[0121] According to an embodiment, the arithmetic encoder 30004 performs entropy coding on the octree and / or approximate octree. For example, the coding scheme includes arithmetic coding. As a result of the coding, a geometry bitstream is generated.

[0122] The color converter 30006, attribute converter 30007, RAHT converter 30008, LOD generator 30009, lifting converter 30010, coefficient quantizer 30011, and / or arithmetic encoder 30012 perform attribute encoding. As described above, a point may have one or more attributes. Attribute encoding according to embodiments is also applied to the attributes of a point. However, when an attribute (e.g., color) includes one or more elements, attribute encoding is applied independently to each element. Attribute encoding according to embodiments includes color transform codecs, attribute transform codecs, region adaptive hierarchical transform (RAHT) codecs, interpolation-based hierarchical nearest neighbor prediction (prediction transform) codecs, and interpolation-based hierarchical nearest neighbor prediction (lifting transform) codecs with an update / lifting step. Depending on the point cloud content, the RAHT codecs, prediction transform codecs, and lifting transform codecs described above may be selectively used, or a combination of one or more codecs may be used. Attribute encoding according to embodiments is not limited to the above examples.

[0123] The color converter 30006 according to an embodiment performs color conversion codec to convert the color values (or textures) included in the attributes. For example, the color converter 30006 may convert the format of the color information (e.g., from RGB to YCbCr). Alternatively, the operation of the color converter 30006 according to an embodiment may be applied based on the color values included in the attributes.

[0124] According to an embodiment, the geometry reconstructor 30005 reconstructs (decompresses) an octree and / or an approximate octree. The geometry reconstructor 30005 reconstructs the octree / voxel based on the results of analyzing the point distribution. The reconstructed octree / voxel may be referred to as reconstructed geometry (restored geometry).

[0125] According to an embodiment, the attribute converter 30007 performs attribute conversion to convert attributes based on reconstructed geometry and / or locations where geometry encoding is not performed. As described above, since attributes depend on geometry, the attribute converter 30007 can convert attributes based on reconstructed geometry information. For example, based on the position value of a point included in a voxel, the attribute converter 30007 can convert the attributes of the point at that position. As described above, when the center position of a voxel is set based on the positions of one or more points included in the voxel, the attribute converter 30007 converts the attributes of one or more points. When triplet geometry encoding is performed, the attribute converter 30007 can convert attributes based on the triplet geometry encoding.

[0126] Attribute converter 30007 performs attribute conversion by calculating the average of the attributes or attribute values (e.g., the color or reflectivity of each point) of neighboring points within a specific position / radius from the center position (or position value) of each voxel. Attribute converter 30007 can apply weights based on the distance from the center to each point when calculating the average. Thus, each voxel has a position and a calculated attribute (or attribute value).

[0127] The attribute converter 30007 can search for neighboring points within a specific location / radius from the center of each voxel based on a KD tree or Morton code. A KD tree is a binary search tree that supports a data structure capable of managing points based on location, allowing for rapid nearest neighbor searches (NNS). The Morton code is generated by representing the coordinates (e.g., (x, y, z)) representing the 3D positions of all points as bit values and then mixing the bits. For example, when the coordinates representing the point position are (5, 9, 1), the bit values of the coordinates are (0101, 1001, 0001). Mixing the bit values in the order of z, y, and x according to the bit index produces 010001000111. This value is represented as the decimal number 1095. That is, the Morton code value of the point with coordinates (5, 9, 1) is 1095. The attribute converter 30007 can sort the points based on the Morton code values and perform NNS using a depth-first traversal process. After the attribute transform operation, KD tree or Morton code is used when NNS is needed in another transform process for attribute encoding and decoding.

[0128] As shown, the transformed attributes are input to the RAHT transformer 30008 and / or the LOD generator 30009.

[0129] The RAHT transformer 30008 according to an embodiment performs RAHT encoding and decoding for predicting attribute information based on the reconstructed geometric information. For example, the RAHT transformer 30008 may predict attribute information of a higher-level node in the octree based on attribute information associated with a lower-level node in the octree.

[0130] According to an embodiment, the LOD generator 30009 generates a level of detail (LOD) to perform predictive transform encoding and decoding. According to an embodiment, the LOD is the level of detail of the point cloud content. As the LOD value decreases, the detail of the point cloud content degrades. As the LOD value increases, the detail of the point cloud content increases. Points can be categorized by LOD.

[0131] The lifting transformer 30010 according to an embodiment performs a lifting transform codec that transforms point cloud attributes based on weights. As described above, the lifting transform codec can be optionally applied.

[0132] The coefficient quantizer 30011 according to an embodiment quantizes the properties of the attribute codec based on the coefficients.

[0133] The arithmetic encoder 30012 according to an embodiment encodes quantized properties based on arithmetic coding.

[0134] Although not shown in the figure, Figure 3 The elements of the point cloud encoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may execute the above Figure 3 In addition, one or more processors may be operable or executable to perform at least one of the operations and / or functions of the elements of the point cloud encoder. Figure 3 The one or more memories of the embodiment may include high-speed random access memory, or non-volatile memory (e.g., one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0135] Figure 4 Examples of octrees and occupancy codes according to an embodiment are shown.

[0136] As reference Figures 1 to 3 As described, the point cloud content providing system (point cloud video encoder 10002) or the point cloud encoder (e.g., octree analyzer 30002) performs octree geometry encoding and decoding (or octree encoding and decoding) based on the octree structure to effectively manage the area and / or position of voxels.

[0137] Figure 4 The upper part of FIG shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes (eg, X-axis, Y-axis, and Z-axis) of the coordinate system. The octree structure is represented by two poles (0, 0, 0) and (2 d , 2 d , 2 d ) to create an octree structure. Here, 2d can be set to the value of the minimum bounding box that surrounds all points of the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following equation. In the following equation, (x int n , y int n , z int n ) represents the position (or position value) of the quantized point.

[0138]

[0139] like Figure 4 As shown in the middle of the upper part of , the entire 3D space can be divided into eight spaces according to the partition. Each divided space is represented by a cube with six faces. Figure 4 As shown in the upper right portion of the octree, each of the eight spaces is further divided based on the axes of the coordinate system (e.g., X, Y, and Z). Thus, each space is divided into eight smaller spaces. Each of these smaller spaces is also represented by a cube with six faces. This division scheme is applied until the leaf nodes of the octree become voxels.

[0140] Figure 4 The lower part shows the octree occupancy code. The occupancy code of the octree is generated to indicate whether each of the eight divided spaces generated by dividing one space contains at least one point. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of the divided space, and the child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to the child node contains at least one point, the node is assigned a value of 1. When the space corresponding to the child node does not contain a point (the space is empty), the node is assigned a value of 0. Since Figure 4 The illustrated occupancy code is 00100001, indicating that the spaces corresponding to the third and eighth child nodes, among the eight child nodes, each contain at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by an 8-bit occupancy code. The figure shows that the occupancy code for the third child node is 10000111, and the occupancy code for the eighth child node is 01001111. A point cloud encoder (e.g., arithmetic encoder 30004) according to an embodiment may perform entropy encoding on the occupancy code. To increase compression efficiency, the point cloud encoder may perform intra / inter encoding and decoding on the occupancy code. A receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to an embodiment reconstructs the occupancy code based on the occupancy code.

[0141] According to an embodiment of the point cloud encoder (eg, Figure 4 The point cloud encoder or octree analyzer 30002 may perform voxelization and octree encoding and decoding to store point locations. However, points are not always evenly distributed in 3D space, so there may be specific areas with fewer points. Therefore, performing voxelization on the entire 3D space is inefficient. For example, if a specific area contains very few points, voxelization is not required in that specific area.

[0142] Therefore, for the above-mentioned specific area (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment can skip voxelization and perform direct encoding and decoding to directly encode the point positions included in the specific area. The coordinates of the directly encoded points according to the embodiment are called direct encoding and decoding mode (DCM). The point cloud encoder according to the embodiment can also perform triplet geometry encoding based on the surface model, which is to reconstruct the point positions in the specific area (or node) based on voxels. Triplet geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and decoding and triplet geometry encoding according to the embodiment can be performed selectively. In addition, direct encoding and decoding and triplet geometry encoding according to the embodiment can be performed in combination with octree geometry encoding and decoding (or octree encoding and decoding).

[0143] To perform direct encoding and decoding, the option to use direct mode for direct encoding and decoding should be enabled. The node to which direct encoding and decoding is to be applied must not be a leaf node, and the number of points within the specific node must be less than a threshold. Additionally, the total number of points to which direct encoding and decoding is to be applied must not exceed a preset threshold. When these conditions are met, the point cloud encoder (or arithmetic encoder 30004) according to this embodiment can perform entropy encoding and decoding on point positions (or position values).

[0144] A point cloud encoder according to an embodiment (e.g., a surface approximation analyzer 30003) may determine a specific level of the octree (a level less than the depth d of the octree) and may use a surface model starting from this level to perform triplet geometry encoding to reconstruct the point positions in the node area based on voxels (triplet mode). A point cloud encoder according to an embodiment may specify the level to which triplet geometry encoding is to be applied. For example, when the specific level is equal to the depth of the octree, the point cloud encoder does not operate in triplet mode. In other words, the point cloud encoder according to an embodiment may operate in triplet mode only when the specified level is less than the depth value of the octree. A 3D cubic area of a node at a specified level according to an embodiment is called a block. A block may include one or more voxels. A block or voxel may correspond to a block. Geometry is represented as a surface within each block. A surface according to an embodiment may intersect each edge of the block at most once.

[0145] A block has 12 edges, so there are at least 12 intersections in a block. Each intersection is called a vertex. Vertices along an edge are detected when there is at least one occupied voxel adjacent to the edge across all blocks that share the edge. An occupied voxel, according to embodiments, refers to a voxel containing a point. The vertex position detected along an edge is the average position of all voxels adjacent to the edge across all blocks along the edge.

[0146] Once the vertices are detected, the point cloud encoder according to an embodiment may perform entropy encoding on the edge's start point (x, y, z), the edge's direction vector (Δx, Δy, Δz), and the vertex position value (relative position value within the edge). When triplet geometry encoding is applied, the point cloud encoder according to an embodiment (e.g., the geometry reconstructor 30005) may generate recovered geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization.

[0147] Vertices located at the edges of a block define a surface that passes through the block. According to an embodiment, the surface is a non-planar polygon. During triangle reconstruction, the surface represented by the triangles is reconstructed based on the edge start points, the edge direction vectors, and the position values of the vertices. The triangle reconstruction process is performed by i) calculating the centroid value of each vertex, ii) subtracting the centroid value from each vertex value, and iii) estimating the sum of the squares of the values obtained by the subtraction.

[0148]

[0149] Estimate the minimum value of the sum and perform the projection process according to the axis with the minimum value. For example, when the element x is the smallest, each vertex is projected on the x-axis relative to the center of the block and projected on the (y, z) plane. When the value obtained by the projection on the (y, z) plane is (ai, bi), the value of θ is estimated by atan2(bi, ai), and the vertices are sorted based on the value of θ. The following table shows the vertex combination for creating triangles according to the number of vertices. The vertices are sorted from 1 to n. Table 1 below shows that for four vertices, two triangles can be constructed according to the vertex combination. The first triangle can be composed of vertices 1, 2, and 3 among the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 among the sorted vertices.

[0150] [Table 1]

[0151] [Table 1] Triangles formed from vertices sorted in 1,…,n order

[0152]

[0153] An upsampling process is performed to add points along the edges of the triangles, and voxelization is performed. The added points are generated based on the upsampling factor and the block width. These added points are referred to as refinement vertices. According to an embodiment, the point cloud encoder may voxelize the refinement vertices. Furthermore, the point cloud encoder may perform attribute encoding based on the voxelized positions (or position values).

[0154] Figure 5 An example of point configuration in each LOD according to an embodiment is shown.

[0155] As reference Figures 1 to 4As described, the encoded geometry is reconstructed (decompressed) before attribute encoding is performed. When direct encoding is used, the geometry reconstruction operation may include changing the placement of directly encoded points (e.g., placing directly encoded points in front of the point cloud data). When triplet geometry encoding is used, the geometry reconstruction process is performed through triangle reconstruction, upsampling, and voxelization. Since attributes depend on geometry, attribute encoding is performed based on the reconstructed geometry.

[0156] The point cloud encoder (e.g., LOD generator 30009) can classify (or reorganize) points by LOD. The figure shows the point cloud content corresponding to the LOD. The leftmost frame in the figure represents the original point cloud content. The second frame from the left in the figure shows the point distribution at the lowest LOD, and the rightmost frame in the figure shows the point distribution at the highest LOD. That is, the points at the lowest LOD are sparsely distributed, while the points at the highest LOD are densely distributed. In other words, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points becomes narrower.

[0157] Figure 6 An example of point configuration for each LOD according to an embodiment is shown.

[0158] As reference Figures 1 to 5 As described, the point cloud content providing system or point cloud encoder (eg, point cloud video encoder 10002, ​ The Point Cloud Encoder or LOD Generator 30009 (of the Point Cloud Encoder) generates LODs. LODs are generated by reorganizing points into a set of refinement levels based on a set LOD distance value (or set of Euclidean distances). The LOD generation process is performed not only by the Point Cloud Encoder but also by the Point Cloud Decoder.

[0159] ​ The upper part of shows examples of points (P0 to P9) of point cloud content distributed in 3D space. ​ In , the original order represents the order of points P0 to P9 before LOD generation. ​ In the LOD-based order, the order of points generated according to the LOD is shown. Points are reorganized by LOD. In addition, higher LODs contain points belonging to lower LODs. ​ As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, LOD1, P9, P8, and P7.

[0160] As reference ​ As described, the point cloud encoder according to the embodiment may selectively or in combination perform prediction transform coding, lifting transform coding, and RAHT transform coding.

[0161] According to an embodiment, a point cloud encoder can generate predictors for each point to perform predictive transform coding and decoding for setting prediction attributes (or predicted attribute values) for each point. Specifically, N predictors can be generated for N points. According to an embodiment, the predictor can calculate weights (=1 / distance) based on the LOD value of each point, index information about neighboring points within a set distance of each LOD, and the distance to the neighboring points.

[0162] The predicted attribute (or attribute value) according to an embodiment is set to the average of the values obtained by multiplying the attribute (or attribute value) (e.g., color, reflectivity, etc.) of the neighboring points set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 30011) according to an embodiment can quantize and inverse quantize the residual (which can be referred to as a residual attribute, residual attribute value, attribute prediction residual, etc.) obtained by subtracting the predicted attribute (or attribute value) of each point from the attribute (attribute value) of each point. Tables 2 and 3 below show the quantization process.

[0163] [Table 2]

[0164]

[0165] [Table 3]

[0166]

[0167] When the predictor for each point has neighboring points, the point cloud encoder (e.g., the arithmetic encoder 30012) according to an embodiment may perform entropy coding and decoding on the quantized and inverse quantized residual values as described above. When the predictor for each point has no neighboring points, the point cloud encoder (e.g., the arithmetic encoder 30012) according to an embodiment may perform entropy coding and decoding on the attributes of the corresponding point without performing the above operation.

[0168] According to an embodiment, a point cloud encoder (e.g., lifting transformer 30010) may generate a predictor for each point, set the calculated LOD, register neighboring points in the predictor, and set weights based on the distance to the neighboring points to perform lifting transform encoding and decoding. The lifting transform encoding and decoding according to an embodiment is similar to the predictive transform encoding and decoding described above, but differs in that weights are cumulatively applied to attribute values. The process of cumulatively applying weights to attribute values according to an embodiment is configured as follows.

[0169] 1) Create an array called quantized weights (QW) to store the weight values for each point. All elements of QW are initially set to 1.0. Multiply the QW value of the predictor index of the neighboring node registered in the predictor by the weight of the predictor for the current point, and add the resulting values.

[0170] 2) Boosting prediction process: The value obtained by multiplying the attribute value of the point by the weight is subtracted from the existing attribute value to calculate the predicted attribute value.

[0171] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0172] 4) Accumulate the weight calculated by multiplying the weight calculated for all predictors by the weight stored in the QW corresponding to the predictor index into the updateweight array as the index of the neighbor node. Accumulate the value obtained by multiplying the attribute value of the neighbor node index by the calculated weight into the update array.

[0173] 5) Boosting update process: Divide the attribute values of the update array of all predictors by the weight value of the updateweight array of the predictor index, and add the existing attribute value to the value obtained by the division.

[0174] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated by the boosting update process by the weight (stored in QW) updated by the boosting prediction process. The point cloud encoder (e.g., coefficient quantizer 30011) according to an embodiment quantizes the predicted attribute value. Furthermore, the point cloud encoder (e.g., arithmetic encoder 30012) performs entropy encoding and decoding on the quantized attribute value.

[0175] A point cloud encoder according to an embodiment (e.g., RAHT transformer 30008) may perform RAHT transform encoding and decoding, in which attributes associated with lower-level nodes in the octree are used to predict attributes of higher-level nodes. RAHT transform encoding and decoding is an example of attribute intra encoding and decoding by scanning backward through the octree. A point cloud encoder according to an embodiment scans the entire region starting from voxels and repeats a merging process at each step, merging voxels into larger blocks, until the root node is reached. The merging process according to an embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on nodes directly above empty nodes.

[0176] The following equation represents the RAHT transformation matrix. In this equation, Indicates level The average attribute value of the voxels at . Can be based on and To be calculated. and The weight is and .

[0177]

[0178] here, is the low-pass value and is used during the next highest level of merging. Denotes the high-pass coefficient. The high-pass coefficient at each step is quantized and subjected to entropy coding (eg, encoded by the arithmetic encoder 30012). The weight is calculated as .pass and Create the root node as follows.

[0179]

[0180] The gDC values are also quantized and subjected to entropy coding like the high-pass coefficients.

[0181] ​ A point cloud decoder according to an embodiment is shown.

[0182] ​ The point cloud decoder shown is an example of a point cloud decoder and can perform decoding operations, which are ​ The inverse process of the encoding operation of the point cloud encoder is shown.

[0183] As reference ​ and ​ As described, the point cloud decoder can perform both geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0184] The point cloud decoder according to an embodiment includes an arithmetic decoder (arithmetic decoding) 7000, an octree synthesizer (synthesize octree) 7001, a surface approximation synthesizer (synthesize surface approximation) 7002 and a geometry reconstructor (reconstruct geometry) 7003, an inverse coordinate transformer (inverse transform coordinates) 7004, an arithmetic decoder (arithmetic decoding) 7005, an inverse quantizer (inverse quantization) 7006, a RAHT transformer 7007, an LOD generator (generate LOD) 7008, an inverse lifting (inverse lifting) 7009 and / or an inverse color transformer (inverse transform color) 7010.

[0185] The arithmetic decoder 7000, the octree synthesizer 7001, the surface approximation synthesizer 7002, the geometry reconstructor 7003, and the coordinate inverse transformer 7004 may perform geometry decoding. The geometry decoding according to the embodiment may include direct decoding and triplet geometry decoding. Direct encoding and decoding and triplet geometry decoding are selectively applied. The geometry decoding is not limited to the above example, and as a reference ​ The inverse process of the geometric encoding described is performed.

[0186] The arithmetic decoder 7000 according to the embodiment decodes the received geometry bitstream based on arithmetic coding and decoding. The operation of the arithmetic decoder 7000 corresponds to the inverse process of the arithmetic encoder 30004.

[0187] The octree synthesizer 7001 according to an embodiment may generate an octree by acquiring an occupancy code (or information about the geometry obtained as a result of decoding) from the decoded geometry bitstream. ​ Describe the configuration in detail.

[0188] When triplet geometry encoding is applied, the surface approximation synthesizer 7002 according to an embodiment may synthesize a surface based on the decoded geometry and / or the generated octree.

[0189] According to an embodiment, the geometry reconstructor 7003 may regenerate the geometry based on the surface and / or decoded geometry. ​ As described, direct coding and triplet geometry coding are selectively applied. Therefore, the geometry reconstructor 7003 directly imports the position information about the points to which direct coding is applied and adds them. When triplet geometry coding is applied, the geometry reconstructor 7003 can reconstruct the geometry by performing the reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 30005. Details and References ​ The reconstructed geometry may include a point cloud frame or screen that does not contain attributes.

[0190] The coordinate inverse transformer 7004 according to an embodiment may acquire a point position by transforming the coordinates based on the reconstructed geometry.

[0191] The arithmetic decoder 7005, inverse quantizer 7006, RAHT converter 7007, LOD generator 7008, inverse lifting 7009, and / or color inverse converter 7010 may perform attribute decoding. Attribute decoding according to embodiments includes region adaptive hierarchical transform (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) decoding, and interpolation-based hierarchical nearest neighbor prediction (lifting transform) decoding with an update / lifting step. The above three decoding schemes may be selectively used, or a combination of one or more decoding schemes may be used. Attribute decoding according to embodiments is not limited to the above examples.

[0192] The arithmetic decoder 7005 according to an embodiment decodes the attribute bitstream through arithmetic coding and decoding.

[0193] The inverse quantizer 7006 according to an embodiment inversely quantizes information about a decoded attribute bitstream or an attribute obtained as a result of decoding, and outputs the inversely quantized attribute (or attribute value). Inverse quantization may be selectively applied based on attribute encoding of the point cloud encoder.

[0194] According to an embodiment, the RAHT transformer 7007, the LOD generator 7008, and / or the inverse lifter 7009 may process the reconstructed geometry and inverse quantized properties. As described above, the RAHT transformer 7007, the LOD generator 7008, and / or the inverse lifter 7009 may selectively perform a decoding operation corresponding to the encoding of the point cloud encoder.

[0195] The color inverse converter 7010 according to an embodiment performs inverse transform coding to inversely transform the color value (or texture) included in the decoded attribute. The operation of the color inverse converter 7010 can be selectively performed based on the operation of the color converter 30006 of the point cloud encoder.

[0196] Although not shown in the figure, ​ The elements of the point cloud decoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may execute the above ​ In addition, one or more processors may be operable or executed to perform at least one or more of the operations and / or functions of the elements of the point cloud decoder. ​ A software program and / or instruction set for the operation and / or functionality of the elements of a point cloud decoder.

[0197] ​ A transmitting device according to an embodiment is shown.

[0198] ​ The sending device shown is ​ The sending device 10000 (or ​ An example of a point cloud encoder. ​ The sending device shown can be executed with reference to ​ The transmitting apparatus according to the embodiment may include a data input unit 8000, a quantization processor 8001, a voxelization processor 8002, an octree occupancy code generator 8003, a surface model processor 8004, an intra / inter codec processor 8005, an arithmetic codec (coder) 8006, a metadata processor 8007, a color transform processor 8008, an attribute transform processor 8009, a prediction / lifting / RAHT transform processor 8010, an arithmetic codec 8011 and / or a transmission processor 8012.

[0199] The data input unit 8000 according to the embodiment receives or acquires point cloud data. The data input unit 8000 may perform the same operation and / or acquisition method as the point cloud video acquisition unit 10001 (or refer to ​The acquisition process 20000) is the same or similar operation and / or acquisition method as described above.

[0200] The data input unit 8000, the quantization processor 8001, the voxelization processor 8002, the octree occupancy code generator 8003, the surface model processor 8004, the intra / inter codec processor 8005 and the arithmetic codec 8006 perform geometric coding. ​ The geometric encoding described is the same or similar, so its detailed description is omitted.

[0201] The quantization processor 8001 according to an embodiment quantizes geometry (eg, position values of points). The operation and / or quantization of the quantization processor 8001 is related to the reference ​ The operation and / or quantization of the quantizer 30001 described above are the same or similar. ​ Same as those described.

[0202] The voxelization processor 8002 according to the embodiment voxelizes the quantized position value of the point. The voxelization processor 8002 may perform the same as the reference. ​ The operation and / or voxelization process of the quantizer 30001 described above is the same or similar to the operation and / or process described above. ​ Same as those described.

[0203] The octree occupancy code generator 8003 according to an embodiment performs octree encoding and decoding on the voxelized position of the point based on the octree structure. The octree occupancy code generator 8003 can generate an occupancy code. The octree occupancy code generator 8003 can perform the same as the reference code. ​ and ​ The operations and / or methods of the point cloud encoder (or octree analyzer 30002) described herein are the same or similar to the operations and / or methods described herein. ​ Same as those described.

[0204] According to an embodiment, the surface model processor 8004 may perform triplet geometry encoding based on the surface model to reconstruct the point position in a specific area (or node) based on voxels. ​ The operations and / or methods described herein are the same as or similar to the operations and / or methods of the point cloud encoder (e.g., surface approximation analyzer 30003). ​ Same as those described.

[0205] The intra / inter codec processor 8005 according to an embodiment may perform intra / inter codec on the point cloud data. The intra / inter codec processor 8005 may perform codecs that are the same as or similar to intra / inter codecs. According to an embodiment, the intra / inter codec processor 8005 may be included in the arithmetic codec 8006.

[0206] According to an embodiment, the arithmetic codec 8006 performs entropy coding on the octree and / or approximate octree of the point cloud data. For example, the coding scheme includes arithmetic coding. The arithmetic codec 8006 performs the same or similar operations and / or methods as the arithmetic coder 30004.

[0207] The metadata processor 8007 according to an embodiment processes metadata (e.g., set values) regarding point cloud data and provides it to necessary processing processes such as geometry coding and / or attribute coding. Furthermore, the metadata processor 8007 according to an embodiment may generate and / or process signaling information related to geometry coding and / or attribute coding. According to an embodiment, the signaling information may be encoded separately from the geometry coding and / or attribute coding. According to an embodiment, the signaling information may be interleaved.

[0208] The color conversion processor 8008, the attribute conversion processor 8009, the prediction / lifting / RAHT conversion processor 8010, and the arithmetic codec 8011 perform attribute coding. ​ The attribute codes described are the same or similar, so their detailed description is omitted.

[0209] The color transform processor 8008 according to an embodiment performs color transform coding to transform the color value included in the attribute. The color transform processor 8008 may perform color transform coding based on the reconstructed geometry. The reconstructed geometry is compared with the reference ​ In addition, its execution is the same as that of reference ​ The operations and / or methods of the color converter 30006 are the same as or similar to those described above, and detailed description thereof is omitted.

[0210] The attribute transformation processor 8009 according to an embodiment performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometry encoding is not performed. ​ The operations and / or methods of the attribute converter 30007 described above are the same as or similar to those of the attribute converter 30007. Detailed description thereof is omitted. The prediction / lifting / RAHT transform processor 8010 according to the embodiment may encode the transformed attributes by any one or a combination of RAHT codec, prediction transform codec, and lifting transform codec. The prediction / lifting / RAHT transform processor 8010 performs the same as the reference codec. ​The operations of the RAHT transformer 30008, the LOD generator 30009 and the lifting transformer 30010 described above are identical or similar to at least one operation. In addition, the prediction transform codec, the lifting transform codec and the RAHT transform codec are the same as those of the reference ​ Those described are the same, so detailed descriptions thereof are omitted.

[0211] The arithmetic codec 8011 according to an embodiment may encode the properties of the codec based on arithmetic coding. The arithmetic codec 8011 performs the same or similar operations and / or methods as those of the arithmetic encoder 30012.

[0212] According to an embodiment, the transmission processor 8012 may transmit individual bitstreams containing coded geometry and / or coded attributes or metadata information, or transmit a bitstream containing coded geometry and / or coded attributes and metadata information. When the coded geometry and / or coded attributes and metadata information according to an embodiment are configured into a bitstream, the bitstream may include one or more sub-bitstreams. According to an embodiment, the bitstream may include signaling information and slice data, and the signaling information includes a sequence parameter set (SPS) for sequence-level signaling, a geometry parameter set (GPS) for signaling of geometry information encoding and decoding, an attribute parameter set (APS) for signaling of attribute information encoding and decoding, and a patch parameter set (TPS) for patch-level signaling. Slice data may include information about one or more slices. According to an embodiment, one slice may include one geometry bitstream Geom0. 0 and one or more attribute bitstreams Attr0 0 and Attr1 0 .

[0213] A slice is a sequence of syntactic elements that represents a whole or part of a coded point cloud frame.

[0214] The TPS according to an embodiment may include information about each of the one or more tiles (for example, coordinate information and height / size information about a bounding box). The geometry bitstream may include a header and a payload. The header of the geometry bitstream according to an embodiment may include a parameter set identifier (geom_parameter_set_id), a tile identifier (geom_tile_id), and a slice identifier (geom_slice_id) included in the GPS, as well as information about the data contained in the payload. As described above, the metadata processor 8007 according to an embodiment may generate and / or process signaling information and send it to the transmission processor 8012. According to an embodiment, an element that performs geometry encoding and an element that performs attribute encoding may share data / information with each other as indicated by the dotted line. The transmission processor 8012 according to an embodiment may perform operations and / or transmission methods that are the same as or similar to those of the transmitter 10003. Details and References ​ and ​ Those described are the same, so their description is omitted.

[0215] ​ A receiving apparatus according to an embodiment is illustrated.

[0216] ​ The receiving device shown is ​ An example of a receiving device 10004. ​ The receiving device shown can execute and refer to ​ One or more operations and methods that are the same or similar to those described for the point cloud decoder.

[0217] The receiving apparatus according to the embodiment may include a receiver 9000, a receiving processor 9001, an arithmetic decoder 9002, an octtree reconstruction processor based on an occupancy code 9003, a surface model processor (triangle reconstruction, upsampling, voxelization) 9004, an inverse quantization processor 9005, a metadata parser 9006, an arithmetic decoder 9007, an inverse quantization processor 9008, a prediction / lifting / RAHT inverse transform processor 9009, a color inverse transform processor 9010, and / or a renderer 9011. Each decoding element according to the embodiment may perform an inverse process of the operation of the corresponding encoding element according to the embodiment.

[0218] The receiver 9000 according to the embodiment receives point cloud data. The receiver 9000 may perform the same ​ The operation and / or receiving method of the receiver 10005 are the same as or similar to the operation and / or receiving method of the receiver 10005. A detailed description thereof is omitted.

[0219] The reception processor 9001 according to an embodiment may acquire a geometry bitstream and / or an attribute bitstream from the received data. The reception processor 9001 may be included in the receiver 9000.

[0220] The arithmetic decoder 9002, the octtree reconstruction processor 9003 based on the occupancy code, the surface model processor 9004 and the inverse quantization processor 9005 may perform geometric decoding. ​ The geometric decoding described is the same or similar, so its detailed description is omitted.

[0221] The arithmetic decoder 9002 according to an embodiment may decode the geometry bitstream based on arithmetic coding and may perform operations and / or coding that are the same as or similar to those of the arithmetic decoder 7000 .

[0222] According to an embodiment, the occupancy code-based octree reconstruction processor 9003 can reconstruct the octree by obtaining an occupancy code from the decoded geometry bitstream (or information about the geometry obtained as a result of decoding). The occupancy code-based octree reconstruction processor 9003 performs operations and / or methods that are the same as or similar to the operations and / or octree generation methods of the octree synthesizer 7001. When triplet geometry coding is applied, the surface model processor 9004 according to an embodiment can perform triplet geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on the surface model method. The surface model processor 9004 performs operations that are the same as or similar to the operations of the surface approximation synthesizer 7002 and / or the geometry reconstructor 7003.

[0223] The inverse quantization processor 9005 according to an embodiment may inversely quantize the decoded geometry.

[0224] According to an embodiment, the metadata parser 9006 can parse metadata (eg, setting values) contained in the received point cloud data. The metadata parser 9006 can deliver the metadata to the geometry decoding and / or attribute decoding. ​ The metadata described are the same, so a detailed description thereof is omitted.

[0225] The arithmetic decoder 9007, the inverse quantization processor 9008, the prediction / lifting / RAHT inverse transform processor 9009 and the color inverse transform processor 9010 perform attribute decoding. ​ The attribute decoding of at least one of the descriptions is the same or similar, so its detailed description is omitted.

[0226] According to an embodiment, the arithmetic decoder 9007 can decode the attribute bitstream through arithmetic coding and decoding. The arithmetic decoder 9007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 9007 performs the same or similar operations and / or coding and decoding as the arithmetic decoder 7005.

[0227] The inverse quantization processor 9008 according to an embodiment may inversely quantize the decoded attribute bitstream. The inverse quantization processor 9008 performs the same or similar operation and / or inverse quantization method as the inverse quantizer 7006.

[0228] The prediction / lifting / RAHT inverse transform processor 9009 according to an embodiment can process the geometric and inverse quantized properties of the reconstruction. ​ The color inverse transform processor 9010 according to the embodiment performs inverse transform encoding and decoding to inversely transform the color value (or texture) included in the decoded attribute. The color inverse transform processor 9010 performs the same or similar operations and / or decoding as the RAHT transformer 7007, the LOD generator 7008 and / or the inverse lifter 7009. ​ The operations and / or inverse transform codec of the color inverse converter 7010 are the same as or similar to those of the color inverse converter 7010. The renderer 9011 according to an embodiment may render point cloud data.

[0229] ​ An exemplary structure operable in conjunction with a method / apparatus for transmitting / receiving point cloud data according to an embodiment is shown.

[0230] ​ The structure of FIG1000 represents a configuration in which at least one of a server 1060, a robot 1010, a self-driving vehicle 1020, an XR device 1030, a smartphone 1040, a home appliance 1050, and / or a head-mounted display (HMD) 1070 is connected to the cloud network 1000. The robot 1010, the self-driving vehicle 1020, the XR device 1030, the smartphone 1040, or the home appliance 1050 is referred to as a device. In addition, the XR device 1030 may correspond to a point cloud data (PCC) device according to an embodiment or may be operatively connected to a PCC device.

[0231] The cloud network 1000 may represent a network that constitutes a part of a cloud computing infrastructure or exists in a cloud computing infrastructure. Here, the cloud network 1000 may be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0232] The server 1060 may be connected to at least one of the robot 1010 , the self-driving vehicle 1020 , the XR device 1030 , the smart phone 1040 , the home appliance 1050 , and / or the HMD 1070 via the cloud network 1000 , and may assist at least a portion of processing of the connected devices 1010 to 1070 .

[0233] The HMD 1070 represents one of implementation types of an XR device and / or a PCC device according to an embodiment. The HMD type device according to an embodiment includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0234] Hereinafter, various embodiments of devices 1010 to 1050 to which the above-described technology is applied will be described. ​ The illustrated devices 1010 to 1050 may be operatively connected / coupled to the point cloud data transmitting device and receiving device according to the above-described embodiments.

[0235] ​

[0236] The XR / PCC device 1030 may employ PCC technology and / or XR (AR+VR) technology and may be implemented as an HMD, a head-up display (HUD) provided in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a fixed robot, or a mobile robot.

[0237] The XR / PCC device 1030 can analyze 3D point cloud data or image data acquired through various sensors or from external devices and generate positional data and attribute data about the 3D points. Thus, the XR / PCC device 1030 can obtain information about the surrounding space or real objects and render and output XR objects. For example, the XR / PCC device 1030 can match an XR object that includes auxiliary information about the recognized object with the recognized object and output the matched XR object.

[0238] ​

[0239] The XR / PCC device 1030 may be implemented as a smart phone 1040 by applying PCC technology.

[0240] Smartphone 1040 can decode and display point cloud content based on PCC technology.

[0241] ​

[0242] The self-driving vehicle 1020 may be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0243] Self-driving vehicle 1020, which utilizes XR / PCC technology, may represent a self-driving vehicle equipped with a means for providing XR images, or a self-driving vehicle that serves as a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, self-driving vehicle 1020 may be distinguished from and operatively connected to XR device 1030.

[0244] The self-driving vehicle 1020, which has a means for providing XR / PCC images, can obtain sensor information from sensors including cameras and output generated XR / PCC images based on the obtained sensor information. For example, the self-driving vehicle 1020 may have a HUD and output XR / PCC images thereto, thereby providing passengers with XR / PCC objects corresponding to real objects or objects displayed on the screen.

[0245] When an XR / PCC object is output to the HUD, at least a portion of the XR / PCC object may be output to overlap with the real object that the passenger's eyes are pointing at. On the other hand, when an XR / PCC object is output on a display provided within a self-driving vehicle, at least a portion of the XR / PCC object may be output to overlap with the object on the screen. For example, the self-driving vehicle 1020 may output XR / PCC objects corresponding to objects such as a road, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, and a building.

[0246] Virtual reality (VR) technology, augmented reality (AR) technology, mixed reality (MR) technology, and / or point cloud compression (PCC) technology according to an embodiment are applicable to various devices.

[0247] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. On the other hand, AR technology refers to a technology that displays a virtually created CG image on an image of a real object. MR technology is similar to the above-mentioned AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology is different from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with properties equivalent to real objects. More specifically, an example of the application of MR technology is holographic services.

[0248] Recently, VR, AR, and MR technologies are often referred to as scaled reality (XR) technologies, rather than being clearly distinguished from each other. Therefore, embodiments of the present disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are applicable to such technologies.

[0249] The PCC method / apparatus according to the embodiment may be applied to a vehicle providing a self-driving service.

[0250] Vehicles providing self-driving services are connected to the PCC device for wired / wireless communication.

[0251] When a point cloud data (PCC) transmission / reception device according to an embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services (which can be provided in conjunction with self-driving services) and transmit it to the vehicle. If the PCC transmission / reception device is installed in a vehicle, the PCC transmission / reception device can receive / process content data related to AR / VR / PCC services based on a user input signal input via a user interface device and provide it to the user. The vehicle or user interface device according to an embodiment can receive the user input signal. The user input signal according to an embodiment may include a signal indicating a self-driving service.

[0252] The point cloud data transmitting method / device according to the embodiment is interpreted as referring to ​ The transmitting device 10000, the point cloud video encoder 10002 and the transmitter 10003, ​ Get 20000 - Encode 20001 - Send 20002, ​ Point cloud video encoder, ​ The sending device, ​ device, ​ A term for a transmitting device.

[0253] The point cloud data receiving method / device according to the embodiment is interpreted as referring to ​ The receiving device 10004, the receiver 10005 and the point cloud video decoder 10006, ​ Send 20002 - decode 20003 - render 20004, ​ Point cloud video decoder, ​ receiving device, ​ device, ​ A term for a receiving device, etc.

[0254] The method / apparatus for sending and receiving point cloud data according to an embodiment may be referred to simply as the method / apparatus according to an embodiment.

[0255] According to the embodiment, geometric data, geometric information, position information, etc. constituting point cloud data are interpreted as having the same meaning. Attribute data, attribute information, etc. constituting point cloud data are interpreted as having the same meaning.

[0256] The method / apparatus according to an embodiment may process point cloud data taking into account scalable transmission.

[0257] Regarding the method / apparatus according to the embodiment, this document discloses a method for efficiently supporting selective decoding of partial data according to receiver performance or transmission speed when transmitting / receiving point cloud data. Specifically, the present disclosure proposes a technology for increasing the efficiency of scalable encoding and decoding, wherein an encoder at a transmitting end can selectively deliver information required by a decoder at a receiving end for compressed data, and the decoder can decode it, wherein the encoding and decoding units can be configured as independent slices in the sense of tree level, LOD, layer group unit, etc.

[0258] Specifically, the present disclosure proposes a method for increasing the efficiency of scalable codecs among point cloud data compression methods. Here, scalable codecs are a technology for gradually changing the data resolution according to the receiver's request / processing speed / performance / transmission bandwidth, etc., so that the compressed data can be efficiently delivered from the sending end and decoded at the receiving end. To this end, the technical application of the present disclosure can be combined with a layer configuration-based grouping for efficiently delivering point cloud data. Specifically, this article proposes a method for configuring slice segments to be more suitable for scalable PCC services and sending / receiving them when a direct compression mode is used for position compression. In addition, a compression method for efficiently storing and transmitting large point cloud data that is widely distributed and has a high point density can be used.

[0259] Reference ​ and ​ In the point cloud data sending / receiving device (which may be referred to as an encoder / decoder for short) of the embodiment shown in , point cloud data consists of a collection of points. Each of the points includes geometric information (or geometry or geometric data) and attribute information (or attributes or attribute data). The geometric information is the three-dimensional position information (xyz) about each point. That is, the position of each point is represented by parameters in a coordinate system representing the three-dimensional space (for example, parameters representing the three axes of the space (x, y, z), such as the X-axis, the Y-axis, and the Z-axis). The attribute information represents the color (RGB, YUV, etc.), reflectivity, normal vector, transparency, etc. of the point. In point cloud compression (PCC), octree-based compression is performed to effectively compress the non-uniform distribution in the three-dimensional space, and the attribute information is compressed based on the octree compression. ​ and ​ The point cloud video encoder and point cloud video decoder shown in FIG can perform operations according to embodiments through their respective components.

[0260] According to an embodiment, a transmitting device compresses geometric information (e.g., position) and attribute information (e.g., color, brightness, reflectivity, etc.) regarding point cloud data and transmits the compressed information to a receiving device. Point cloud data can be configured according to an octree structure with layers based on levels of detail, or levels of detail (LoD). Scalable point cloud data encoding, decoding, and presentation can then be performed based on this configuration. In this case, depending on the performance or transmission rate of the receiving device, only a portion of the point cloud data may be decoded or presented. However, currently, there is no method for preemptively removing unnecessary data.

[0261] In other words, in the case where only a portion of the scalable point cloud compressed bitstream needs to be sent (e.g., only a portion of a layer is decoded in scalable decoding), it is not possible to select and send the necessary portion. Therefore, the transmitting device should re-encode the necessary portion after decoding, such as ​ , or when the entire data is transmitted to the receiving device, the receiving device should selectively apply necessary data after decoding, as shown in ​ As shown in .

[0262] However, in ​ In this case, there may be a delay due to the time used for decoding and re-encoding. ​ In the case of fixed bandwidth, bandwidth efficiency may be reduced due to unnecessary data being transmitted to the receiving device. In addition, when using a fixed bandwidth, data quality may need to be reduced to send data.

[0263] Therefore, the method / apparatus according to an embodiment may provide slicing, so that a point cloud is divided into regions for processing.

[0264] Specifically, in the case of octree-based position compression, entropy-based compression and direct compression can be used together. In this regard, this paper proposes a slice configuration for effectively exploiting scalability.

[0265] Furthermore, with respect to the method / apparatus according to the embodiment, a slice segmentation structure of point cloud data may be defined, and scalable layers and slice structures for scalable transmission may be signaled.

[0266] The method / apparatus according to the embodiment may divide and process a bitstream into specific units for efficient bitstream delivery and decoding.

[0267] The method / apparatus according to the embodiment can achieve selective transmission and decoding of layered point cloud data in a bitstream unit.

[0268] Units according to embodiments may be referred to as levels of detail (LODs), layers, slices, etc. LODs are the same term as LODs in attribute data codecs, but in another sense can refer to data units used in the hierarchical structure of a bitstream. LODs can be based on the hierarchical structure of point cloud data, for example, the concept of a depth (level) in an octree or multitree corresponding to one depth or a bundle of two or more depths. Similarly, layers are provided to generate sub-bitstreams. They correspond to one depth or a bundle of two or more depths and can correspond to one LOD or two or more LODs. Furthermore, slices are the units used to configure sub-bitstreams and can correspond to one depth, a portion of one depth, or two or more depths. Furthermore, slices can correspond to one LOD, a portion of one LOD, or two or more LODs. Depending on the embodiment, LODs, layers, and slices may correspond to each other, or one of these may be included within the other. Furthermore, units according to the embodiment may include LODs, layers, slices, layer groups, or subgroups, and references to these may be interchangeable. Depending on the embodiment, in the octree structure, layers, depths, levels, and depth levels may have the same meaning.

[0269] ​ is a diagram illustrating an example of layer-based point cloud data configuration according to an embodiment. ​ An example of an octree structure is illustrated, in which the depth level of the root node is set to 0 and the depth level of the leaf node is set to 7.

[0270] The method / apparatus according to the embodiment may be configured as follows ​ to encode and decode the point cloud data.

[0271] The layering of point cloud data according to the embodiment may have a layer structure in terms of SNR, spatial resolution, color, temporal frequency, bit depth, etc. depending on the application field, and the layers may be configured in the direction of increasing data density based on an octree structure or an LOD structure.

[0272] That is, when LOD is generated based on the octree structure, LOD can be defined as increasing in the direction of increasing detail, that is, the direction of increasing octree depth level. In this disclosure, layer can have the same meaning as level, depth and depth level.

[0273] refer to ​, for example, in an octree structure having 7 depth levels other than a root node level (or root level), LOD 0 is configured to include levels from the root node level to octree depth level 4, and LOD 1 is configured to include levels from the root node level to octree depth level 5, and LOD 2 is configured to include levels from the root node level to octree depth level 7.

[0274] ​ - (a) illustrates the bitstream structure of geometry data according to an embodiment, and ​ - (b) illustrates a bitstream structure of attribute data according to an embodiment.

[0275] The method / apparatus according to the embodiment can be based on ​ The hierarchical generation of LOD is done by using the octree structure shown in the figure and configured as follows ​ -(a) and ​ - The geometry bitstream and attribute bitstream shown in (b).

[0276] The sending device according to this embodiment can divide the bit stream obtained through point cloud compression into a geometry bit stream and an attribute bit stream according to the data type to send the bit stream.

[0277] In this case, each bitstream may be composed of multiple slices and transmitted. According to an embodiment, the geometry bitstream (e.g., ​ -(a)) and attribute bitstreams (e.g., ​ -(b)) can be configured as one slice and delivered regardless of layer information or LoD information. In this case, in order to use only some layers or LoDs, an operation of decoding the bitstream, an operation of selecting only the portion to be used and removing unnecessary portions, and an operation of re-encoding based only on the necessary information should be performed.

[0278] The present disclosure proposes a method to divide a bitstream into layers (or LoDs) to avoid such unnecessary intermediate operations.

[0279] ​ is a diagram illustrating an example of a bitstream configuration for delivering a bitstream divided into layers (or LoDs) according to an embodiment.

[0280] For example, in the case of the LoD-based PCC technology, the lower LoD is included in the higher LoD. That is, the higher LoD includes all points of the lower LoD. In addition, when information about points included in the current LoD but not included in the previous LoD, that is, new points added to each LoD, is defined as R (remaining or reserved), the transmitting device can divide the initial LoD information and the information R newly included in each LoD into independent units (for example, slices) and transmit them, as shown in FIG. ​ As shown in .

[0281] In other words, a set of new points added to configure each LoD compared to the previous LoD may be defined as information R. ​ An example is illustrated in which LoD1 includes LoD0 and information R1, and LoD2 includes LoD1 and information R2.

[0282] According to an embodiment, points sampled for one or more octree depth levels can be determined as data belonging to information R. That is, a set of points sampled for one or more octree depth levels (i.e., points matching occupied nodes) can be defined as information R. According to another embodiment, points sampled for one octree depth level can be divided into multiple pieces of information R according to a predetermined criterion. In this case, various criteria for dividing one octree depth level into multiple pieces of information R can be considered as follows. For example, when dividing one octree depth level into M pieces of information R, the M pieces of information R can be configured so that the data in the information R has continuous Morton codes, or can be grouped so that the data in the information R has the same remainder obtained by dividing the Morton code sequence index by M. Alternatively, the M pieces of information R can be configured by grouping data at the same position when grouped into sibling nodes. According to another embodiment, if necessary, some sample points at multiple octree depth levels can be determined as information R.

[0283] exist ​ In the example shown in FIG, according to an embodiment, both the geometry bitstream and the attribute bitstream are divided into three slices. Each slice includes a header and a payload (also known as a data unit) containing the actual data (e.g., geometry data, attribute data). The header may include information about the corresponding slice. In addition, the header may further include reference information related to the previous slice, the previous LoD, or the previous layer used for LoD configuration.

[0284] For example, reference ​The geometry bitstream is divided into slices carrying geometry data belonging to LoD0, slices carrying geometry data belonging to information R1, and slices carrying geometry data belonging to information R2. The attribute bitstream is divided into slices carrying attribute data belonging to LoD0, slices carrying attribute data belonging to information R1, and slices carrying attribute data belonging to information R2.

[0285] The receiving method / apparatus according to the embodiment can receive a bitstream divided into LODs or layers, and can efficiently decode only data to be used without a complicated intermediate process.

[0286] In this regard, various embodiments may be applied to transmit a bitstream.

[0287] For example, the geometry bitstream and the attribute bitstream may be delivered separately. Alternatively, the geometry bitstream and the attribute bitstream may be multiplexed into one bitstream and delivered.

[0288] When each bitstream contains LoD0 and one or more pieces of information R, the delivery order of LoD0 and the one or more pieces of information R may vary.

[0289] exist ​ In the example of , the geometry bitstream and the attribute bitstream are delivered separately. In this case, LoD0 including the geometry bitstream and two pieces of information R (R1, R2) is delivered in sequence, and LoD0 including the attribute bitstream and two pieces of information R (R1, R2) is delivered in sequence.

[0290] ​ FIGURE 2 illustrates an exemplary bitstream ordering used when multiplexing a geometry bitstream and an attribute bitstream into one bitstream according to an embodiment.

[0291] The sending method / device according to the embodiment can send the following information in series when sending a bit stream: ​ In this operation, depending on the data type, the entire geometry data (or geometry information) may be sent first, followed by the attribute data (or attribute information). In this case, the geometry data can be quickly reconstructed based on the transmitted bitstream information.

[0292] For example, reference ​ , the layer (LOD) containing geometry data can be positioned first in the bitstream, and the layer (LOD) containing attribute data can be positioned after the geometry layer. Because attribute data depends on geometry data, the layer (LOD) containing geometry data can be positioned before the layer (LOD) containing attribute data. ​The diagram illustrates an example where LoD0, containing geometry data and two pieces of information R(R1, R2), is delivered sequentially, and then LoD0, containing attribute data and two pieces of information R(R1, R2), is delivered sequentially. This placement can vary depending on the embodiment. Furthermore, references can be made between geometry headers, and between attribute and geometry headers.

[0293] ​ FIG. 2 illustrates another exemplary bitstream sorting method when a geometry bitstream and an attribute bitstream are multiplexed into one bitstream according to an embodiment.

[0294] When sending a bit stream, the sending method / device according to the embodiment can send the following serially: ​ In this case, bitstreams containing geometry data and attribute data constituting the same layer can be bundled and transmitted. In this case, when compression technology for decoding geometry and attributes in parallel is used, decoding execution time can be shortened. In this regard, information that needs to be processed first can be processed first (low LoD and geometry should be prioritized over attributes).

[0295] ​ The diagram shows an example where LoD0 containing geometry data, LoD0 containing attribute data, information R1 containing geometry data, information R1 containing attribute data, information R2 containing geometry data, and information R2 containing attribute data are transmitted in this order. In this case, these positions can be adjusted depending on the embodiment. Furthermore, references can be made between geometry headers, and between attribute headers and geometry headers.

[0296] The transmitting / receiving method / device according to the embodiment can effectively select the layer (or LoD) required by the application field at the bitstream level when transmitting and receiving the bitstream. ​ When the geometric information shown in is grouped and sent, blank sections may appear in the middle after the bitstream level is selected. In this case, the bitstream may need to be reprocessed.

[0297] By following ​ The layers shown in are used to bundle and send geometric data and attribute data, and necessary information can be selectively delivered and / or unnecessary information can be selectively removed according to the application field, such as ​ -(a) to ​ -(c) or ​ -(a) to ​ - as shown in (c).

[0298] ​ -(a) to ​- (c) is a diagram illustrating an example of symmetric geometric property selection according to an embodiment.

[0299] For example, reference ​ -(a) to ​ (c) When a portion of the bitstream needs to be selected according to an embodiment, the transmitting device selects and transmits only data up to LoD1 (i.e., LoD0+R1), and removes information R2 corresponding to the upper layer (i.e., the new portion of LoD2) from the bitstream and does not transmit it. In the case of symmetric geometric attribute selection, geometric data and attribute data of the same layer are simultaneously selected and transmitted, or simultaneously selected and removed.

[0300] ​ -(a) to ​ (c) is a diagram illustrating an example of asymmetric geometric attribute selection according to an embodiment. In the case of asymmetric geometric attribute selection, only one of geometric data and attribute data of the same layer is selected and transmitted or removed.

[0301] For example, reference ​ -(a) to ​ (c) When selecting a portion of the bitstream according to an embodiment, the transmitting device selects and transmits LoD1 (LoD0 + R1) containing both geometry and attribute data, LoD1 (LoD0 + R1) containing attribute data, and R2 containing geometry data. R2 containing attribute data is removed from the bitstream and not transmitted. In other words, for attribute data, data other than the upper layer (R2) is selected and transmitted. For geometry data, data from all layers (from level 0 (root level) to level 7 (leaf level) in the octree structure) is transmitted.

[0302] When a portion of the bitstream needs to be selected according to an embodiment, the ​ -(a) to ​ -(c) symmetric geometric property selection method, ​ -(a) to ​ - The asymmetric geometric attribute selection method of (c), or a combination of the symmetric geometric attribute selection method and the asymmetric geometric attribute selection method to select a portion of the bitstream.

[0303] The above-mentioned segmentation of the bitstream, selection of the bitstream parts, etc. are intended to support the scalability of point cloud data.

[0304] refer to ​ , when point cloud data is represented in an octree structure and divided into LODs (or layers), scalable encoding / decoding (scalability) can be supported.

[0305] Scalability functionality according to embodiments may include slice-level scalability and / or octree-level scalability.

[0306] The LoD according to an embodiment may be used as a unit indicating a set of one or more octree layers. In addition, the LoD may mean a bundle of octree layers to be configured on a slice-by-slice basis.

[0307] The LOD according to the embodiment may be used in a broad sense, such as a unit for detailed division of data, beyond the meaning of the LOD in attribute encoding / decoding.

[0308] That is, the spatial scalability of the actual octree layer (or scalable attribute layer) can be provided for each octree layer. However, when the scalability is configured in the slice level before bitstream parsing, the selection can be performed in the LoD level.

[0309] For example, reference ​ , in the octree structure, the levels from the root level to level 4 correspond to LoD0, and the levels from the root level to level 5 correspond to LoD1. In addition, the levels from the root level to level 8 (ie, leaf level) correspond to LoD2.

[0310] In other words, in ​ In the example, when scalability is used in the slice level, the scalable operations provided correspond to three steps of LoD0, LoD1 and LoD2, and the scalable operations that the octree structure can provide in the decoding operation correspond to 8 steps from the root level to the leaf level.

[0311] According to an embodiment, when LoD0 to LoD2 are composed of respective slices, a transcoder of a receiver or a transmitter (see ​ ) You can select only LoD0, only LoD1, or only LoD2 for scalable processing. ​ , LoD1 includes LoD0, and LOD2 includes LoD1 and LoD2.

[0312] For example, when only LoD0 is selected, the maximum octree level is 4, and in the decoding operation, one scalable layer can be selected from among octree layers 0 to 4. In this case, the receiving device can regard the node size obtainable by the maximum octree level (or depth) as a leaf node and transmit the node size through signaling information.

[0313] For example, when LoD1 is selected, layer 5 may be added. Therefore, the maximum octree level may be 5, and a scalable layer may be selected from octree layers 0 to 5 in the decoding operation. In this case, the receiving device may regard the node size obtainable by the maximum octree level (or depth) as a leaf node and may transmit the node size through signaling information. According to an embodiment, the octree depth, octree layer, and octree level may be units in which data is divided in detail.

[0314] For example, when LoD2 is selected, layers 6 and 7 may be added. Therefore, the maximum octree level may be 7, and a scalable layer may be selected from among octree layers 0 to 7 in a decoding operation. In this case, the receiving device may regard the node size obtainable through the maximum octree level (or depth) as a leaf node and may transmit the node size through signaling information.

[0315] ​ -(a) to ​ - (c) illustrates an exemplary method of configuring a slice comprising point cloud data according to an embodiment.

[0316] The transmission method / apparatus / encoder according to the embodiment may configure a G-PCC bitstream by segmenting the bitstream in a slice structure. The data unit for detailed data representation may be a slice.

[0317] For example, one or more octree levels (or depths) can match a slice.

[0318] According to the transmission method / apparatus of the embodiment, for example, an encoder, a bitstream can be configured based on a slice 41001 by scanning the nodes (points) included in the octree in the direction of the scan order 41000. A slice may include nodes at one or more levels of the octree structure, may include only nodes at a specific level, or may include only some nodes at a specific level. Alternatively, it may include only some nodes at one or more levels.

[0319] ​ -(a) illustrates an exemplary octree structure consisting of 7 slices. In this example, slice 41002 may include nodes of levels 0 to 4, and slice 41003 may include some nodes of level 5. Slice 41004 may include some other nodes of level 5, and slice 41005 may include some other nodes of level 5. That is, in ​ -(a), level 5 is divided into three slices. Similarly, in ​ -(a), level 6 (i.e., leaf level) is also divided into three slices. In other words, a slice can be composed of some nodes at a specific level.

[0320] ​ (b) illustrates an exemplary octree structure consisting of four slices. In this example, one slice includes nodes at levels 0 to 3 and some nodes at level 4, and another slice includes other nodes at level 4 and some nodes at level 5. In addition, another slice includes other nodes at level 5 and some nodes at level 6, and another slice includes other nodes at level 6.

[0321] ​ (c) illustrates an exemplary octree structure consisting of five slices. One slice consists of nodes at levels 0 to 3, and four slices consist of nodes at levels 4 to 6. That is, a slice includes some nodes at level 4, some nodes at level 5, and some nodes at level 6. In other words, in levels 4 to 6, a slice may include some data at level 4 and data at level 5 or level 6 corresponding to the child nodes of the data.

[0322] In other words, if ​ -(b) and ​ As shown in (c), when multiple octree layers are matched to a slice, only some nodes of each layer can be included in the slice. When multiple slices constitute a geometry / attribute frame in this manner, information necessary for the receiving device to configure the layers can be sent to the receiving device through signaling information. For example, the signaling information may include information about the layers included in each slice and information about the nodes included in each layer.

[0323] The encoder and the device corresponding to the encoder according to this embodiment can encode point cloud data and generate and transmit a bit stream including the encoded data and signaling information (or parameter information) related to the point cloud data.

[0324] In addition, when generating a bitstream, it is possible to generate a bitstream based on the bitstream structure according to the embodiment (for example, see ​ Therefore, a receiving device, a decoder, a corresponding device, etc. according to an embodiment can receive and parse a bitstream configured to be suitable for selectively decoding some data, thereby decoding and effectively providing only a portion of the point cloud data.

[0325] Next, scalable transmission of point cloud data will be described.

[0326] The point cloud data transmission method / apparatus according to the embodiment can scalably transmit a bit stream containing point cloud data, and the point cloud data receiving method / apparatus according to the embodiment can scalably receive and decode the bit stream.

[0327] When having ​ When a bitstream having the structure shown in FIG. 1 is used for scalable transmission, signaling information for selecting the slice required by the receiving device can be sent to the receiving device. Scalable transmission may not mean transmitting or decoding the entire bitstream, but may mean transmitting or decoding only a portion of the bitstream. Therefore, low-resolution point cloud data can be provided.

[0328] When scalable transmission is applied to an octree-based geometry bitstream according to an embodiment, for each octree layer ( ​) should allow configuring point cloud data based only on information about layers up to a specific octree level.

[0329] To this end, the target octree layer should not have dependencies on information from lower octree layers. This can be a constraint that is applied jointly to geometry codec / attribute codec.

[0330] In addition, in scalable transmission, the scalable structure used for the transmitting / receiving device to select the scalable layer needs to be sent to the receiving device. Considering the octree structure according to the embodiment, all octree layers can support scalable transmission, or scalable transmission may be allowed only for specific octree layers and lower layers. For example, when some of the octree layers are included, signaling information may be delivered to the receiving device to indicate the scalable layer including the slice. Therefore, the receiving device can determine whether the slice is necessary / unnecessary at the bitstream stage. ​ In the example of (a), level 0 (i.e., root level) to level 4 41002 may constitute a scalable layer that does not support scalable transmission, and the lower octree layers can be matched with the scalable layers in a one-to-one correspondence. Generally, scalability can be supported for the portion corresponding to the leaf node. ​ As shown in -(c), when a slice includes multiple octree layers, it can be defined that these layers should be configured with a scalable layer.

[0331] In this case, scalable transmission and scalable decoding can be used separately depending on the purpose. According to an embodiment, scalable transmission can be used so that a transmitting / receiving device can select information up to a specific layer without involving a decoder. According to an embodiment, scalable decoding can be used to select a specific layer during encoding and decoding. That is, scalable transmission can support the selection of necessary information in a compressed state (i.e., at the bitstream stage) without involving a decoder, so that a transmitting or receiving device can determine a specific layer. On the other hand, in the case of scalable decoding, encoding / decoding of only a necessary part of the information in the encoding / decoding process can be supported. Therefore, scalable decoding can be used in operations such as scalable representation.

[0332] In this case, the layer configuration for scalable transmission may differ from the layer configuration for scalable decoding. For example, in the case of scalable transmission, the lower three octree layers including the leaf nodes may constitute one layer. On the other hand, in the case of scalable decoding, when all layer information is included, scalable decoding may be enabled for each of the leaf node layer, leaf node layer-1, and leaf node layer-2.

[0333] ​ -(a) and ​ -(b) illustrates the geometric codec layer structure according to an embodiment. Specifically, ​- (a) illustrates an example of three slices generated by a layer group structure at an encoder at a transmitting end, and ​ - (b) illustrates an example of the output of partial decoding using two slices at the decoder at the receiving end.

[0334] When fine-grained slicing (FGS) is enabled, the G-PCC bitstream can be sliced into multiple sub-bitstreams. Here, fine-grained slicing can be referred to as layer group-based slicing. To effectively use the layered structure of G-PCC, each slice can include codec data from some codec layers or some regions. Segmentation or partitioning of slices paired with the codec layer structure can effectively support use cases such as scalable transmission or spatial random access.

[0335] ​

[0336] In fine-grained slicing, each slice segment may contain data encoded from layer groups defined as follows.

[0337] A layer group can be defined as a set of consecutive tree layers, where the start and end depths of the tree layers of the group can be any number in the tree depth, and the start depth can be less than the end depth. The order of data encoded and decoded in a slice segment can be the same as the order of data encoded and decoded in a single slice.

[0338] For example, consider ​ - In the geometric codec layer structure with eight codec layers shown in (a), there are three layer groups, each layer group matches a different slice. More specifically, layer group 1 for codec layers 0 to 4 matches slice 1, layer group 2 for codec layer 54 matches slice 2, and layer group 3 for codec layers 6 and 74 matches slice 3. When the first two slices (i.e., slices 1 and 2) are sent or selected, the decoded output will be partial layers 0 to 5, as shown in ​ By using slices in the layer group structure, partial decoding of codec layers can be supported without accessing the entire bitstream.

[0339] According to an embodiment, the bitstream and point cloud data can be generated based on codec layer-based slice segmentation. By slicing the bitstream at the end of the codec layer during the encoding process, the method / apparatus according to an embodiment can select relevant slices to support scalable transmission or partial decoding.

[0340] ​(a) shows a geometric codec layer structure with 8 layers, where each slice corresponds to a layer group. Layer group 1 includes codec layers 0 to 4. Layer group 2 includes codec layer 5. Layer group 3 is a group for codec layers 6 and 7. When the geometry (or attribute) has an eight-level (depth) tree structure, the bitstream can be hierarchically configured by grouping data corresponding to one or more levels (depths). Each group can be included in one slice.

[0341] ​ (b) shows the decoded output when two slices are selected from three slices. When the decoder selects Group 1 and Group 2, some layers of the tree with levels (depths) 0 to 5 are selected. This means that even without access to the entire bitstream, partial decoding of codec layers can be supported by using slices with a layer group structure.

[0342] For the partial decoding process according to an embodiment, the encoder may generate three slices based on the layer group structure. The decoder according to an embodiment may select two slices from among the three slices and perform partial decoding.

[0343] The bitstream according to an embodiment may include slices based on layer groups. Each slice may include a header containing signaling information related to the point cloud data (i.e., geometric data and / or attribute data) included in the slice. The receiving method / apparatus according to an embodiment may select a slice and decode the point cloud data included in the slice payload based on the header included in the selected slice.

[0344] Considering the use case of spatial random access in addition to the layer group structure, the method / apparatus according to the embodiment can further divide the layer group into several subgroups. The subgroups are mutually exclusive, and the set of subgroups can be the same as the layer group. Because the points of each subgroup form a boundary in the spatial domain, the subgroup can be represented by subgroup bounding box information. Based on the spatial information, the layer group and subgroup structure can support access to the region of interest (ROI) by selecting a slice that covers the region. By effectively comparing the ROI with the bounding box information about each slice, spatial random access within a frame or tile can be supported.

[0345] The method / apparatus according to an embodiment may be configured to deliver slices of point cloud data, such as ​ -As shown in (a).

[0346] According to an embodiment, the entire coded bitstream can be included in a single slice. For multiple slices, each slice can contain a sub-bitstream. The order of the slices can be the same as the order of the sub-bitstreams. In addition, each slice can be matched with a layer group in a tree structure.

[0347] Furthermore, a slice may not affect a previous slice, just as higher levels of the geometry tree do not affect lower levels.

[0348] The segmented slices according to the embodiment are very efficient in terms of error robustness, efficient transmission, region of interest support, etc.

[0349] 1) Error recovery

[0350] Compared to a single-slice structure, segmented slices are more robust to errors. In other words, when a slice contains the entire bitstream of a frame, data loss may affect the entire frame data. On the other hand, when the bitstream is segmented into multiple slices, at least one slice can be decoded without being affected by the loss even when at least one slice is lost.

[0351] 2) Scalable transmission

[0352] In the present disclosure, multiple decoders with different capabilities may be supported.

[0353] When the encoded point cloud data (i.e., point cloud compression (PCC) bitstream) is contained in a single slice, the LOD of the encoded point cloud data can be determined before encoding. Therefore, multiple pre-encoded bitstreams with different point cloud data resolutions can be sent independently, which may be inefficient in terms of large bandwidth or storage space.

[0354] When the coded point cloud data (i.e., PCC bitstream) is contained in segmented slices, a single bitstream can support different levels of decoders. From the decoder's perspective, the receiving device can select the target layer and deliver the selected portion of the bitstream to the decoder. Similarly, by using a single bitstream without partitioning the entire bitstream, the transmitting device can efficiently generate partial bitstreams.

[0355] 3) Region-based spatial scalability

[0356] In terms of G-PCC requirements according to an embodiment, region-based spatial scalability may be defined as follows: the compressed bitstream consists of one or more layers, and therefore a specific ROI has a higher density with additional layers, and layers can be predicted from lower layers.

[0357] To meet this requirement, it is necessary to support different levels of detail on a region-by-region basis. For example, in VR / AR applications, distant objects can be represented with lower precision, while nearby objects can be represented with higher precision. Furthermore, the decoder can increase the resolution of a region of interest upon request. This can be achieved using geometric octrees and scalable structures such as G-PCC, a scalable attribute codec scheme.

[0358] According to an embodiment, a decoder should access the entire bitstream based on the current slice structure containing the entire geometry or attributes, which may result in inefficiencies in bandwidth, memory, and decoder. On the other hand, when the bitstream is segmented into multiple slices and each slice contains a sub-bitstream according to a scalable layer, a decoder according to an embodiment can efficiently select a slice as needed before parsing the bitstream.

[0359] The method / apparatus according to the embodiment may generate a layer group using a tree structure (or layer structure) of point cloud data.

[0360] refer to ​ (a) As an example, the geometry codec layer structure (e.g., octree structure) contains eight layers, and three slices can be used to contain one or more layers. A group represents a group of layers. When using scalable attribute codecs, the tree structure is the same as the geometry tree structure. The same octree-to-slice mapping can be used to create attribute slice segments.

[0361] The layer group according to an embodiment means a bundle of layer structure units generated in G-PCC codec, such as an octree layer or a LoD layer.

[0362] A subgroup can be represented as a collection of adjacent nodes within a layer group. For example, it can be configured as a collection of nodes adjacent to each other in Morton code order, a collection of nodes adjacent based on distance, or a collection of nodes adjacent based on codec order. Nodes in a parent-child relationship can also exist within a subgroup.

[0363] When defining subgroups, boundaries may occur in the middle of a layer, and a parameter such as entropy_continuation_enabled_flag may be signaled to indicate whether entropy continuity is maintained at the boundary. Continuity may also be maintained by referencing the previous slice via ref_slice_id.

[0364] The tree structure according to an embodiment may be an octree structure, and the attribute layer structure or attribute codec tree according to an embodiment may include a level of detail (LOD) structure. In other words, the tree structure for point cloud data may include layers corresponding to depths or levels, and these layers may be grouped.

[0365] According to the method / apparatus of the embodiment (e.g., ​ Octree analyzer 30002 or LOD generator 30009 in ​ The octree synthesizer 7002 or LOD generator 7008 in the embodiment of the present invention can generate an octree structure of geometry or an LOD tree structure of attributes. In addition, the point cloud data can be grouped based on the layer of the tree structure.

[0366] refer to ​-(a), a plurality of layers are grouped into first to third groups. The groups can be further divided to form subgroups.

[0367] According to an embodiment, a slice may contain data from a codec of a layer group. Here, a layer group is defined as a set of consecutive tree layers. The start and end depths of a tree layer may be specific numbers in the tree depth, where the start number is less than the end number.

[0368] Although ​ - (a) illustrates a geometric codec layer structure as an example of a tree structure. A codec layer structure for attributes can also be created in a similar manner.

[0369] ​ is a diagram illustrating a layer group and subgroup structure according to an embodiment.

[0370] refer to ​ ,Point cloud data and bitstream can be distinguished and represented by bounding boxes.

[0371] exist ​ , the subgroup structure and the bounding boxes corresponding to the subgroups are shown. Layer group 2 is divided into two subgroups (group 2-1 and group 2-2), which are included in different slices, and layer group 3 is divided into four subgroups (group 3-1, group 3-2, group 3-3, and group 3-4), which are included in different slices. Given the slices and bounding box information for the layer group and subgroups, spatial access can be performed by: 1) comparing the bounding box of each slice with the ROI; 2) selecting the slice whose subgroup bounding box overlaps with the ROI; and 3) decoding the selected slice.

[0372] When considering the ROI in region 3-3, slices 1, 3, and 6 are selected as the subgroup bounding box of layer group 1 and subgroups 2-2 and 3-3 covering the ROI. To achieve efficient spatial access, it is assumed that there are no dependencies between subgroups from the same layer group. In live or low-latency use cases, time efficiency can be improved by performing selection and decoding as each slice segment is received.

[0373] When encoding geometry and / or attributes, the method / apparatus according to an embodiment may present the data as a tree 2200 consisting of layers (which may be referred to as depths, levels, etc.). Point cloud data corresponding to each layer (depth / level) may be grouped into layer groups (or groups) 45000. Layer group 2 may be further divided (segmented) into two subgroups 45002, and layer group 3 may be further divided (segmented) into four subgroups 45003. Each subgroup may be configured as a slice to generate a bitstream.

[0374] A receiving device according to an embodiment can receive a bitstream, select a specific slice from the bitstream, and decode the bounding boxes corresponding to the subgroups included in the selected slice. For example, when slice 1 is selected, bounding box 45004 corresponding to layer group 1 can be decoded. Layer group 1 may be data corresponding to the largest area. When additional detailed areas are displayed for layer group 1, the method / apparatus according to an embodiment can select slice 3 and / or slice 6 and hierarchically partially access the bounding boxes (point cloud data) of subgroup 2-2 and / or subgroup 3-3 for the detailed areas included in the area of layer group 1.

[0375] use ​ The encoding and decoding of point cloud data can be done by ​ The sending / receiving device ​ encoding and decoding, ​ Transmitting device / method, ​ Receiving device / method, ​ and 9 Transmitting / receiving device / method, ​ device, ​ 、 33 and the sending / receiving method of 34, ​ and 32 The sending / receiving device and ​ and 45 The send / receive method is executed.

[0376] ​ -(a) to ​ - (c) illustrates a representation of point cloud data based on layer groups according to an embodiment.

[0377] The apparatus / method according to the embodiment can provide efficient access to large-scale point cloud data or dense point cloud data through layer group slicing based on scalability and spatial access capabilities. Due to the high number of points and large data size, point cloud data may take a considerable time to render or display content. Therefore, as an alternative, the level of detail can be adjusted based on the viewer's interest. For example, when the viewer is far away from the scene or object, structural or global area information is more important than local detailed information. On the other hand, when the viewer moves closer to a specific area or object, detailed information about the ROI is required. Using an adaptive method, the renderer according to the embodiment can effectively provide data of sufficient quality to the viewer. ​ -(a) to ​ - (c) illustrates an example of increasing detail for three levels of viewing distance based on ROI changes.

[0378] ​ - The high-level view in (a) shows coarse details, ​- The intermediate view in (b) shows medium detail, and ​ - The low-level view in (c) shows fine-grained details.

[0379] ​ A point cloud data transmitting / receiving apparatus / method according to an embodiment is illustrated.

[0380] When layer group slices are used to generate the G-PCC bitstream, multi-resolution ROIs can be supported.

[0381] refer to ​ , multi-resolution ROIs can be supported through scalability and spatial accessibility of hierarchical slices. ​ In the embodiment of the present invention, the encoder 47001 at the transmitting end can generate bitstream slices for spatial subgroups of each layer group, or bitstream slices for octree layer groups. Upon request, slices matching the ROI of each resolution are selected and sent to the receiving end. Compared with the tile-based approach, the size of the entire bitstream is reduced because it does not include details beyond the requested ROI. At the receiving end, the decoder 47004 can combine the slices to produce three outputs, for example, 1) a high-level view output from the layer group; 2) a mid-level view output from the selected subgroups of layer group 1 and layer group 2; 3) a low-level view output of high-quality details from layer groups 2 and 3 and the selected subgroup of layer group 1. The outputs can be generated progressively, and thus the receiver can provide a viewing experience such as zooming that gradually increases the resolution from the high-level view to the low-level view.

[0382] Depending on the embodiment, encoder 47001 may correspond to a geometry encoder and / or an attribute encoder as a point cloud encoder. The encoder may slice the point cloud data based on layer groups (or multiple groups). Layers may be referred to as tree depths, LOD levels, etc. As shown in section 47002, the depth of the geometry octree and / or the levels of the attribute layers may be divided into layer groups (or subgroups).

[0383] The slice selector 47003 connected to the encoder 47001 can select segmented slices (or sub-slices) and selectively send data partially, such as layer group 1 to layer group 3.

[0384] Decoder 47004 can decode selectively transmitted partial point cloud data. For example, it can decode layer group 1 of a high-level view (which has a high depth / layer / level or an index of 0, or is close to the root). Furthermore, for a mid-level view, decoding can be performed based on layer groups 1 and 2 by increasing the index of the depth / level above layer group 1. Furthermore, for a low-level view, decoding can be performed based on layer groups 1 to 3.

[0385] refer to ​, according to an embodiment, the encoder 47001 can receive point cloud data as input and slice it into layer groups. That is, the point cloud data can be hierarchically structured and divided into multiple layer groups. The hierarchical structure can refer to an octree structure or a level of detail (LoD). Portion 47002 represents point cloud data divided into multiple layer groups. The slice selector 47003 can select a layer group (or a corresponding slice) and send the selected slice to the decoder 47004 at the receiving end. The decoder 47004 can reconstruct only layer group 1, reconstruct layer groups 1 and 2, or reconstruct all received layer groups by combining the received slices according to user needs. The layer groups are hierarchical and have different levels of detail. Reconstructing only layer group 1 can result in a wider reconstruction range and coarser details. Reconstructing all layer groups 1 to 3 can result in a narrower reconstruction range and finer details.

[0386] Scalable encoding and decoding of geometric data has been described above.

[0387] According to an embodiment, scalable coding and decoding may also be applied to attribute data.

[0388] This disclosure describes a layer-based attribute compression method for improving the transmission efficiency of scalable attribute compression.

[0389] Specifically, the present disclosure proposes a LoD generation method for partially encoding or decoding attribute data.

[0390] More specifically, the present disclosure proposes a method to efficiently perform LoD generation and nearest neighbor (NN) search in the presence of missing slices.

[0391] Furthermore, the present disclosure proposes a method of performing NN search and position correction on a per-subgroup basis.

[0392] In the present disclosure, a layer group can be further divided into multiple subgroups. Subgroups according to embodiments are mutually exclusive, and the set of subgroups can be the same as the layer group. Points in each subgroup are bounded in a spatial region, and thus a subgroup can be indicated by subgroup bounding box information. Furthermore, a layer group can also be referred to as a subgroup. In the present disclosure, the term subgroup is used interchangeably with slice or FGS. Therefore, a missing (or lost) slice can be referred to as a missing subgroup or a missing FGS.

[0393] For example, a subgroup (or subgroup or subgroup) can be represented as a set of neighboring nodes in a layer group. In this case, it can be configured as a set of nodes that are adjacent to each other in Morton code order, a set of nodes that are adjacent to each other based on distance, or a set of nodes that are adjacent to each other in codec order.

[0394] A subgroup may correspond to one LOD, or may correspond to two or more LODs. In addition, a slice may correspond to one LOD, or may correspond to two or more LODs. Therefore, in the present disclosure, subgroup may be used interchangeably with slice or FGS. Depending on the embodiment, the terms layer, depth, level, and depth level may be used interchangeably in the octree structure.

[0395] In the present disclosure, subgroups may have a parent-child relationship. In other words, a parent subgroup may have one or more child subgroups. In addition, child subgroups at the same level may be referred to as sibling subgroups.

[0396] Furthermore, while the octree-based geometry codec continues with step-by-step segmentation from the root level toward the leaf level, the subsampling performed to generate the LoD continues from the leaf level to the root level. ​ In , the arrow indicates the direction of subsampling. LOD is the level of detail of the point cloud content, where a decrease in the value of LOD indicates a degradation of the details of the point cloud content, while an increase in the value of LOD indicates an enhancement of the details of the point cloud content. ​ , LoD N-1 is the lowest level of detail and LoD N+2 is the highest level of detail.

[0397] In one embodiment, the LoD generation and NN search performed by the transmitting device of the present disclosure is performed by ​ LoD generator 30009 or ​ This embodiment is intended only to provide understanding for those skilled in the art, and LoD generation and NN search may be performed by respective blocks or modules. In one embodiment, LoD generation and NN search may be performed by the prediction / lifting / RAHT transformation processor 8010 in the receiving device of the present disclosure. ​ LoD Generator 7008 or ​ The prediction / lifting / RAHT inverse transform processor 9009 is performed. This embodiment is intended only to provide understanding for those skilled in the art, and LoD generation and NN search can be performed by respective blocks or modules.

[0398] This disclosure describes a method for generating a LoD when encoding (or decoding) geometry information based on a geometry tree (e.g., an octree) and there are missing slices (i.e., missing FGS). Furthermore, a NN search method used in this case is described.

[0399] ​ is a flowchart illustrating an exemplary LoD generation method according to an embodiment.

[0400] exist ​In

[15] , it is assumed that the geometry encoded (or decoded) points are the input for attribute encoding (or attribute decoding). In this case, the LoD can be the maximum depth of the geometry tree.

[0401] That is, subsampling for LOD generation is performed (operation 48001). Then, a determination is made as to whether the current LOD is the finest LOD for the current layer group (operation 48002). Here, FGS (or layer group-based slicing) refers to dividing a slice (i.e., a layer group) into multiple slices (i.e., subgroups). In operation 48002, a determination is made as to whether the current LOD is at the bottom of the layer group.

[0402] If the LoD is not the finest LOD of the current layer group, operation 48001 is performed. If the LoD is the finest LOD of the current layer group, operation 48003 is performed to determine whether a sub-FGS (also called a sub-slice or sub-sub-group) is missing (operation 48003). In other words, it is determined whether there is a missing FGS (or slice or sub-group).

[0403] If it is determined in operation 48003 that there is no missing FGS, operation 48001 is performed. If it is determined that there is a missing FGS, the missing node is detected (operation 48004). Then, a replacement node for the detected missing node is added to the corresponding LOD (operation 48005). In other words, if there is a missing FGS, the node required to generate the LOD is searched.

[0404] Then, it is determined whether the end of the LoD has been reached (operation 48006). When it is determined that the end of the LoD has not been reached, operation 48001 is performed. When it is determined that the end of the LoD has been reached, LOD generation is terminated and an NN search is performed based on the generated LoD.

[0405] Operation 48003 and operation 48004 are described in detail below.

[0406] As described above, the point cloud data transmission method / apparatus can send a bit stream containing point cloud data in a scalable manner based on each slice (or subgroup), and the point cloud data receiving method / apparatus according to the embodiment can receive and decode the bit stream in a scalable manner based on each slice (or subgroup).

[0407] Therefore, during decoding, the decoder at the receiving end may have all slices or some missing slices. For example, when the decoder performs partial decoding, some slices (or subgroups or FGS) may be missing. These slices may be referred to as missing (i.e., lost) slices (or subgroups or FGS).

[0408] Therefore, the encoder on the transmitting end according to the present disclosure generates LoD in consideration of these situations.

[0409] ​ is a diagram illustrating an exemplary LoD generation method used when there is a missing FGS (ie, a missing subgroup).

[0410] exist ​ , 48010 represents a parent subgroup, and 48021 and 48025 represent sub-subgroups of the parent subgroup 48010. For simplicity, 48021 will be referred to as the first sub-subgroup, and 48025 will be referred to as the second sub-subgroup. It is assumed that the second sub-subgroup 48025 is missing.

[0411] As described above, subsampling proceeds from the leaf level to the root level. Therefore, in the subsampling, the nodes in section 48022-1 of layer LoD N+2 in the first child subgroup 48021 are sibling nodes, and one of the three sibling nodes is selected as the parent node 48022. Similarly, in the right section, one of the four sibling nodes is selected as the parent node. Then, one of the two parent nodes in layer LoD N+1 is selected as the parent node 48020 of parent subgroup 48010. In other words, node 48020 in parent subgroup 48010 is a node generated by subsampling the nodes in the child subgroup in the lower layer.

[0412] However, the second child subgroup 48025 is missing. Therefore, nodes 48023 and 48024 in the parent subgroup 48010 may not be generated by subsampling the nodes in the child subgroup in the lower layer. In other words, nodes 48023 and 48024 are missing nodes in the parent subgroup that may not be generated by subsampling because the child subgroup is missing.

[0413] In one embodiment, when a child FGS is missing or lost, the missing node can be detected at the finest LoD of the parent child group (i.e., the bottom LoD of the parent child group). For this process, the intermediate (or temporary) output node of the geometry FGS corresponding to the parent child group of the current attribute can be used as a reference node list (see operation 48004). In other words, in one embodiment, when there is a missing subgroup (or FGS or slice) related to attribute information, the corresponding geometric subgroup (or geometric FGS) can be used to detect at least one missing node in the parent child group. For this purpose, the present disclosure assumes that the group structure and / or subgroup structure are the same for the geometry and attribute layers.

[0414] According to an embodiment, after comparing the subsampled nodes from the current sub-subgroup with the intermediate geometry slice output nodes, nodes from the geometry output but not in the subsampled output are determined to be missing nodes, and the geometry nodes corresponding to the missing nodes are appended to the LoD of the current subgroup. In other words, LoD N is generated from the subsampled nodes from LoD N+1 and / or the missing nodes of the current layer group. The missing nodes of the current layer group are detected from the nodes of the corresponding geometry layer group and appended to the LoD. In other words, LoD N is generated using the subsampled nodes from the sub-subgroups of the current attribute and / or the nodes from the geometry parent subgroup corresponding to the parent subgroup of the current attribute.

[0415] In this disclosure, subsampled nodes from LoD N+1 are indexed to LoD N through the octree-based subsampling process in G-PCCv1. Missing nodes caused by missing sub-FGS are then detected from the geometry FGS intermediate output. The detected missing nodes are appended to LoD N. In this disclosure, it is assumed that the LOD of an attribute corresponds to the octree level of the geometry. In other words, LOD N of an attribute corresponds to octree level N of the geometry. Therefore, in this disclosure, one of the nodes in octree N of the geometry is set to the missing node in LOD N of the attribute.

[0416] Thus, since the missing subgroup of the attribute exists in the layer group of the geometry, the missing nodes of the parent subgroup (LoD N+1) of the missing subgroup (LoD N) among the nodes of the corresponding subgroup of the geometry are used to generate LoD N. In other words, the nodes of the corresponding subgroup of the geometry are appended to LoD N.

[0417] In other words, LoD N of an attribute may include at least one subsampled node from LoD N+1 and / or at least one node from the corresponding subgroup of the geometry (ie, the missing node).

[0418] Next, a process of performing a nearest neighbor (NN) search based on the LoD generated as described above will be described below.

[0419] In other words, according to the present disclosure, when performing an NN search, only points within the same layer group (or subgroup or slice) are used to search for the NN of a point. This is because, as described above, there may be missing slices (or missing FGSs or missing subgroups). For example, during scalable transmission, slices may be missing because they are not transmitted to the receiving end. Furthermore, when there are missing slices, nodes within the missing slices cannot be referenced during the NN search. Therefore, in one embodiment, the NN search can be performed by only referencing nodes within the subgroup to which the current point to be decoded (or encoded) belongs.

[0420] ​is a diagram illustrating an exemplary method of searching for nearest neighbors according to an embodiment.

[0421] In the NN search, the index subgroup includes at least one LOD to which the point to be encoded (or decoded) belongs, while the retained subgroup includes at least one lower LOD based on the at least one LOD included in the index subgroup. According to an embodiment, the points of the at least one LOD in the index subgroup are sorted in ascending order based on the magnitude of the Morton code, and the points of the at least one LOD in the retained subgroup are also sorted in ascending order based on the magnitude of the Morton code. In one embodiment, the NN of the current point in the index subgroup (i.e., the point to be encoded or decoded) is searched among the points belonging to the at least one LOD in the retained subgroup and / or the points belonging to the at least one LOD in the index subgroup.

[0422] exist ​ 48030 is the current point to be encoded (or decoded) in the index subgroup. In one embodiment, when performing an NN search, the NN of current point 48030 may be searched among points belonging to at least one LOD in a reserved subgroup within subgroup 48033 to which current point 48030 belongs, and / or points belonging to a subgroup to which current point 48030 belongs. In other words, when searching for the NN of current point 48030, points belonging to other subgroups 48031 to which current point 48030 does not belong are not referenced.

[0423] In other words, when performing the NN search, point 9 (i.e., a node) outside the boundary of the subgroup to which the current point 48030 belongs, for which at least one neighbor node is searched (i.e., outside the boundary of the subgroup to which the current point 48030 belongs) is not considered as a neighbor of the current point 48030. This is because the subgroup 48031 may be missing.

[0424] Thus, in the present disclosure, searching for the NN of point 48030 belonging to the current LoD includes searching for prediction candidates among neighbor nodes selected from at least one higher LoD than the current LoD and the node encoded or decoded in the current LoD. Specifically, in one embodiment, to achieve rapid search speed, the search range is bounded by a cubic boundary and the number of points in the Morton code sequence. In this regard, the present disclosure proposes an additional constraint that considers subgroup boundaries when the neighbor candidate is in the same layer group as the current node, which is referred to as an intra-layer group search boundary. In other words, the search for the NN of the current point is performed only in the subgroup to which the current point belongs, which is referred to as an intra-layer group search.

[0425] Search boundary within a layer group: When neighbor candidates are in the same layer group as the current node, limit the use of neighbor nodes in the same subgroup boundary.

[0426] In other words, in the present disclosure, neighbor nodes 48037 and 48038 outside the subgroup bounding box 48033 cannot be neighbor candidates (ie, neighbor nodes or neighbor points) of the current node 48030 .

[0427] In addition, in the present disclosure, layer group adaptive positions can be used in the calculation of subgroup estimation and neighbor distance, which can prevent mismatch between encoder and decoder by considering the case of missing subgroups.

[0428] In other words, for scalable transmission, the attribute encoder creates a downsampled geometric mesh by sampling the remaining layers except for one or more skipped layers, and corrects the positions of occupied nodes, including the neighbor search target node, by upsampling based on information related to the skipped layers. Then, a NN search is performed based on the corrected positions to obtain the distances between the nodes. Attribute prediction and compression can then be performed based on the obtained NN nodes.

[0429] According to an embodiment, the layer group adaptive position is the codec's position right-shifted by the number of LODs in codec's descendant (eg, child) subgroups, and then left-shifted by the number of LODs in codec's descendant and skipped descendant subgroups.

[0430] In other embodiments of the present disclosure, in performing an NN search for a current point to find at least one neighbor node, nodes belonging to a parent slice of a slice (i.e., a sub-subgroup) to which the current point belongs may be used as reference nodes even when they are outside the boundaries of the sub-slice (i.e., sub-group).

[0431] ​ is a diagram illustrating another example method of searching for nearest neighbors according to an embodiment.

[0432] In one embodiment, an NN search may be performed by referring to node 48054 in subgroup 48053 to which current point 48050 belongs, as well as nodes 48041 and 48043 in subgroup 48040, the parent of subgroup 48053 to which current point 48050 belongs. This NN search is referred to herein as an inter-layer group search. In this case, node 48052 in another subgroup 48051 at the same layer as subgroup 48053 to which current point 48050 belongs (i.e., a sibling subgroup of the subgroup to which current point 48050 belongs) is not considered (or used) as a reference node in the NN search for current point 48050.

[0433] In other words, when the current LoD (LoD N) is 48055 at the coarsest level of the current layer group, the neighbor candidate node may be in another layer group. In this case, if the node (or neighbor candidate node) is in the parent-child group boundary, it is allowed to find neighbors across the child group boundary.

[0434] According to an embodiment, for searching boundaries across layer groups, when a neighbor candidate node is in a parent layer group, the use of neighbor nodes in the parent-child group boundary is restricted.

[0435] In addition, ​ When using layer group adaptive position, the codec position of a node within a parent node may be handled differently when the node is within the child group bounding box of the current node.

[0436] The layer group adaptive position is described below.

[0437] Nodes within the child subgroup bounding box: The codec position is right-shifted by the number of LoDs in the codec's descendant subgroups of the current subgroup, and then left-shifted by the number of LoDs in the codec's descendant subgroups and skipped descendant subgroups of the current subgroup.

[0438] Nodes outside the child subgroup bounding box: The codec position is right-shifted by the number of LoDs in the parent subgroup's codec's descendant (e.g., child) subgroups, and then left-shifted by the number of LoDs in the parent subgroup's codec's descendant and skipped descendant subgroups.

[0439] ​ is a diagram illustrating yet another exemplary method of searching for nearest neighbors according to an embodiment.

[0440] exist ​ , 48060 represents the parent subgroup, 48061 represents a neighbor candidate outside the bounding box of the subgroup 48070, 48070 represents the current point to be encoded (or decoded) within the subgroup 48070, and 48072 represents a neighbor candidate within the bounding box of the subgroup 48070. In other words, 48061 represents a node outside the bounding box of the subgroup, and 48072 represents a node within the bounding box of the subgroup.

[0441] According to an embodiment, during the NN search, when a neighbor candidate point is within the subgroup bounding box (48072), the geometric location is considered down to the finest level (e.g., bottom level) of the child subgroup (48070). On the other hand, when a neighbor candidate point is outside the subgroup bounding box (48061), the geometric location is considered down to the finest level (e.g., bottom level) of the parent subgroup. In other words, when a neighbor candidate point is outside the subgroup bounding box (48061), the child subgroup is not considered when determining the position of the point in the NN search.

[0442] ​ Another exemplary point cloud transmitting device according to an embodiment is shown. ​ The elements of the point cloud sending device illustrated in the figure can be implemented as hardware, software, a processor and / or a combination thereof.

[0443] According to an embodiment, the point cloud sending device may include a data input unit 51001, a signaling processor 51002, a geometry encoder 51003, an attribute encoder 51004 and a transmission processor 51005.

[0444] The geometry encoder 51003 and the attribute encoder 51004 can perform the same ​ Point cloud video encoder 10002, ​ Code 20001, ​ Point cloud video encoder, ​ Point cloud video encoder, ​ The attribute encoding and ​ The attributes encode some or all of the related operations.

[0445] The data input unit 51001 according to the embodiment receives or acquires point cloud data. The data input unit 51001 may perform ​ The point cloud video acquisition unit 10001 may perform some or all operations, or may perform ​ Partial or complete operation of the data input unit 8000.

[0446] The data input unit 51001 outputs the position of the point in the point cloud data to the geometry encoder 51003, and outputs the attributes of the point in the point cloud data to the attribute encoder 51004. In addition, it outputs parameters to the signaling processor 51002. According to an embodiment, these parameters can be provided to the geometry encoder 51003 and the attribute encoder 51004.

[0447] The geometry encoder 51003 constructs an octree based on the positions of the input points and performs geometry compression based on the octree. The geometry encoder 51003 performs entropy coding on the compressed geometry information and outputs the coded information to the transmission processor 51005 in the form of a geometry bitstream.

[0448] The geometry encoder 51003 reconstructs the geometry information based on the position changed by compression, and outputs the reconstructed (or decoded) geometry information to the attribute encoder 51004.

[0449] According to an embodiment, the geometry encoder 51003 constructs an octree based on the position of the input point, performs layer group-based slicing in the octree, selects one or more slices, and then compresses geometric information about the one or more selected slices. ​ The layer group-based slicing and slice-by-slice geometry compression according to the embodiment are described in detail, and thus a description thereof is skipped below to avoid redundancy.

[0450] The attribute encoder 51004 compresses the attribute information based on the position and / or reconstructed geometry information that has not yet been subjected to geometry encoding. Depending on the embodiment, the attribute information may be encoded and decoded using a combination of one or more of RAHT codecs, LOD-based predictive transform codecs, and lifting transform codecs. The attribute encoder 51004 performs entropy encoding on the compressed attribute information and outputs the information to the transport processor 51005 in the form of an attribute bitstream.

[0451] According to an embodiment, the attribute encoder 51004 performs a reference ​ In other words, in one embodiment, when there is a missing subgroup (or FGS or slice) associated with attribute information, the attribute encoder 51004 can use the corresponding subgroup of geometry (or FGS of geometry) to detect at least one missing node in the parent subgroup. According to an embodiment, after comparing the subsampled nodes from the current subgroup with the intermediate geometry slice output nodes, the missing nodes in the geometry output but not in the subsampled output are appended to the LoD of the current subgroup. In other words, the LoD N can include subsampled nodes from the subgroup of the current attribute and / or nodes from the geometry parent subgroup corresponding to the parent subgroup of the current attribute.

[0452] Furthermore, when searching for the NN of the current point, neighbors can be selected based on distance, which may include neighbor nodes outside the subgroup to which the current point belongs.

[0453] According to an embodiment, when a neighboring node belongs to a sibling subgroup of a subgroup, i.e., a subgroup at the same level, the attribute encoder 51004 does not consider neighboring nodes outside the subgroup to which the current point belongs as neighbors of the current point. In other words, based on the sub-level subgroup boundary, nodes outside the boundary are not selected as neighbors. Therefore, by imposing the restriction that only nodes within the same subgroup bounding box (or boundary) should be used when generating the predictor based on the aforementioned criteria, accurate decoding can be performed even when partially decoding segmented slices. In this case, it is possible to refer to the root layer group in all cases.

[0454] According to an embodiment, when a neighbor node outside the subgroup to which the current point belongs belongs to a parent subgroup of the subgroup, the attribute encoder 51004 regards the neighbor node as a neighbor node of the current point.

[0455] According to an embodiment, when performing the NN search, when a neighbor candidate is located within the subgroup bounding box, the attribute encoder 51004 considers the details of the geometric position of the sub-subgroup down to the finest level (e.g., the bottom level) of the sub-subgroup. On the other hand, when the neighbor candidate is located outside the subgroup bounding box, the details of the geometric position are considered down to the finest level (e.g., the bottom level) of the parent subgroup.

[0456] The attribute encoder 51004 then performs attribute prediction and compression based on the NN nodes obtained as described above.

[0457] The signaling processor 51002 may generate and / or process signaling information required for encoding / decoding / rendering geometry information and attribute information, and provide the processed information to the geometry encoder 51003, the attribute encoder 51004, and / or the transport processor 51005. Alternatively, the signaling information generated by the geometry encoder 51003, the attribute encoder 51004, and / or the transport processor 51005 may be provided to the signaling processor 51002. The signaling processor 51002 may provide information fed back from the receiving device (e.g., head orientation information and / or viewport information) to the geometry encoder 51003, the attribute encoder 51004, and / or the transport processor 51005.

[0458] In the present disclosure, signaling information (LGSI) including information related to layer group-based slices can be signaled and transmitted at the parameter set level (sequence parameter set (SPS), geometry parameter set (GPS), attribute parameter set (APS), tile parameter set (TPS) (or tile list), etc.) and / or attribute data unit (i.e., attribute slice). In other words, it can also be signaled and transmitted per codec unit (or compression unit or prediction unit) of each picture, such as a slice or tile.

[0459] The transport processor 51005 can execute ​ The operations and / or transmission methods of the transmission processor 8012 are the same as or similar to the operations and / or transmission methods of the transmission processor 8012, and can perform the same as ​ For details that will not be described below, please refer to ​ or ​ Description.

[0460] The transport processor 51005 may multiplex the geometry bitstream output from the geometry encoder 51003, the attribute bitstream output from the attribute encoder 51004, and the signaling bitstream output from the signaling processor 51002 into one bitstream. The multiplexed bitstream may be transmitted without being changed, or may be encapsulated in a file or fragment and transmitted. In one embodiment of the present disclosure, the file may be in the ISOBMFF file format.

[0461] Depending on the embodiment, the file or segment can be transmitted to a receiving device or stored in a digital storage medium (e.g., a USB drive, SD card, CD, DVD, Blu-ray disc, HDD, SSD, etc.). The transmission processor 51005 according to the embodiment can communicate with the receiving device via wired / wireless communication over a network such as 4G, 5G, or 6G. Furthermore, the transmission processor 51005 can perform necessary data processing based on the network system (e.g., a 4G, 5G, or 6G communication network system). The transmission processor 51005 can transmit the encapsulated data as needed.

[0462] ​ is a flow chart illustrating an example method of encoding point cloud data according to an embodiment. More specifically, ​ It is an icon ​ Flowchart of an example encoding method in which the attribute encoder 51004 performs LoD-based attribute encoding and decoding.

[0463] ​ The point cloud data encoding can be achieved through ​ The sending device, ​ The encoding, ​ Transmitting device / method, ​ Transmitting device / method, ​ device, ​ The encoder, ​ The sending device or a combination thereof is executed.

[0464] In the encoding method according to this embodiment, parameters such as a sequence parameter set (SPS), a geometry parameter set (GPS), an attribute parameter set (APS), and a LGSI are generated, and a layer group structure is configured. Configuration of the layer group structure can be performed during or before LGSI generation. After geometry encoding and decoding is performed, Location of Dimension (LoD) generation is performed based on the recovered (or reconstructed or derived) geometry nodes (operation 58081).

[0465] Specifically, in operation 58081, reference ​According to one embodiment, operation 58081 may include detecting at least one missing node in a parent subgroup using the corresponding geometry subgroup (or geometry FGS) based on the presence of missing subgroups (or FGSs or slices) associated with attribute information. According to an embodiment, after comparing subsampled nodes from the current subgroup with an intermediate geometry slice output node, missing nodes that are present in the geometry output but not in the subsampled output are appended to the LoD of the current subgroup. In other words, LoD N is generated from subsampled nodes from LoD N+1 and / or missing nodes of the current layer group. Missing nodes of the current layer group are detected from nodes of the corresponding geometry layer group. In other words, LoD N is generated using subsampled nodes from the current attribute's subgroup and / or nodes from the geometry parent group corresponding to the current attribute's parent subgroup. Then, during the NN search for the current point, neighbors may be selected based on distance, which may include neighbor nodes outside the subgroup to which the current point belongs.

[0466] According to an embodiment, when a neighbor node belongs to a sibling subgroup of a subgroup, that is, a subgroup at the same level, neighbor nodes outside the subgroup to which the current point belongs are not considered neighbors of the current point. In other words, based on the sub-level subgroup boundary, nodes outside the boundary are not selected as neighbors. Therefore, by imposing the restriction that only nodes within the same subgroup bounding box should be used when generating the predictor based on the aforementioned criteria, accurate decoding can be performed even when partially decoding segmented slices. In this case, reference to the root layer group can be allowed in all cases.

[0467] According to an embodiment, when a neighbor node outside the subgroup to which the current point belongs belongs to a parent subgroup of the subgroup, the neighbor node is regarded as a neighbor node of the current point.

[0468] After performing LoD generation and NN search in operation 58081 , LoDs are mapped to layer groups (operation 58082 ).

[0469] In other words, by applying layer group slicing to the attribute bitstream, attributes corresponding to the partially decoded geometry bitstream can be provided. In this case, considering that the layer group is configured based on the geometry codec layer, the attribute codec layer can be configured as follows. According to an embodiment, when partial decoding is performed by matching the LoD with the layer group according to the layer group structure configured based on the geometry codec layer, the geometry output and the attribute output can be matched.

[0470] For example, a geometry codec layer can be matched to an attribute codec layer in a one-to-one correspondence. In this case, octree-based subsampling and RAHT-based attribute codec can be performed. As another example, a layer group can be matched to an attribute codec layer in a one-to-one correspondence, and a layer group can include multiple attribute codec layers. In other words, in the case of LoD-based attribute codec, the method of generating layer groups may affect compression efficiency because performance may vary due to changes in the nearest neighbors (NN) depending on the LoD configuration.

[0471] Then, quantization weights are derived based on the LoD (operation 58083).

[0472] For example, when nodes are delivered in a separated manner due to layer group slicing, subgroup boundaries and layer group boundaries should also be considered for quantization weights. In other words, the layer group and sublayer group to which the node to be encoded (or to which quantization weights are to be applied) belongs are determined (operations 58084 and 58085). The present disclosure can support partial decoding by restricting quantization weight updates to be performed within subgroup boundaries in leaf layer groups. That is, in the present disclosure, parent-child subgroup boundaries can be aligned and restrictions can be imposed so that only nodes within child-subgroup boundaries are selected as neighbors. In layer group slicing, the root layer group may not be subdivided, and fixed weights may be used for the root layer group as a method of supporting partial decoding. As another example, quantization weights can also be adaptively applied on a subgroup-by-subgroup basis.

[0473] Thereafter, whenever the attribute codec layer is changed, a check is performed to determine whether the layer group has been changed, and also to determine whether each node subgroup in the attribute codec layer has been changed (operation 58086). In other words, one or more attribute codec layers constitute a layer group. Therefore, when an attribute codec layer is changed, the layer group to which the changed attribute codec layer belongs may or may not change. Furthermore, the subgroup to which the changed attribute codec layer belongs may or may not change. For example, a node to be encoded in the attribute codec layer before the change may belong to subgroup 3-1. However, a node to be encoded in the changed attribute codec layer may belong to subgroup 3-2. Therefore, when a subgroup or layer group is changed, attribute encoder information related to the previous subgroup can be stored, and the attribute encoder for the current subgroup can be used to ensure continuity of context, zero runs, etc. within the subgroup, as well as independence from neighboring subgroups (operation 58087). Specifically, for zero runs, higher compression efficiency can be provided when zeros appear continuously. By maintaining the continuity of zero runs within the same layer group and subgroup, additional compression efficiency can be provided when the attribute codec layer is changed. Once compression of nodes belonging to all attribute codec layers is completed, the coded bitstreams of the respective subgroups are packaged into respective slices.

[0474] Although ​The description is based on the LoD attribute codec, but the coding according to the embodiment can also be applied to other attribute codec methods, such as RAHT.

[0475] The transmitting apparatus / method according to the embodiment may perform operations including generating a LoD (operation 58081), mapping the LoD to a layer group (operation 58082), and deriving quantization weights (operation 58083). These operations may further include determining a layer group (operation 58084), determining a subgroup (operation 58085), checking whether the subgroup has changed (operation 58086), storing and retrieving context information (operation 58087), and encoding attribute information (operation 58088). The aforementioned operations may be performed per node or per LoD level.

[0476] According to an embodiment, the generation of the LoD may include subsampling, constructing a predictor, or performing a neighbor search. In the LoD generation, the construction of the predictor or the neighbor search (i.e., NN search) may be performed based on the subgroup boundaries. For example, the neighbors of the first node may be selected from the nodes within the boundary of the subgroup to which the first node belongs.

[0477] When attribute layer group slicing is applied. In this case, if all LoD layers are generated from full depth geometry data and then attribute slices are decoded, some decoding use cases may experience delays. To address this issue, this disclosure proposes a method for generating LoD based on subgroups.

[0478] In other words, in one embodiment of the present disclosure, when attribute layer group slicing is applied, LoD generation can be performed on a subgroup basis. In this case, the input nodes (or points) for LoD generation in the current subgroup can include not only the nodes of the current subgroup, but also the nodes of the parent subgroup of the current subgroup. In other words, the nodes of the current subgroup are input to a portion of the LoD generator. Furthermore, the nodes of the parent subgroup are also used when decoding the current subgroup. Specifically, in one embodiment of the present disclosure, the parent node (or parent point) in the parent subgroup selected for LoD generation in the current subgroup can be a node outside the bounding box of the current subgroup. In other words, the node in the parent subgroup selected for LoD generation in the current subgroup is a node not included in the bounding box of the current subgroup. Here, "not included in the bounds of the current subgroup" means outside the bounds of the current subgroup. For example, assume that the parent subgroup includes a first subgroup and a second subgroup, and the first subgroup is the current subgroup. The second subgroup corresponds to a subgroup not included in the bounds of the current subgroup or outside the bounds of the current subgroup. In other words, the input to the LoD generator for the current subgroup (LoD generation input) includes the nodes belonging to the current subgroup and the nodes belonging to the parent subgroup (current point + parent point), excluding any duplicate nodes. In other words, from an LoD perspective, nodes belonging to the bottom LoD layer (e.g., LoDn-1) of the parent subgroup are subsampled from the top LoD (i.e., nodes) of the child subgroup (e.g., LoDn). To avoid duplicating nodes from both LoDs within the layer group boundaries, nodes of the parent subgroup outside the current subgroup's bounding box are included.

[0479] To this end, the disclosed subgroup-based LoD generation may include operations such as generating inputs to a subgroup decoder (or LoD generator for the current subgroup, or partial LoD generator) and inheriting parent attributes. Thereafter, a prediction or lifting transform or inverse transform is performed. In one embodiment, subgroup LoD generation and attribute transform (or inverse transform) are performed for all selected slices.

[0480] In the present disclosure, the encoder of the transmitting device and the decoder of the receiving device may perform the same or different LoD generation for the current subgroup and the NN search based on the node of the current subgroup and the node of the parent subgroup of the current subgroup. For example, the encoder of the transmitting device may perform the LoD generation and NN search based on the layer group, while the decoder of the receiving device may perform the LoD generation and NN search based on the node of the current subgroup and the node of the parent subgroup of the current subgroup.

[0481] ​ Another example of the point cloud receiving device according to the embodiment is illustrated. ​ The elements of the point cloud receiving device shown in the figure can be implemented as hardware, software, a processor and / or a combination thereof.

[0482] According to an embodiment, the point cloud receiving device may include a receiving processor 61001, a signaling processor 61002, a geometry decoder 61003, an attribute decoder 61004, a geometry buffer 61005, an attribute buffer 61006 and a post-processor 61007.

[0483] Depending on the embodiment, the receiving processor 61001 may receive a single bitstream, or may receive a geometry bitstream, an attribute bitstream, and a signaling bitstream separately. When a file and / or a fragment is received, the receiving processor 61001 may decapsulate the received file and / or fragment into a bitstream.

[0484] When a single bitstream is received or decapsulated, the receiving processor 61001 may demultiplex it into a geometry bitstream, an attribute bitstream, and / or a signaling bitstream. The demultiplexed signaling bitstream may be output to the signaling processor 61002, the geometry bitstream may be output to the geometry decoder 61003, and the attribute bitstream may be output to the attribute decoder 61004.

[0485] When the geometry bitstream, attribute bitstream and / or signaling bitstream are received (or decapsulated) separately, the receiving processor 61001 can deliver the signaling bitstream to the signaling processor 61002, the geometry bitstream to the geometry decoder 61003, and the attribute bitstream to the attribute decoder 61004.

[0486] The receiving processor 61001 may divide the geometry bitstream and the attribute bitstream into slices or subgroups based on the signaling information processed by the signaling processor 61002. The geometry bitstream at the slice or subgroup level may be delivered to the geometry decoder 61003, and the attribute bitstream at the slice or subgroup level may be delivered to the attribute decoder 61004. For example, the input to the attribute decoder 61004 may include an attribute fine-grained slice bitstream (or an attribute layer group-based slice bitstream), decoded geometry data, and a layer group structure.

[0487] The signaling processor 61002 can parse and process information contained in signaling information, such as SPS, GPS, APS, TPS, and metadata, from the input signaling bitstream and provide it to the receive processor 61001, the geometry decoder 61003, the attribute decoder 61004, the geometry buffer 61005, the attribute buffer 61006, and the post-processor 61007. In another embodiment, the signaling processor 61002 can pre-parse the signaling information contained in the geometry data unit header and / or the attribute data unit header before decoding the corresponding slice (or subgroup) data. In the present disclosure, the geometry buffer 61005 can store nodes belonging to the current subgroup and / or nodes belonging to the parent subgroup of the current subgroup for geometry decoding. Similarly, the attribute buffer 61006 can store nodes belonging to the current subgroup and / or nodes belonging to the parent subgroup of the current subgroup for attribute decoding. The attribute buffer 61005 can be used as an input buffer for LoD generation.

[0488] According to an embodiment, the signaling processor 61002 can parse and process the signaling information (e.g., information related to layer group-based slices) sent by signal in the sequence parameter set and / or attribute data unit header, and provide it to the attribute decoder 61004 and the attribute buffer 61006.

[0489] According to an embodiment, the geometry decoder 61003 can perform the same decoding as the one on the compressed geometry bit stream based on the signaling information. ​ The geometry decoder 61003 performs the reverse process of the operation of the geometry encoder 51003 to reconstruct the geometry. The geometry information recovered (or reconstructed or derived) by the geometry decoder 61003 is provided to the attribute decoder 61004. The attribute decoder 61004 can perform the same decoding as the decoding of the compressed attribute bitstream based on the signaling information and the reconstructed geometry information. ​ The reverse process of the operation of the attribute encoder 51004 is used to restore (or export) attributes.

[0490] According to an embodiment, the attribute decoder 61004 may perform LoD generation and nearest neighbor (NN) search based on subgroups. ​ Describe the detailed process.

[0491] According to an embodiment, the post-processor 61005 can reconstruct and display / render point cloud data by matching the geometric information (ie, position) reconstructed and output by the geometry decoder 61003 with the attribute information recovered (or exported) and output by the attribute decoder 61004.

[0492] ​ The method of decoding point cloud data according to the embodiment is illustrated. More specifically, ​ This diagram is used to ​Flowchart of an exemplary decoding method in which the attribute decoder 61004 performs LoD-based attribute decoding.

[0493] ​ The decoding of the point cloud data in can be done by ​ receiving device, ​ Decoding, ​ Receiving device / method, ​ Receiving device / method, ​ device, ​ or any combination thereof.

[0494] The input to the attribute decoder 61004 is the attribute fine-grained slice bitstream or the attribute layer group based slice bitstream, the decoded geometry data and the layer group structure. After the geometry decoding, LoD generation and NN search are performed based on the decoded geometry data. Here, reference has been made to ​ The LOD generation and NN search are described in detail, and therefore their description will be omitted below to avoid redundancy. That is, the encoder on the transmitting end and the decoder on the receiving end can perform LOD generation and NN search in the same manner. In other words, the LOD generation and NN search process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0495] Then, weight derivation is performed based on the relationship of the nodes according to the LoD. The information generated at this time can be used in the subsequent attribute decoding process. The first attribute slice can be operated in the same way as the previous attribute slice. Then, when the layer group slice is enabled, the dependency attribute data unit header can be parsed, and the layer group identification information (dadu_layer_group_id) and subgroup identification information (dadu_subgroup_id) can be obtained. If necessary, information about the LoD can be sent by signal. Based on this information, the layer group that matches the current LoD can be searched and the parent-child group can be selected. When the layer group structure is applied to geometry and attributes in the same manner according to the present disclosure, the reference information and parent information used in geometry decoding can be used. Based on this information, the information in the dependency attribute data unit can be decoded and the point cloud data can be reconstructed.

[0496] According to an embodiment, the LoD generation and weight derivation method used by the encoder 51004 can be used for decoding.

[0497] refer to ​, the method for decoding point cloud data according to an embodiment may include generating LoD (operation 62091) and deriving quantization weights (operation 62092). Operations 62091 and 62092 may correspond to LoD generation, NN search, and weight derivation performed by the attribute encoder 51004 of the transmitting device. The decoding method according to the embodiment may generate LoD and derive quantization weights that take into account subgroup boundaries, similar to the LoD generation, NN search, and weight derivation performed during the encoding process. The operation of generating LoD may include constructing a predictor and searching NN. Because in the above reference ​ The construction of the predictor and the neighbor search are described in detail, so their detailed description will be omitted below.

[0498] The decoding method according to an embodiment may include parsing the attribute data unit header (operation 62093) and decoding the attribute data unit (operation 62094). Then, determining whether layer group slicing is enabled (operation 62095). If layer group slicing is enabled, parsing the dependency attribute data unit header (operation 62096). Then, the LoD may be mapped to the layer group (operation 62097), and a parent-child group may be selected (operation 62098) to decode the dependency attribute data unit.

[0499] refer to ​ In operation 62095, whether layer group slicing is enabled may be determined by layer_group_enabled_flag. Layer_group_enabled_flag is set to 1 to indicate that layer group slicing is enabled. Enabling the layer group slicing function may indicate that point cloud data (geometry or attributes) is grouped into layer groups or subgroups for processing.

[0500] Once the dependency attribute data unit header is parsed in operation 62096, the LoDs may be mapped to layer groups (operation 62097), a parent-child group may be selected (operation 62098), and the attributes contained in the dependency data unit may be decoded.

[0501] The dependency attribute data unit header may include layer group identification information (dadu_layer_group_id) and subgroup identification information (dadu_subgroup_id) to specify the layer group index and subgroup index. The dependency attribute data unit header may further include subgroup-specific bounding box origin and size information. The dependency attribute data unit header may further include information about the reference layer group and reference subgroup, as well as quantization weight information related to the current subgroup.

[0502] According to an embodiment, dadu_layer_group_id and dadu_subgroup_id may be used in mapping LoDs to layer groups (operation 62097). In addition, bounding box origin and size information associated with a subgroup may be used in selecting a parent subgroup for comparison of bounding box information (operation 62098).

[0503] The decoding method according to the embodiment may include performing decoding by referring to context information based on layer groups and subgroups. The context information may be referred to as context state information, state information, etc. The context information may be referred to using signaling information such as information about a reference layer group and a reference subgroup, or by deriving a parent-child group based on bounding box information related to the subgroup.

[0504] As described above, attribute encoding and decoding based on LoDs can be performed as follows. That is, after generating LoDs based on the points decoded by geometry, attribute encoding and decoding is performed using predictions based on the neighbor relationships among the nodes belonging to each LoD. Specifically, LoD generation can be performed after geometry decoding is complete. Therefore, even when the geometry / attribute bitstream consists of slices, attribute slice decoding only begins after all geometry slices in the region of interest have been decoded, introducing a delay factor when quickly decoding the region of interest.

[0505] To address this issue, the present disclosure proposes a method for generating LoD based on subgroups. In other words, in the present disclosure, geometry encoding and decoding, LoD generation, and attribute encoding and decoding can be performed on a slice / subgroup basis, which can help to more actively cope with low-latency environments.

[0506] In addition, the present disclosure can improve attribute decoding speed by considering inferred direct coding and decoding mode (IDCM) (or direct compression mode) nodes to perform LoD generation and NN search. In the present disclosure, IDCM nodes can be referred to as direct compression nodes.

[0507] In one embodiment of the present disclosure, slice / subgroup-based geometric coding and decoding, LoD generation, and attribute coding and decoding may be performed by an encoder of a transmitting device, while slice / subgroup-based geometric decoding, LoD generation, and attribute decoding may be performed by a decoder of a receiving device.

[0508] ​ Another example of a method for decoding point cloud data according to an embodiment is shown. More specifically, ​ This diagram is used by ​ Flowchart of another example of a decoding method for performing attribute decoding based on LoD by the attribute decoder 61004. That is, ​ The illustration considers an example method in which an IDCM node performs geometry decoding, LoD generation, and attribute decoding on a slice / subgroup basis.

[0509] ​ The point cloud data decoding can be done by ​ receiving device, ​ Decoding, ​ Receiving device / method, ​ Receiving device / method, ​ device, ​ decoder, or a combination thereof.

[0510] According to an embodiment, the input to the attribute decoder may include the attribute fine-grained slice bitstream (or attribute layer group based slice bitstream), decoded geometry data and layer group structure.

[0511] In addition, the attribute decoder may include a LoD generator to perform LoD generation and NN search based on the subgroup taking into account the IDCM node (or direct compression node). In the present disclosure, the LoD generator may include an input node selector configured to select (or generate or determine) an input node for LoD generation of the current subgroup from the current subgroup and the parent subgroup of the current subgroup; a partial LoD generator configured to generate the LoD of the current subgroup based on the selected input node; and a parent attribute inheritor configured to inherit parent attributes as attributes of some nodes in the current subgroup. ​ Flowchart showing an example of LoD generation performed by an input node selector, a partial LoD generator, and a parent attribute inheritor.

[0512] In the hierarchical structure (eg, tree structure) of the present disclosure, the term "node" may be used interchangeably with "point." That is, a node may correspond to a single point, or may include two or more points, depending on its position in the hierarchical structure.

[0513] That is, in ​ In

[15] , when layer group slicing is enabled, it is considered that the IDCM node generates LoDs based on subgroups, derives weights based on the LoDs generated on a subgroup basis, and performs attribute transformation (e.g., inverse prediction / lifting transformation) on points within the LoD to derive (or restore) attributes.

[0514] In addition, ​ In the case of a MPEG-4 video stream, geometry slices and corresponding attribute slices may be delivered alternately. In this case, if there are remaining geometry slices, the geometry decoder is restarted, and then the corresponding attribute decoder is operated. In the case where the geometry slices are delivered first and then the attribute slices are delivered, all attribute slices may be decoded after all geometry slices are decoded.

[0515] exist Figure 34In one embodiment, SPS, GPS, APS and LGSI can be parsed by the signaling processor 61002, and geometry decoding can be performed based on the signaling information and the geometry buffer 61005. Figure 7 or Figure 9 The geometry decoder or Figure 32 In one embodiment, the remaining operations (eg, operations 63010 to 63020) including parsing of the attribute data unit header (or dependent attribute data unit header) are performed based on the attribute buffer 61006 by Figure 7 or Figure 9 Attribute decoder or Figure 32 The attribute decoder 61004 is executed.

[0516] According to an embodiment, signaling information including slice-related information based on a layer group may be signaled in at least one of an SPS, a GPS, an APS, a TPS (also referred to as a tile list), or an attribute data unit (ie, an attribute slice).

[0517] According to an embodiment, layer group-based slicing related information may include layer_group_enabled_flag. Setting layer_group_enabled_flag to 1 indicates that layer group slicing is performed, while setting layer_group_enabled_flag to 0 indicates that layer group slicing is skipped. In the present disclosure, performing layer group slicing means grouping point cloud data (geometry and / or attributes) into layer groups or subgroups for processing.

[0518] According to an embodiment, the attribute data unit header is parsed in operation 63010. In operation 63011, when the value of layer_group_enabled_flag is 1, the attribute decoder 61004 generates LoD based on the sub-group.

[0519] In one embodiment of the present disclosure, the LoD generation based on the subgroup can be performed by a LoD generator in the attribute decoder. The LoD generator can correspond to Figure 7 LoD Generator 7008 or Figure 9 Prediction / lifting / RAHT inverse transform processor 9009.

[0520] In the present disclosure, the LoD generator must first receive input of nodes (points) for the subgroup in order to generate LoD based on the subgroup (operation 63015). That is, when layer group slicing is enabled in operation 63011 (e.g., the value of layer_group_enabled_flag is 1), operation 63015 is performed to generate the subgroup input point cloud.

[0521] According to an embodiment, in operation 63015 , in order to increase compression efficiency of subgroup-based attribute codec, at least one node (point) in a subgroup related to the current subgroup may be utilized.

[0522] In one embodiment, the point cloud input to the subgroup decoder (i.e., the LoD generator for the current subgroup, also known as the partial LoD generator) may include nodes in the current subgroup and points in the parent subgroup of the current subgroup. In this disclosure, nodes may be used interchangeably with points depending on the location of the current subgroup.

[0523] The operation of selecting (or determining) the input nodes for which LoD generation is to be performed in the current subgroup in operation 63015 is referred to herein as preprocessing. In addition, in one embodiment, operation 63015 may be performed by an input node selector (not shown). The input node selector may be referred to as a preprocessor.

[0524] According to an embodiment, in operation 63015, position correction may be performed on the nodes belonging to the current subgroup. That is, because geometry tree decoding may be performed without reaching the full depth of the geometry tree, the difference between the final geometry tree depth and the geometry tree depth for the current subgroup may be compensated. In other words, because attribute decoding may be performed without fully decoding the geometry, the difference between the final geometry tree depth and the maximum geometry tree depth for the current subgroup may be compensated.

[0525] According to the present disclosure, the position of the node input to the LoD generator (or referred to as a subgroup decoder or partial LoD generator) for the current subgroup may be corrected as shown in Equation 1 below.

[0526] [Formula 1]

[0527] treeLvlGap = Final Geometry Tree Depth – Maximum geometry tree depth to use for the current subgroup.

[0528] For example, when the final geometry tree depth is 10 and the maximum geometry tree depth for the current subgroup is 7, LoD generation is performed after compensating for the depth 3 using zero padding through shifting.

[0529] The code below shows an example of correcting the positions of nodes input to the LoD generator (also called subgroup decoder) for the current subgroup (curSubgroup) (using treeLvlGap).

[0530] if (curLayerGroup < layerGroupParams.numLayerGroupsMinus1 - _params.numSkipLayerGroups) {

[0531] auto pointCloud_subgroup_fifo = _layerGroupParams.pointCloud_tempNodes[curLayerGroup][curSubgroup];

[0532] tempPointCloud.resize(pointCloud_subgroup_fifo.getPointCount());

[0533] for (int i = 0; i < pointCloud_subgroup_fifo.getPointCount(); i++) {

[0534] tempPointCloud[i] = pointCloud_subgroup_fifo[i] << treeLvlGap;

[0535] tempIdxToPointIdx.push_back(-1);

[0536] }

[0537] tempPointCloudIdx.push_back(tempPointCloud.getPointCount());

[0538] }

[0539] According to an embodiment, operation 63015 (or input node selector) may perform position accuracy correction between different layer groups through the above process, and then generate nodes belonging to the parent subgroup and nodes belonging to the current subgroup as input to the LoD generator for the current subgroup (i.e., subgroup decoder or partial LoD generator). This is intended to consider the nodes in the parent subgroup as neighbor candidates when performing an NN search after LoD generation.

[0540] Figure 35 An example of points (or nodes) included in each subgroup according to an embodiment is illustrated.

[0541] exist Figure 35, assuming the current subgroup is subgroup C, the parent subgroup of subgroup C can be considered as subgroup A. In this case, subgroup A includes two child subgroups (subgroup B and subgroup C), and these two child subgroups (subgroup B and subgroup C) are siblings. The input to the partial LoD generator (i.e., subgroup decoder) for the current subgroup (e.g., subgroup C) can include nodes f to q belonging to the current subgroup (i.e., subgroup C), and nodes d, h, o, and r belonging to the parent subgroup (i.e., subgroup A). In other words, nodes a to e belonging to the sibling subgroup (i.e., subgroup B) for the current subgroup (i.e., subgroup C) are not included in (i.e., excluded from) the input nodes of the partial LoD generator (i.e., subgroup decoder) for the current subgroup (i.e., subgroup C). In one embodiment of the present disclosure, there is no dependency relationship between subgroup B and subgroup C.

[0542] According to an embodiment, when the input includes nodes belonging to a parent subgroup related to the current subgroup, operation 63015 (or the input node selector) may only add points that are not included in the scope of the current subgroup. This is because, according to the definition of LoD—a lower LoD should include all nodes of a higher LoD—nodes belonging to the bottom LoD (or bottom layer or bottom LoD layer) of the parent subgroup are included in the top LoD (or top layer or top LoD layer) of the child subgroup. For IDCM nodes, the above LoD definition may not apply. The processing of IDCM nodes will be described in detail later.

[0543] In other words, any node included in the parent subgroup that is a duplicate of a node included in the current child subgroup is not included in (i.e., excluded from) the input to the partial LoD generator (i.e., subgroup decoder) for the current subgroup. In this case, the IDCM node belonging to the parent subgroup may be included in the input to the partial LoD generator for the current subgroup.

[0544] Figure 36 1 illustrates an example of LoD generated by subsampling according to an embodiment. For simplicity, element 65011 corresponds to Figure 35 Element 65012 corresponds to the subgroup A in Figure 35 and is referred to as the first sub-subgroup or first subgroup, and element 65013 corresponds to Figure 35Subgroup C in is referred to as the second subgroup or second subgroup. In other words, first subgroup 65012 and second subgroup 65013 are sibling subgroups, and 65011 corresponds to the parent subgroup of first subgroup 65012 and second subgroup 65013. In one embodiment of the present disclosure, there is no dependency between first subgroup 65012 and second subgroup 65013. That is, it is assumed that there is no dependency between sibling subgroups. Furthermore, 65014 corresponds to a layer group boundary separating two layer groups, and 65015 corresponds to a subgroup boundary separating two subgroups (e.g., first subgroup 65012 and second subgroup 65012). That is, first subgroup 65012 and second subgroup 65012 constitute one layer group, and parent subgroup 65011 constitutes another layer group. In other words, a layer group can correspond to a single subgroup or can be divided into two or more subgroups. Here, "first" and "second" are terms used solely to distinguish between the two subgroups.

[0545] Furthermore, in a hierarchical structure (e.g., a tree structure) for attribute decoding, a layer group (or subgroup) can be defined as a set of consecutive layers. In each group, the highest layer is referred to as the top layer (or top LoD layer), and the lowest layer is referred to as the bottom layer (or bottom LoD layer). For example, in the parent-child group 65011, 65016 corresponds to the bottom layer (or bottom LoD layer) of the parent-child group 65011. In the second subgroup 65013, 65016 corresponds to the top layer (top LoD layer) of the second subgroup 65013.

[0546] In addition, in a subgroup, a layer can correspond to a LoD. That is, when a subgroup has two layers, two LoDs can be generated in the subgroup. For example, Figure 6 In the second subgroup 65013, two LoDs (i.e., LoDn+1 and LoDn) are generated. Similarly, in the first subgroup 65012, two LoDs (i.e., LoDn+1 and LoDn) are generated. This is because LoDs are generated based on subgroups.

[0547] In this case, when considering subsampling in the LoD generation operation, some nodes belonging to the lower LoD are subsampled and included in the higher LoD, such as Figure 36 As shown. Figure 36 In , the dotted circles represent nodes that are subsampled in the corresponding LoD.

[0548] For example, reference Figure 36In the first subgroup 65012, node a among nodes a, b, and c in LoDn+1 and node d between nodes d and e are subsampled to be included in LoDn. In addition, between nodes a and d in LoDn, node d is subsampled to be included in the bottom LoD of the parent subgroup 65011, that is, LoDn-1. Similarly, in the second subgroup 65013, node f, among nodes h to k, node l among nodes l to n, and node o among nodes o to q in LoDn+1 are subsampled and included in LoDn. In addition, node h between nodes f and h and node o between nodes l and o in LoDn are subsampled to be included in the bottom LoD of the parent subgroup, that is, LoDn-1. In one embodiment of the present disclosure, when subsampling is performed in a tree structure, at least one of the child nodes having the same parent may be subsampled.

[0549] refer to Figure 36 , among nodes d, h, and o belonging to the parent subgroup 65011, node d is a node sub-sampled from the first subgroup 65012, and nodes h and o are nodes sub-sampled from the second subgroup 65013.

[0550] In this case, when the nodes belonging to the parent subgroup 65011 are included as input to the partial LoD generator of the second subgroup 65013, assuming that the current subgroup is the second subgroup 65013, the nodes h and o subsampled from the second subgroup 65013 are excluded.

[0551] In other words, when all nodes in the parent-child group are included without considering subsampling, nodes h and o will be redundantly included. That is, nodes h and o are included in both the second child group 65013 and the parent-child group 65011. In this case, a mismatch with the encoder may occur, resulting in a decoding error by the attribute decoder.

[0552] Therefore, in one embodiment, in order to generate input for the subgroup decoder (or partial LoD generator) while avoiding redundancy of nodes, operation 63015 (or input node selector) may include only nodes outside the bounding box of the current subgroup among the nodes belonging to the parent subgroup, as shown in the following code. That is, among the nodes in the parent subgroup, only nodes outside the bounding box of the current subgroup are selected as input nodes for the partial LoD generator of the current subgroup. The selected nodes can be used as neighbor candidates in the NN search after LoD generation. Figure 36, only node d is selected from nodes d, h, and o for LoD generation of the second subgroup 65013. This is intended to avoid redundancy because nodes f and h to q belonging to the second subgroup 65013 are all included as input nodes for the partial LoD generator for the current subgroup (e.g., the second subgroup 65013). In one embodiment, the nodes generated for the LoD of the current subgroup (or by the input node selector) in operation 63015 can be stored in the input buffer ( Figure 32 In one embodiment of the present disclosure, the input nodes for the current subgroup stored in the input buffer are provided to the partial LoD generator for LoD generation in the current subgroup.

[0553] According to an embodiment, in the following code, the underlined portion is configured to select only points located outside the bounding box of the current subgroup from among the points in the parent subgroup as input points for the partial LoD generator of the current subgroup.

[0554] if (curLayerGroup > 0) {

[0555] / / Parent-child group nodes outside the current bbox

[0556] auto bbox_min = _layerGroupParams.subgrpBboxOrigin[curLayerGroup][curSubgroup];

[0557] auto bbox_max = bbox_min + _layerGroupParams.subgrpBboxSize[curLayerGroup][curSubgroup];

[0558] int treeLvlGap_parent = _sps->root_node_size_log2.max() - accLayer[parentLayerGroup];

[0559] auto pointCloud_subgroup_fifo_parent = _layerGroupParams.pointCloud_tempNodes[parentLayerGroup][parentSubgroup];

[0560] int pointCount_parentNonIdcm = pointCloud_subgroup_fifo_parent.getPointCount();

[0561] auto bbox_min_parent = _layerGroupParams.subgrpBboxOrigin[parentLayerGroup][parentSubgroup];

[0562] auto bbox_max_parent = bbox_min_parent + _layerGroupParams.subgrpBboxSize[parentLayerGroup][parentSubgroup];

[0563] int pointCount = tempPointCloud.getPointCount();

[0564] tempPointCloud.resize(pointCount + pointCount_parentNonIdcm);

[0565] int count = 0;

[0566] if (!(bbox_min == bbox_min_parent && bbox_max == bbox_max_parent) {

[0567] for (int i = 0; i < pointCount_parentNonIdcm; i++) {

[0568] auto pos = pointCloud_subgroup_fifo_parent[i] << treeLvlGap_parent;

[0569] if (!(pos >= bbox_min && pos < bbox_max)) {

[0570] int tempIdx = pointCount + count++;

[0571] tempPointCloud[tempIdx] = pos;

[0572] if (attr_sps.attr_num_dimensions_minus1 == 0)

[0573] tempPointCloud.setReflectance(tempIdx, pointCloud_subgroup_fifo_parent.getReflectance(i));

[0574] else if (attr_sps.attr_num_dimensions_minus1 == 2)

[0575] tempPointCloud.setColor(tempIdx, pointCloud_subgroup_fifo_parent.getColor(i));

[0576] tempIdxToPointIdx.push_back(-1);

[0577] }

[0578] }

[0579] }

[0580] else

[0581] tempPointCloud.resize(pointCount + count);

[0582] tempPointCloudIdx.push_back(tempPointCloud.getPointCount());

[0583] }

[0584] In this way, when attribute layer group slicing is applied, the input used to generate the LoD for the current subgroup can include not only the nodes of the current subgroup, but also the nodes of the current subgroup's parent subgroup. In other words, the nodes of the current subgroup are input to the partial LoD generator. The nodes of the current subgroup's parent subgroup are also input to the partial LoD generator to generate the LoD for the current subgroup. In one embodiment, the node belonging to the parent subgroup (parent node or parent point) selected to generate the LoD for the current subgroup is a node located outside the bounding box of the current subgroup. In other words, the node belonging to the parent subgroup selected to generate the LoD for the current subgroup is a node not included in the scope of the current subgroup. The term "not included in the scope of the current subgroup" means located outside the boundaries of the current subgroup. For example, when the parent subgroup includes a first subgroup and a second subgroup, the first subgroup is the current subgroup, and the second subgroup corresponds to a subgroup not included in the scope of the current subgroup or located outside the boundaries of the current subgroup. In other words, from an LoD perspective, nodes belonging to the bottom LoD layer (e.g., LoDn-1) of the parent subgroup are subsampled from the top LoD (i.e., nodes) of the child subgroup (e.g., LoDn). Therefore, to avoid reusing two LoDs at the layer group boundary, nodes in the parent subgroup that are outside the bounding box of the current subgroup are included.

[0585] According to an embodiment, when there is no IDCM node (or directly compressed node) in either the current subgroup or the parent subgroup, the nodes in the current subgroup and the nodes in the parent subgroup are sequentially stored in the input buffer 61006 so as to be provided as input to the partial LoD generator as described above. In order to prevent the redundant use of the same node by the partial LoD generator, the node in the parent subgroup that is outside the bounding box of the current subgroup is selected as the input for the partial LoD generation in the current subgroup.

[0586] In this case, the current sub-group and / or the parent sub-group of the current sub-group may not include an IDCM node or may include one or more IDCM nodes.

[0587] For example, in Figure 36 , node 66011 may be a top-level IDCM node belonging to the current subgroup, and node 66012 may be a bottom-level IDCM node belonging to the parent subgroup.

[0588] That is, when the hierarchical structure for attribute decoding is in a tree form, some subgroups may include at least one IDCM node.

[0589] According to an embodiment, the present disclosure defines a node that satisfies the following conditions as an IDCM node, and some subgroups may include at least one IDCM node that satisfies the conditions.

[0590] 1) Parent-based eligibility: From the perspective of the parent of the current node (point), the current node is the only occupied child node. From the perspective of the grandparent, the number of occupied child nodes (i.e., occupied sibling nodes of the parent node) is at most one (which means two occupied child nodes).

[0591] 2) 6N eligible condition: From the perspective of the parent node, the current node is the only occupied child node, and none of the 6 neighbors (nodes with shared faces) are occupied.

[0592] Nodes that meet the above conditions are referred to herein as IDCM nodes (or points), and nodes that do not meet the above conditions are referred to herein as non-IDCM nodes (or points). In each subgroup, a layer may not include an IDCM node, or may include one or more IDCM nodes.

[0593] In the present disclosure, it is assumed that the layer group structure is applied equally to geometry and attributes. This means that the geometry tree structure for geometry decoding is aligned with the attribute tree structure for attribute decoding. In other words, the geometry codec layer structure (i.e., layer group structure) for geometry decoding is the same as the attribute codec layer structure (i.e., layer group structure) for attribute decoding. That is, the layer group / subgroup structure in the geometry tree is the same as the layer group / subgroup structure in the attribute tree. This may imply that when an IDCM node is included in a specific geometry layer in the geometry tree, the same attribute layer in the attribute tree may also include an IDCM node.

[0594] As an example, refer to Figure 22 , assuming that the geometry tree structure includes eight geometry layers 0 to 7 and three layer groups, where layer group 2 is divided into two subgroups, layer group 3 is divided into four subgroups, and layer group 1, the two subgroups divided from layer group 2, and the four subgroups divided from layer group 3 all constitute geometry slices. In this case, the attribute tree structure also includes eight attribute layers 0 to 7 and three layer groups, where layer group 2 is divided into two subgroups, and layer group 3 is divided into four subgroups. In addition, layer group 1, the two subgroups divided from layer group 2, and the four subgroups divided from layer group 3 all constitute attribute slices.

[0595] exist Figure 22 In

[0045] , an undivided layer group, for example, layer group 1, may be referred to as a single subgroup. In addition, a geometry layer may be referred to as a geometry codec layer, and an attribute layer may also be referred to as an attribute codec layer.

[0596] Even when scalable decoding is performed, geometry decoding is first performed on the selected slice. Therefore, by referring to signaling information, the layer group and subgroup to which the node to be attribute-decoded belongs can be identified, and whether the node is a non-IDCM node or an IDCM node can also be identified. In this case, the layer group and subgroup to which the node to be attribute-decoded belongs, and whether the node is a non-IDCM node or an IDCM node can be identified based on the decoded geometry and / or signaling information.

[0597] In one embodiment, when nodes are stored in the input buffer 61006 for LoD generation according to the above features, the attribute decoder 61004 may assign an index to each node. In the following description, the index may be referred to as a pointer. In one embodiment, the index assigned to each node may be a value that identifies the layer group and subgroup to which the node belongs, and also identifies whether the node is a non-IDCM node or an IDCM node.

[0598] In other words, since geometry decoding has been performed and each subgroup and the non-IDCM nodes and IDCM nodes included in each subgroup have been decoded during the geometry decoding process, indexes can be assigned to the nodes stored in the input buffer 61006 with reference to the geometry decoding order. This is merely an embodiment, and any indexes can be assigned to the nodes as long as they have values that can identify whether the nodes stored in the input buffer belong to the current subgroup or the parent subgroup, or whether the nodes are non-IDCM nodes or IDCM nodes.

[0599] In one embodiment of the present disclosure, subgroup-based LoD generation and NN search may be performed based on the index of each node stored in the input buffer 61006 .

[0600] By performing LoD generation and NN search based on the value of the index of each node stored in the input buffer 61006, the speed of attribute decoding can be improved because the processes of determining the position of each node, determining the layer group / subgroup to which each node belongs based on the determined position, and determining whether the node is a non-IDCM node or an IDCM node can be skipped.

[0601] In one embodiment, when IDCM nodes (also called direct compression nodes) exist in the current subgroup and / or the parent subgroup, the IDCM nodes belonging to the current subgroup and the parent subgroup may be additionally input into the partial LoD generator for the current subgroup.

[0602] The IDCM nodes are sequentially input to the partial LoD generator for the current subgroup in the order of their geometry layers. Figure 37 As shown in , nodes belonging to the current subgroup or parent subgroup can be stored in the input buffer 61006 in sequence.

[0603] Figure 37 Illustrated is an example of nodes of a current child group and a parent child group stored in an input buffer according to an embodiment.

[0604] According to an embodiment, the input buffer 61006 may include a storage area for the current subgroup and a storage area for the parent subgroup of the current subgroup. The storage area for the current subgroup may be divided into a non-IDCM node storage area and an IDCM node storage area. Similarly, the storage area for the parent subgroup may be divided into a non-IDCM node storage area and an IDCM node storage area.

[0605] The non-IDCM nodes stored in the non-IDCM node storage area and the IDCM nodes stored in the IDCM node storage area for the parent child group do not overlap (or do not repeat) with the nodes belonging to the current child group.

[0606] In one embodiment of the present disclosure, the non-IDCM node storage area for the current subgroup may sequentially store the non-IDCM nodes from the current subgroup, and the non-IDCM node storage area for the parent subgroup may sequentially store the non-IDCM nodes from the parent subgroup. In one embodiment, the non-IDCM nodes stored in the non-IDCM node storage area for the current subgroup and the non-IDCM nodes in the non-IDCM node storage area for the parent subgroup are distinguished by an index assigned to each non-IDCM node.

[0607] In one embodiment, the IDCM node storage area for the current subgroup may be divided by layers and store the IDCM node of the current subgroup, and the IDCM node storage area for the parent subgroup may be divided by layers and store the IDCM node of the parent subgroup.

[0608] For example, when the current subgroup consists of two layers, the IDCM node storage area for the current subgroup in the input buffer 61006 is further divided into a first area and a second area. The first area sequentially stores the IDCM nodes from the top layer of the current subgroup, and the second area sequentially stores the IDCM nodes from the bottom layer of the current subgroup. When the current subgroup consists of multiple layers, the IDCM node storage area can be divided into multiple areas. In this case, in one embodiment, the IDCM nodes in each area can be identified by the index assigned to each IDCM node.

[0609] As another example, when the parent-child group includes two layers, the IDCM node storage area for the parent-child group in the input buffer 61006 is further divided into a first area and a second area. The first area can store the IDCM nodes from the top layer of the parent-child group, and the second area can store the IDCM nodes from the bottom layer of the parent-child group. When the parent-child group includes multiple layers, the IDCM node storage area can be divided into multiple areas. In this case, in one embodiment, the IDCM nodes in each area can be identified by the index assigned to each IDCM node.

[0610] In one embodiment, the index assigned to each node can be used to identify whether each node stored in the input buffer 61006 belongs to the current subgroup or the parent subgroup, and whether it is a non-IDCM node or an IDCM node. Based on the index assigned to each node, each node can be identified, and LoD generation and NN search can be performed. Therefore, the speed of attribute decoding can be improved.

[0611] In the present disclosure, the input buffer 61006 is configured in the form of an array, and the index of each node stored in the input buffer 61006 can be referred to as a pointer.

[0612] The code given below shows an example for storing the IDCM nodes belonging to the current subgroup in the input buffer 61006. In the following code, tempPointCloudIdx can be used as a pointer to the IDCM nodes belonging to the current subgroup. Figure 37 61006 . This is a pointer (or index) to the node of each block (or region) illustrated in . It can be used the same number of times as the number of nodes (or points) belonging to the input buffer 61006 at each boundary. That is, in the following code, tempPointCloud is the data about the current subgroup stored in the input buffer 61006 and is then input to the partial LoD generator. tempPointCloud can be identified by tempPointCloudIdx (i.e., index).

[0613] / / 1-2. Idcm node: current

[0614] int numLayers = _layerGroupParams.numLayersPerLayerGroup[curLayerGroup];

[0615] int accLayer_prev = accLayer[curLayerGroup] - numLayers;

[0616] for (int i = 0; i < numLayers; i++) {

[0617] if (curLayerGroup == _layerGroupParams.numLayerGroupsMinus1 && i ==numLayers - 1) {

[0618] }

[0619] else {

[0620] auto dcmNodesIdx = _layerGroupParams.dcmNodesIdx[accLayer_prev + i][curSubgroup];

[0621] int pointCount_add = dcmNodesIdx.size();

[0622] if (pointCount_add) {

[0623] int pointCount = tempPointCloud.getPointCount();

[0624] tempPointCloud.resize(pointCount + pointCount_add);

[0625] for (int n = 0; n < pointCount_add; n++) {

[0626] int pointIdx = dcmNodesIdx[n];

[0627] tempPointCloud[pointCount + n] = _currentPointCloud[pointIdx];

[0628] tempIdxToPointIdx.push_back(pointIdx);

[0629] }

[0630] }

[0631] }

[0632] tempPointCloudIdx.push_back(tempPointCloud.getPointCount());

[0633] }

[0634] The code given below shows an example of storing the IDCM nodes belonging to the parent-child group in the input buffer 61006. In one embodiment, because attribute decoding has been performed on the IDCM nodes belonging to the parent-child group, the IDCM nodes can be stored in the input buffer 61006 together with their attribute components (e.g., reflectance and / or color). In one embodiment, in the case where the IDCM node in the bottom layer of the parent-child group cannot be obtained from the child-child group through subsampling, the IDCM node can be stored separately in the input buffer 61006. In other words, in the case where the IDCM node in the bottom layer belonging to the parent-child group does not have a child node, it may not be possible to obtain the IDCM node from a lower LoD through subsampling (e.g., Figure 36 66012 in ). In this case, the IDCM node is stored separately in the input buffer 61006 along with its index. In the following code, tempPointCloud is the data about the parent-child group stored in the input buffer 61006 and is then input to the partial LoD generator. tempPointCloud can be identified by tempPointCloudIdx (i.e., index).

[0635] / / 2-2. IDCM nodes in parent-child groups

[0636] / / Find the parent-child group list

[0637] std::vector <int>subgroupList;

[0638] subgroupList.resize(curLayerGroup);

[0639] int childSubgroup = curSubgroup;

[0640] int childLayerGroup = curLayerGroup;

[0641] for (int prtLayerGroup = curLayerGroup - 1; prtLayerGroup >= 0;prtLayerGroup--) {

[0642] int prtSubgroup = _layerGroupParams.parentSubgroupId[childLayerGroup][childSubgroup];

[0643] subgroupList[prtLayerGroup] = prtSubgroup;

[0644] childLayerGroup = prtLayerGroup;

[0645] childSubgroup = prtSubgroup;

[0646] }

[0647] for (int prtLayerGroup = 0; prtLayerGroup < curLayerGroup; prtLayerGroup++) {

[0648] int prtSubgroup = subgroupList[prtLayerGroup];

[0649] / Compare the parent subgroup bounding box (bbox_parent) with the current subgroup bounding box (bbox). If they are the same size, skip the following process (for speed).

[0650] auto bbox_min_parent = _layerGroupParams.subgrpBboxOrigin[prtLayerGroup][prtSubgroup];

[0651] auto bbox_max_parent = bbox_min_parent + _layerGroupParams.subgrpBboxSize[prtLayerGroup][prtSubgroup];

[0652] bool sameBboxFlag = false;

[0653] int numLayers = _layerGroupParams.numLayersPerLayerGroup[prtLayerGroup];

[0654] int accLayer_prev = accLayer[prtLayerGroup] - numLayers;

[0655] for (int i = 0; i < numLayers; i++) {

[0656] if(i<numLayers - 1 && !sameBboxFlag) {

[0657] int pointCount = tempPointCloud.getPointCount();

[0658] int pointCount_add = dcmNodesIdx.size();

[0659] if (pointCount_add) {

[0660] tempPointCloud.resize(pointCount + pointCount_add);

[0661] int count = 0;

[0662] for (int n = 0; n < pointCount_add; ++)

[0663] int pointIdx = dcmNodesIdx[n];

[0664] auto pos = _currentPointCloud[pointIdx];

[0665] int tempIdx = pointCount + count++;

[0666] if (attr_sps.attr_num_dimensions_minus1 == 0)

[0667] tempPointCloud.setReflectance(tempIdx, _currentPointCloud.getReflectance(pointIdx));

[0668] else if (attr_sps.attr_num_dimensions_minus1 == 2)

[0669] tempPointCloud.setColor(tempIdx, _currentPointCloud.getColor(pointIdx));

[0670] tempIdxToPointIdx.push_back(-1);

[0671] }

[0672] tempPointCloud.resize(pointCount + count);

[0673] }

[0674] }

[0675] else if (i == numLayers - 1) {

[0676] The bottom-level IDCM nodes of the parent-child group cannot be obtained by subsampling, so they should be stored separately in the input buffer.

[0677] auto dcmNodesIdx = _layerGroupParams.dcmNodesIdx[accLayer_prev + i][prtSubgroup];

[0678] int pointCount = tempPointCloud.getPointCount();

[0679] int pointCount_add = dcmNodesIdx.size();

[0680] std::cout << "layerIdx = " << i << "\tpointCount_add = " <<pointCount_add;

[0681] if (pointCount_add) {

[0682] tempPointCloud.resize(pointCount + pointCount_add);

[0683] int count = 0;

[0684] for (int n = 0; n < pointCount_add; n++) {

[0685] int pointIdx = dcmNodesIdx[n];

[0686] auto pos = _currentPointCloud[pointIdx];

[0687] int tempIdx = pointCount + count++;

[0688] tempPointCloud[tempIdx] = _currentPointCloud[pointIdx];

[0689] if (attr_sps.attr_num_dimensions_minus1 == 0)

[0690] tempPointCloud.setReflectance(tempIdx, _currentPointCloud.getReflectance(pointIdx));

[0691] else if (attr_sps.attr_num_dimensions_minus1 == 2)

[0692] tempPointCloud.setColor(tempIdx, _currentPointCloud.getColor(pointIdx));

[0693] tempIdxToPointIdx.push_back(-1);

[0694] }

[0695] tempPointCloud.resize(pointCount + count);

[0696] }

[0697] }

[0698] tempPointCloudIdx.push_back(tempPointCloud.getPointCount());}

[0699] }

[0700] As described above, once the input node for LoD generation in the current subgroup is selected (or determined) in operation 63015 (or by the input node selector), the partial LoD generator for the current subgroup will generate LoD in the current subgroup based on the selected node (operation 63016).

[0701] That is, the input node selector reads the non-IDCM nodes / IDCM nodes of the current subgroup stored in the current subgroup storage area in input buffer 61006 and the non-IDCM nodes / IDCM nodes of the parent subgroup stored in the parent subgroup storage area based on the indexes assigned to the respective nodes, and inputs them to the partial LoD generator for LoD generation in the current subgroup. In one embodiment, to avoid using nodes outside the subgroup boundary, the neighbor search (i.e., NN search) is limited to nodes in the same subgroup or parent subgroup.

[0702] In the present disclosure, partial LoD generation is used when the starting and ending LoDs are changed to the bottom LoD layer of the current subgroup and the LoD layer above the top LoD layer of the current subgroup, respectively. That is, the input for partial LoD generation for the current subgroup includes nodes belonging to the current subgroup and nodes belonging to the parent subgroup (current node + parent node). In one embodiment, among the nodes belonging to the parent subgroup (parent node or parent point), the node selected for the current subgroup may be a node outside the bounding box of the current subgroup and / or an IDCM node. In other words, the node of the parent subgroup selected for LoD generation in the current subgroup is a node not included in the scope of the current subgroup. Here, the term "not included in the scope of the current subgroup" means located outside the boundaries of the current subgroup. For example, assuming that the parent subgroup includes a first subgroup and a second subgroup, and the first subgroup is the current subgroup, the second subgroup corresponds to a subgroup not included in the scope of the current subgroup or located outside the boundaries of the current subgroup.

[0703] In other words, the input to the partial LoD generator for the current subgroup includes nodes belonging to the current subgroup and nodes belonging to the parent subgroup, wherein nodes belonging to the parent subgroup that are duplicates of nodes belonging to the current subgroup are not selected.

[0704] Furthermore, when the input to the partial LoD generator for the current subgroup includes nodes belonging to the current subgroup and nodes belonging to the parent subgroup, the LoD generation output may include the parent subgroup's bottom LoD and the current subgroup's LoD (parent bottom LoD + current LoD).

[0705] According to an embodiment, for each LoD, a partial LoD generator may include an IDCM node generated in the geometry layer corresponding to each LoD, so that the attribute layer structure has a one-to-one correspondence with the geometry layer structure. To this end, the subsampling used in the existing LoD generation can be used to identify (or select) non-IDCM nodes belonging to each LoD, and the IDCM nodes can be added to the corresponding LoD.

[0706] To this end, it is necessary to identify (or determine) whether the nodes included in the input to the partial LoD generator are IDCM nodes or non-IDCM nodes. To this end, the input node selector can divide the nodes of the current subgroup / parent subgroup stored in the input buffer 61006 into non-IDCM nodes and IDCM nodes based on the index of each node, and deliver them to the partial LoD generator, as shown in the code below. Therefore, the LoD generation in the current subgroup can be performed efficiently. In the code below, tempPointCloud is the input data for the partial LoD generator, and the input node selector can identify (or select) the non-IDCM node based on the index (or pointer) of tempPointCloud (i.e., tempPointCloudIdx) to configure the subsampling input.

[0707] for (uint32_t i = 0; i < pointCount; ++i) {

[0708] int pointIdx = packedVoxel[i].index;

[0709] if (pointIdx < tempPointCloudIdx[curNonidcmArrayIdx] / / current non-idcm nodes

[0710] || (sliceLayerGroup > 0

[0711] && pointIdx >= tempPointCloudIdx[curIdcmArrayIdx] && pointIdx <tempPointCloudIdx[parentNonidcmArrayIdx])

[0712] || (sliceLayerGroup < layerGroupParams.numLayerGroupsMinus1 &&numIdcmNodesCurrentSubgroup

[0713] && pointIdx >= tempPointCloudIdx[curIdcmArrayIdx - 1] && pointIdx <tempPointCloudIdx[curIdcmArrayIdx]))

[0714] input[idx++] = i;

[0715] Else

[0716] dcmIdx++;

[0717] }

[0718] The top-level LoD of the partial LoD generator is a subset of the nodes in the bottom level of the parent-child group in the input point cloud data. Therefore, when there are no IDCM nodes, just include all the nodes that are retained. On the other hand, when there are IDCM nodes, the IDCM nodes belonging to the parent-child group are used as input to the partial LoD generator. In this case, the IDCM nodes of the parent-child group can be identified and added based on the index of tempPointCloud (i.e., tempPointCloudIdx), as shown in the code below.

[0719] auto numInput = input.size();

[0720] for (int i = 0; i < numInput; i++) {

[0721] auto pointIndex = packedVoxel[input[i]].index;

[0722] indexes.push_back(pointIndex);

[0723] predictors[--predIndex].neighborCount = 0;

[0724] pointIndexToPredictorIndex[pointIndex] = predIndex;

[0725] }

[0726] if (numIdcmNodesParentSubgroup) {

[0727] int parentIdcmIdx_start = tempPointCloudIdx[parentNonidcmArrayIdx];

[0728] int parentIdcmIdx_end = tempPointCloudIdx[parentIdcmArrayIdx];

[0729] for (int pointIndex = parentIdcmIdx_start; pointIndex <parentIdcmIdx_end; pointIndex++) {

[0730] indexes.push_back(pointIndex);

[0731] predictors[--predIndex].neighborCount = 0;

[0732] pointIndexToPredictorIndex[pointIndex] = predIndex;

[0733] }

[0734] }

[0735] That is, through the preprocessing performed in operation 63015 (or through the input node selector), the input to the subgroup LoD generator (or subgroup LoD generation input) includes both the nodes of the current subgroup and the nodes of the parent subgroup outside the current subgroup boundary. In addition, the IDCM node of the parent subgroup is also included. In addition, the nodes newly included in each LoD through LoD generation are included in the index.

[0736] According to an embodiment, the partial LoD generator for the current subgroup generates one or more LoDs in the current subgroup based on the non-IDCM nodes / IDCM nodes of the current subgroup read from the input buffer 61006 based on the index of each node by the input node selector, and the nodes / IDCM nodes of the parent subgroup outside the boundary of the current subgroup. In one embodiment, the nodes of the parent subgroup outside the boundary of the current subgroup include nodes and / or IDCM nodes subsampled from sibling subgroups of the current subgroup.

[0737] According to the present disclosure, once LoD is generated for each subgroup, a neighbor search (or NN search) is performed on each subgroup based on the generated LoD.

[0738] Before performing a neighbor search based on the generated LoD, only nodes within the subgroup boundary are selected for neighbor search (ie, NN search), as shown in the code below. In one embodiment, in addition to the point position resolution, the geometric resolution of the current subgroup can also be considered.

[0739] The following code includes code for finding the starting position of a neighbor search (ie, NN search) when LoDs are generated and sorted based on subgroups.

[0740] Specifically, when the position of the current node to be decoded is greater than or equal to the minimum value of the bounding box and less than the maximum value of the bounding box (pos >= bbox_min && pos < bbox_max), the NN search starts from the current bounding box.

[0741] for (int i = startIndex; i < endIndex; i++) {

[0742] int packedVoxelIdx = indexes[i];

[0743] int pointIndex = packedVoxel[packedVoxelIdx].index;

[0744] auto pos = pointCloud[pointIndex];

[0745] if (pos >= bbox_min && pos < bbox_max)

[0746] indexes_subgroup.push_back(packedVoxelIdx);

[0747] }

[0748] for (int k = 0; k < indexes_subgroup.size(); k++) {

[0749] auto point = pointCloud[packedVoxel[indexes_subgroup[k]].index];

[0750] biasedPos_indexes.push_back(times((point >> shiftLayerGroup) <<(shiftLayerGroup + treeLvlGap), aps.lodNeighBias));

[0751] }

[0752] During the subsampling operation, nodes present in higher LoDs are included in the retained nodes. In one embodiment, only nodes within the boundaries of the child group to which the parent LoD belongs can be selectively used for NN search (or neighbor search), as shown in the code below. Alternatively, for all parent LoDs, whether to use a node for NN search can be determined based on the bounding box of the parent-child group.

[0753] for (int i = 0; i < retained.size(); i++) {

[0754] int packedVoxelIdx = retained[i];

[0755] int pointIdx = packedVoxel[packedVoxelIdx].index;

[0756] auto pos = pointCloud[pointIdx];

[0757] if (pos >= parent_bbox_min && pos < parent_bbox_max)

[0758] retained_subgroup.push_back(packedVoxelIdx);

[0759] }

[0760] Additionally, in the neighbor search, adjacent nodes can be found among nodes belonging to the upper LoD for the current LoD. Because IDCM nodes are included in the LoD, it is necessary to include the IDCM nodes in the upper LoD. In this case, the partial LoD generator can include the IDCM nodes of the upper layer of the geometry layer matching the current LoD in the neighbor search (i.e., neighbor candidates), as shown in the code below.

[0761] if (idcmNodesPresentFlag) {

[0762] if(currentGeomLayerIndex > 0) {

[0763] if (sliceLayerGroup > 0 && numLayersParentSubgroups) {

[0764] int startIdx = tempPointCloudIdx[parentNonidcmArrayIdx];

[0765] int endIdx = tempPointCloudIdx[parentNonidcmArrayIdx +numLayersParentSubgroups];

[0766] for (int pointIdx = startIdx; pointIdx < endIdx; pointIdx++) {

[0767] auto packedVoxelIndex = pointIdxToPackedVoxelIdx[pointIdx];

[0768] }

[0769] }

[0770] if (currentGeomLayerIndex - numLayersParentSubgroups > 0 &&numLayersCurrentSubgroup) {

[0771] int endIdx = tempPointCloudIdx[curNonidcmArrayIdx +currentGeomLayerIndex - numLayersParentSubgroups];

[0772] for (int pointIdx = startIdx; pointIdx < endIdx; pointIdx++) {

[0773] auto packedVoxelIndex = pointIdxToPackedVoxelIdx[pointIdx];dcmNodeList_ParentLayers.push_back(packedVoxelIndex);

[0774] }

[0775] }

[0776] }

[0777] }

[0778] for (int i = 0; i < retained.size(); i++) {

[0779] int packedVoxelIdx = retained[i];

[0780] int pointIdx = packedVoxel[packedVoxelIdx].index;

[0781] auto pos = pointCloud[pointIdx];

[0782] if (pos.x() >= parent_bbox_min.x() && pos.x() < parent_bbox_max.x()

[0783] && pos.y() >= parent_bbox_min.y() && pos.y() < parent_bbox_max.y()

[0784] && pos.z() >= parent_bbox_min.z() && pos.z() < parent_bbox_max.z())

[0785] retained_subgroup.push_back(packedVoxelIdx);

[0786] if (dcmNodeList_ParentLayers.size()) {

[0787] for (int j = 0; j < dcmNodeList_ParentLayers.size(); j++)

[0788] retained_subgroup.push_back(dcmNodeList_ParentLayers[j]);

[0789] std::sort(retained_subgroup.begin(), retained_subgroup.end());

[0790] }

[0791] Additionally, as input for the next LoD subsampling, the IDCM node of the geometry layer corresponding to the next LoD can be added along with the node delivered to the current LoD from the previous LoD subsampling, as shown in the code below. Even in this case, the IDCM node can be identified based on the index (or pointer) of tempPointCloud (i.e., tempPointCloudIdx) without any additional processing. In other words, in this case, the IDCM node can be identified based on the index assigned to each node, and the IDCM node in the geometry layer corresponding to the next LoD can be added.

[0792] if (idcmNodesPresentFlag) {

[0793] / The IDCM nodes in the immediately above layer are retained (to be used as input for the next round)

[0794] if(parentGeomLayerIndex >= numLayersParentSubgroups) { auto prev =retained.size();

[0795] / The starting point of the current layer should be subtracted.

[0796] int startArrayIdx = curNonidcmArrayIdx + (parentGeomLayerIndex -numLayersParentSubgroups);

[0797] int endArrayIdx = curNonidcmArrayIdx + (currentGeomLayerIndex -numLayersParentSubgroups);

[0798] for (int pointIndex = tempPointCloudIdx[startArrayIdx]; pointIndex <tempPointCloudIdx[endArrayIdx]; pointIndex++) {

[0799] auto packedVoxelIndex = pointIdxToPackedVoxelIdx[pointIndex];

[0800] retained.push_back(packedVoxelIndex); / input[n-1] = retained[n] +idcm[n-1]

[0801] }

[0802] std::sort(retained.begin(), retained.end());

[0803] }

[0804] }

[0805] According to an embodiment, for geometric accuracy, the geometric accuracy of the parent child group can be used for nodes outside the bounding box of the current child group, as shown in the following code. Figure 36 As an example, the underlined portion of the code below corresponds to nodes h and o, and the bold portion of the code corresponds to node d.

[0806] for (int k = 0; k < retained_subgroup.size(); k++) {

[0807] auto point = pointCloud[packedVoxel[retained_subgroup[k]].index];

[0808] if (prtLayerGroup != curLayerGroup) {

[0809] if (pos >= cur_bbox_min && pos.x() < cur_bbox_max)

[0810] biasedPos_retained.push_back(times((point >> shiftLayerGroup) <<(shiftLayerGroup + treeLvlGap), aps.lodNeighBias));

[0811] else

[0812] biasedPos_retained.push_back(times((point >> shiftPrtLayerGroup) <<(shiftPrtLayerGroup + treeLvlGap), aps.lodNeighBias));

[0813] }

[0814] else

[0815] biasedPos_retained.push_back(times((point >> shiftLayerGroup) <<(shiftLayerGroup + treeLvlGap), aps.lodNeighBias));

[0816] }

[0817] Next, once LoD generation and NN search are performed based on the subgroup in operation 63016 (or LoD generation) as described above, at least one node in the current subgroup may inherit the attributes of at least one point in the parent subgroup (operation 63017). In the present disclosure, the operation of at least one node in the current subgroup inheriting the attributes of at least one node in the parent subgroup in operation 63017 is referred to herein as post-processing. Furthermore, in one embodiment, operation 63017 may be performed by an attribute inheritance component (not shown). This attribute inheritance component may be referred to as a post-processor.

[0818] According to an embodiment, due to the definition of LoD—a lower LoD should include all nodes of a higher LoD—nodes belonging to the bottom LoD of a parent subgroup are included in the top LoD of a child subgroup. In other words, some nodes belonging to the top LoD of a child subgroup match nodes in the bottom LoD of the parent subgroup. In this case, the attributes of the nodes belonging to the top LoD of the child subgroup can inherit the attributes of the corresponding nodes belonging to the bottom LoD of the parent subgroup.

[0819] In other words, due to subsampling during LoD generation, some nodes in the top LoD of the current subgroup match nodes in the bottom LoD of the parent subgroup. To provide links between LoD layers at layer group boundaries, the properties of the decoded nodes of the parent subgroup are inherited by the child nodes in the top LoD layer of the current subgroup. The matching is performed taking into account the sampling direction in LoDn, as Figure 38 As shown in the figure.

[0820] Figure 38 1 illustrates an example of a property inheritance process according to an embodiment. For simplicity, in this disclosure, 65011 corresponds to Figure 35 and is referred to herein as the parent-child group. 65012 corresponds to Figure 35 and is referred to herein as the first sub-subgroup or first subgroup, and 65013 corresponds to Figure 35 65011 corresponds to the parent subgroup of the first subgroup 65012 and the second subgroup 65013. In one embodiment of the present disclosure, there is no dependency between the first subgroup 65012 and the second subgroup 65013. In other words, it is assumed that there is no dependency between the sibling subgroups.

[0821] refer to Figure 38 As an example, among nodes d, h, and o belonging to the bottom layer (or bottom LoD) of the parent subgroup, nodes h and o match nodes f, h, i, and o belonging to the top layer (or top LoD) of the current subgroup 65013 (e.g., subgroup C). In this case, based on the subsampling relationship, the decoded attributes of nodes h and o are matched, as indicated by arrows 67001 and 67002. In other words, based on the current subgroup 65013, the attributes for the parent subgroup 65011 have already been decoded. Therefore, the attributes of nodes h and o in the current subgroup 65013 match (or are inherited from) the decoded attributes of nodes h and o in the parent subgroup 65011, and attribute decoding is skipped for nodes h and o in the current subgroup 65013.

[0822] To do this, as shown in the code below, for nodes that fall within the bounding box of the current subgroup among the nodes belonging to the bottom layer (or bottom LoD) of the parent subgroup, the properties of the parent subgroup can be referenced to find the properties of the corresponding nodes in the current subgroup for nodes whose positions match those of nodes in the top layer (or top LoD) of the parent subgroup.

[0823] if (curLayerGroup > 0) {

[0824] int parentLayerGroup = curLayerGroup - 1;

[0825] int parentSubgroup = layerGroupParams.parentSubgroupId[curLayerGroup][curSubgroup];

[0826] int accLayer = 0;

[0827] for (int i = 0; i <= parentLayerGroup; i++)

[0828] accLayer += layerGroupParams.numLayersPerLayerGroup[i];

[0829] int treeLvlGap_parent = sps.root_node_size_log2.max() -accLayer;

[0830] auto bbox_min = layerGroupParams.subgrpBboxOrigin[curLayerGroup][curSubgroup];

[0831] auto bbox_max = bbox_min + layerGroupParams.subgrpBboxSize[curLayerGroup][curSubgroup];

[0832] std::vector <int>pointIndexInTheBbox;

[0833] auto pointCloud_subgroup_fifo_parent = layerGroupParams.pointCloud_tempNodes[parentLayerGroup][parentSubgroup];

[0834] int currentSubgroupIdxEnd = tempPointCloudIdx[layerGroupParams.numLayersPerLayerGroup[curLayerGroup]];

[0835] for (int i = 0; i < _lods.numPointsInLod[0]; i++) {

[0836] int pointIdx = _lods.indexes[i];

[0837] if (pointIdx < currentSubgroupIdxEnd)

[0838] pointIndexInTheBbox.push_back(pointIdx);

[0839] }

[0840] int numParentNodesInTheBbox = pointIndexInTheBbox.size();

[0841] int sampledNodeIdx = numParentNodesInTheBbox - 1;

[0842] for (int fifoIdx = 0; fifoIdx < pointCloud_subgroup_fifo_parent.getPointCount(); fifoIdx++) {

[0843] pcc::Vec3 <int>pos_parent = pointCloud_subgroup_fifo_parent[fifoIdx]<< treeLvlGap_parent;

[0844] if (pos_parent >= bbox_min && pos_parent < bbox_max)

[0845] int pointIdx = pointIndexInTheBbox[sampledNodeIdx];

[0846] auto pos_lod0 = (pointCloud[pointIdx] >> treeLvlGap_parent) <<treeLvlGap_parent;

[0847] if (mortonAddr(pos_lod0) == mortonAddr(pos_parent)) {

[0848] if (attr_desc.attr_num_dimensions_minus1 == 0)

[0849] pointCloud.setReflectance(pointIdx, pointCloud_subgroup_fifo_parent.getReflectance(fifoIdx));

[0850] else if (attr_desc.attr_num_dimensions_minus1 == 2)

[0851] pointCloud.setColor(pointIdx, pointCloud_subgroup_fifo_parent.getColor(fifoIdx));

[0852] if (!sampledNodeIdx)

[0853] break;

[0854] else

[0855] sampledNodeIdx--;

[0856] }

[0857] else {

[0858] fifoIdx--;

[0859] sampledNodeIdx--;

[0860] }

[0861] The attributes found through the above process can be used as neighbor references during subsequent attribute transformation operations.

[0862] For example, attribute transformation is performed on nodes f to q in the current child group 65013 except for nodes h and o that have inherited attributes from the parent child group 65011.

[0863] According to an embodiment, after at least one node in the current subgroup inherits the attributes of at least one node in the parent subgroup (i.e., the node at the same position) in operation 63017 (or attribute inheritance), quantization weights may be derived based on the subgroups (operation 63018), and attribute transformation (e.g., inverse prediction / lifting transformation) may be performed on the remaining nodes in the current subgroup (operation 63019). In other words, attribute transformation, i.e., attribute decoding, is performed on a subgroup-by-subgroup basis to reconstruct the attributes of the remaining nodes in the current subgroup.

[0864] The decoded attributes are then stored on a sub-group by sub-group basis (operation 63020).

[0865] That is, after completing the attribute decoding of the current subgroup, the attribute decoder 61004 can store the IDCM nodes and the non-IDCM nodes separately. At this time, without a separate identification operation, the non-IDCM nodes belonging to the current subgroup can be selected based on the index of tempPointCloud, as shown in the following code.

[0866] if (curLayerGroup < _layerGroupParams.numLayerGroupsMinus1) { / intermediate layer-groups

[0867] auto& pointCloud_subgroup_fifo = _pointCloud_tempNodes[curLayerGroup][curSubgroup];

[0868] for (int i = 0; i < pointCloud_subgroup_fifo.getPointCount(); i++) {

[0869] if (attr_sps.attr_num_dimensions_minus1 == 0)

[0870] pointCloud_subgroup_fifo.setReflectance(i,tempPointCloud.getReflectance(i));

[0871] else if (attr_sps.attr_num_dimensions_minus1 == 2)

[0872] pointCloud_subgroup_fifo.setColor(i, tempPointCloud.getColor(i));

[0873] }

[0874] }

[0875] In addition, when configuring tempPointCloud, you can pre-store information indicating whether the IDCM node belongs to the current subgroup. In the following code, when the value of tempIdxToPointIdx is negative, it can indicate that the node is not the IDCM node of the current subgroup. When tempIdxToPointIdx is greater than 0, it can indicate the index of the IDCM node in the output point cloud data.

[0876] int pointCount = tempPointCloud.getPointCount();

[0877] for (int i = 0; i < pointCount; i++) {

[0878] int pointIdx = tempIdxToPointIdx[i];

[0879] if (pointIdx >= 0) {

[0880] if (attr_sps.attr_num_dimensions_minus1 == 0)

[0881] _currentPointCloud.setReflectance(pointIdx,tempPointCloud.getReflectance(i));

[0882] else if (attr_sps.attr_num_dimensions_minus1 == 2)

[0883] _currentPointCloud.setColor(pointIdx, tempPointCloud.getColor(i));

[0884] }

[0885] }

[0886] According to an embodiment, Figure 34 In

[15] , geometry slices and corresponding attribute slices may be delivered alternately. In this case, if there are remaining geometry slices, the geometry decoder is restarted, and then the corresponding attribute decoder is operated. If the attribute slice is delivered after the geometry slice, all attribute slices may be decoded after all geometry slices are decoded.

[0887] Through the above process, the output point cloud is generated, and the geometric decoding and attribute decoding are completed.

[0888] If layer group slicing is not enabled, an LoD is generated based on the reconstructed geometry (operation 63012), weights are derived (operation 63013), and then an attribute transformation (e.g., prediction / inverse lifting transformation) is performed (operation 63015). Operations 63012 and 63013 may correspond to LoD generation, NN search, and weight derivation by the attribute encoder 51004 of the transmitting device.

[0889] According to an embodiment, the encoder of the transmitting device and the decoder of the receiving device can perform LoD generation, NN search, and attribute transformation in the same manner or in different manners.

[0890] For example, the encoder of the transmitting device and the decoder of the receiving device can be based on Figures 34 to 38 and respective codes perform LoD generation, NN search and attribute transformation based on the subgroup. In another example, the encoder of the transmitting device can be based on Figure 31 The method illustrated in FIG performs LoD generation, NN search and attribute transformation, and the decoder of the receiving device can perform LoD generation, NN search and attribute transformation according to Figures 34 to 38 and its corresponding codes perform LoD generation, NN search, and attribute transformation based on subgroups.

[0891] Figure 39 FIGURE 1 shows an exemplary bit stream structure for transmitting / receiving point cloud data according to an embodiment. Figure 1 、 Figure 2 、 Figure 3 、 Figure 8 and Figure 30 The bitstream output by any point cloud video encoder can be Figure 39 form.

[0892] According to embodiments, the bitstream of point cloud data provides tiles or slices to allow the point cloud data to be divided into regions to be processed. Each region of the bitstream according to embodiments can have different importance. Therefore, when point cloud data is divided into tiles, different filters (encoding methods) and different filtering units can be applied to each tile. When point cloud data is divided into slices, different filters and different filtering units can be applied to each slice.

[0893] The transmitting device according to the embodiment can be configured as follows: Figure 39 The bitstream structure shown in [1] transmits point cloud data, thereby providing a method for applying different encoding operations according to importance and using a high-quality encoding method for important areas. Furthermore, efficient encoding and transmission based on the characteristics of point cloud data can be supported, and attribute values based on user needs can be provided.

[0894] The receiving device according to the embodiment can Figure 39 The bitstream structure shown in [1] receives point cloud data and, therefore, can apply different filtering methods (decoding methods) to each region (areas divided into tiles or slices) based on the processing capabilities of the receiving device, rather than applying complex decoding (filtering) methods to the entire point cloud data. This ensures better image quality in areas important to the user and maintains appropriate latency on the system.

[0895] When the geometry bitstream, attribute bitstream and / or signaling bitstream (or signaling information) according to the embodiment are constituted as follows Figure 39 When a bitstream (or G-PCC bitstream) is shown, the bitstream may include one or more sub-bitstreams. According to an embodiment, the bitstream may include a sequence parameter set (SPS) signaled at the sequence level, a geometry parameter set (GPS) signaled for geometry information encoding and decoding, one or more attribute parameter sets (APS) signaled for attribute information encoding and decoding (APS0 and APS1), a tile list (also known as a TPS) signaled at the tile level, and one or more slices (Slice 0 to Slice n). That is, the bitstream of point cloud data according to an embodiment may include one or more tiles, each of which may be a slice group including one or more slices (Slice 0 to Slice n). The tile list (i.e., TPS) may include information about each of the one or more tiles (e.g., coordinate value information and height / size information related to the tile bounding box). Each slice may include a geometry bitstream (Geom0) and / or one or more attribute bitstreams (Attr0 and Attr1). For example, slice 0 may include a geometry bitstream (Geom00) and one or more attribute bitstreams (Attr00 and Attr10).

[0896] The geometry bitstream in each slice may include a geometry slice header (geom_slice_header) and geometry slice data (geom_slice_data). Depending on the embodiment, the geometry bitstream in each slice may be referred to as a geometry data unit, and the geometry slice header may be referred to as a geometry data unit header. In addition, the geometry slice data may be referred to as geometry data unit data. Depending on the embodiment, the geometry slice header (or geometry data unit header) may include identification information (geom_parameter_set_id) related to a parameter set included in GPS, a tile identifier (geom_tile_id), a slice identifier (geom_slice_id), and information related to the data included in the geometry slice data (geom_slice_data) (geomBoxOrigin, geom_box_log2_scale, geom_max_node_size_log2, geom_num_points). geomBoxOrigin is geometry box origin information indicating the box origin of geometry slice data, geom_box_log2_scale is information indicating the logarithmic scale of geometry slice data, geom_max_node_size_log2 is information indicating the root geometry octree node size, and geom_num_points is information related to the number of points of geometry slice data. Geometry slice data (or geometry data unit data) may include geometry information (or geometry data) related to point cloud data in the corresponding slice.

[0897] Each attribute bitstream in each slice may include an attribute slice header (attr_slice_header) and attribute slice data (attr_slice_data). According to an embodiment, the attribute bitstream in each slice may be referred to as an attribute data unit. In addition, the attribute slice header may be referred to as an attribute data unit header, and the attribute slice data may be referred to as attribute data unit data. According to an embodiment, the attribute slice header (or attribute data unit header) may include information about the corresponding attribute slice data (or the corresponding attribute data unit), and the attribute slice data may include attribute information (also referred to as attribute data or attribute value) related to the point cloud data in the corresponding slice. When there are multiple attribute bitstreams in a slice, each attribute bitstream may include different attribute information. For example, one attribute bitstream may include attribute information corresponding to color, and another attribute bitstream may include attribute information corresponding to reflectivity.

[0898] According to an embodiment, the parameters required for encoding and / or decoding point cloud data may be defined in the parameter set of the point cloud data (e.g., SPS, GPS, APS, and TPS (also known as tile list)) and / or in the header of the corresponding slice. For example, when encoding and / or decoding geometric information, the parameters may be added to the GPS. When encoding and / or decoding tile-based information, the parameters may be added to the tile and / or slice header.

[0899] According to an embodiment, information related to layer group-based slices (eg, LGSI) may be signaled in the SPS and / or APS.

[0900] According to an embodiment, information related to layer group based slices may be signaled in a TPS and / or attribute data unit header.

[0901] According to an embodiment, when the syntax elements defined below are applicable to multiple point cloud data streams as well as the current point cloud data stream, information related to layer group-based slices may be carried in a higher parameter set.

[0902] According to an embodiment, information related to the layer group-based slice may be defined in a corresponding location or a separate location depending on the application or system to use different application scopes, different application methods, etc. As used in the syntax described herein, a field may have the same meaning as a parameter or a syntax element.

[0903] According to an embodiment, parameters (which may be referred to as metadata, signaling information, etc.) containing information related to slices based on layer groups may be generated by a metadata processor (or metadata generator) or a signaling processor of a transmitting device and delivered to a receiving device for use in decoding / reconstruction operations. For example, the parameters generated and transmitted by the transmitting device may be retrieved by a metadata parser of the receiving device.

[0904] Figure 40 An embodiment of the syntax structure of a sequence parameter set (seq_parameter_set()) (SPS) according to the present disclosure is shown. The SPS may include sequence information related to a point cloud data bitstream. Specifically, it is exemplarily shown as including information related to slices based on layer groups.

[0905] Figure 40 The syntax can be included in Figure 39 and can be generated by a point cloud encoder and decoded by a point cloud decoder according to an embodiment.

[0906] exist Figure 40 In the

[0064] field, the simple_profile_compatibility_flag field may indicate whether the bitstream is compatible with the main profile. For example, simple_profile_compatibility_flag equal to 1 may indicate that the bitstream is compatible with the simple profile. For example, simple_profile_compatibility_flag equal to 0 may indicate that the bitstream is compatible with profiles other than the simple profile.

[0907] When unique_point_positions_constraint_flag is equal to 1, all output points may have unique positions in each point cloud frame referenced by the current SPS. When unique_point_positions_constraint_flag is equal to 0, two or more output points may have the same position in any point cloud frame referenced by the current SPS. For example, even when all points are unique in their respective slices, other points in the slices and frames may overlap. In this case, unique_point_positions_constraint_flag is set to 0.

[0908] level_idc indicates the level that the bitstream complies with.

[0909] sps_seq_parameter_set_id provides an identifier for the SPS for reference by other syntax elements.

[0910] sps_num_attribute_sets indicates the number of attributes coded in the bitstream.

[0911] The SPS according to an embodiment includes an iteration statement that iterates as many times as the value of the sps_num_attribute_sets field. In an embodiment, i is initialized to 0 and incremented by 1 each time the iteration statement is executed. The iteration statement is repeated until the value of i becomes equal to the value of the sps_num_attribute_sets field. The iteration statement may include an attribute_dimension_minus1[i] field and an attribute_instance_id[i] field. Attribute_dimension_minus1[i] plus 1 indicates the number of components of the i-th attribute.

[0912] The attribute_instance_id[i] field specifies the instance ID of the i-th attribute.

[0913] The known_attribute_label_flag[i] field indicates whether the known_attribute_label[i] field or the attribute_label_four_bytes[i] field is signaled for the i-th attribute. For example, when known_attribute_label_flag[i] is 0, it indicates that the known_attribute_label[i] field is signaled for the i-th attribute. When known_attribute_label_flag[i] is 1, it indicates that the attribute_label_four_bytes[i] field is signaled for the i-th attribute.

[0914] known_attribute_label[i] specifies the type of the i-th attribute. For example, a known_attribute_label[i] of 0 specifies that the i-th attribute is color. A known_attribute_label[i] of 1 specifies that the i-th attribute is reflectance. A known_attribute_label[i] of 2 specifies that the i-th attribute is a frame index. Additionally, a known_attribute_label[i] of 4 specifies that the i-th attribute is transparency. A known_attribute_label[i] of 5 specifies that the i-th attribute is normal.

[0915] axis_coding_order indicates the correspondence between the X, Y, and Z output axis labels and the three position components in the reconstructed point cloud RecPic[pointidx][axis], where axis=0..2.

[0916] sps_bypass_stream_enabled_flag equal to 1 may indicate that the bypass codec mode is used in reading the bitstream. As another example, sps_bypass_stream_enabled_flag equal to 0 may indicate that the bypass codec mode is not used in reading the bitstream.

[0917] exist Figure 40 In the SPS syntax structure, sps_extension_flag indicates whether the sps_extension_data syntax structure is present in the SPS syntax structure. For example, sps_extension_present_flag equal to 1 indicates that the sps_extension_data syntax structure is present in the SPS syntax structure. sps_extension_present_flag equal to 0 indicates that this syntax structure is not present.

[0918] The following is an example of information related to layer group-based slices included in sps_extension_data when a value of the sps_extension_present_flag field is 1.

[0919] layer_group_enabled_flag equal to 1 indicates that the geometry (or attribute) bitstream of a slice is contained in multiple slices matching the codec layer group or its subgroup. layer_group_enabled_flag equal to 0 indicates that the geometry (or attribute) bitstream is contained in a single slice.

[0920] When layer_group_enabled_flag is equal to 1, num_layer_groups_minus1 + 1 indicates the number of layer groups, which represent groups of consecutive tree layers that are part of the geometric codec tree structure. The value of num_layer_groups_minus1 ranges from 0 to the number of codec tree layers.

[0921] The loop iterating by the value of num_layer_groups_minus1 may include the layer_group_id[i], num_layers_minus1[i], and subgroup_enabled_flag[i] fields.

[0922] layer_group_id[i] indicates the identifier of the i-th layer group. The value range of layer_group_id is 0 to num_layer_groups_minus1.

[0923] num_layers_minus1[i] + 1 indicates the number of codec layers included in the i-th layer group. The total number of layer groups can be derived by adding all (num_layers_minus1[i] + 1) from 0 to num_layer_groups_minus1 for i.

[0924] subgroup_enabled_flag[i] equal to 1 indicates that the i-th layer group is divided into two or more subgroups. In this case, the set of points in the subgroup is the same as the set of points in the layer group. When subgroup_enabled_flag[i] of the i-th layer group is equal to 1 and j is greater than or equal to i, the subgroup_enabled_flag of the j-th layer group will be equal to 1. subgroup_enabled_flag equal to 0 indicates that the current layer group is contained in a single slice and is not divided into multiple subgroups.

[0925] When layer_group_enabled_flag is equal to 1, the SPS may include subgroup_bbox_origin_bits_minus1 and subgroup_bbox_size_bits_minus1.

[0926] subgroup_bbox_origin_bits_minus1 + 1 indicates the length in bits of the syntax element (ie, field) subgroup_bbox_origin.

[0927] subgroup_bbox_size_bits_minus1 + 1 indicates the length in bits of the syntax element subgroup_bbox_size.

[0928] According to an embodiment, when layer_group_enabled_flag is equal to 1, the SPS may further include root_subgroup_bbox_origin and root_subgroup_bbox_size.

[0929] root_subgroup_bbox_origin may indicate the location of the origin of the subgroup bounding box of the root subgroup.

[0930] root_subgroup_bbox_size may indicate the size of the subgroup bounding box of the root subgroup.

[0931] When sps_extension_flag is equal to 1, the SPS according to an embodiment may further include a sps_extension_data_flag field. The sps_extension_data_flag field may have any value.

[0932] Figure 41 An embodiment of a syntax structure of a dependency attribute data unit header (dependent_attribute_data_unit_header()) according to the present disclosure is illustrated. Figure 41 The diagram illustrates an example in which a dependency attribute data unit header contains information related to layer group-based slices.

[0933] Figure 41 The syntax can be included in Figure 39 and can be generated by a point cloud encoder and decoded by a point cloud decoder according to an embodiment.

[0934] According to an embodiment, the dependency attribute data unit header may be included in the attribute data unit.The term attribute data unit may be used interchangeably with dependency attribute data unit or attribute slice.

[0935] According to an embodiment, the dependency attribute data unit header may be included in at least one of SPS, APS, GPS, or TPS.

[0936] exist Figure 41 In the slice, dadu_attribute_parameter_set_id indicates the active APS as indicated by aps_attr_parameter_set_id. The value of dgdu_attribute_parameter_set_id is equal to the value of adu_attribute_parameter_set_id in the corresponding slice.

[0937] dadu_sps_attr_idx identifies the codec's attribute by its index in the active SPS attribute list.

[0938] dadu_slice_id specifies the attribute slice to which the current dependency attribute data unit belongs.

[0939] dadu_layer_group_id may be an indicator of the layer group of the slice. The value of dadu_layer_group_id ranges from 0 to num_layer_groups_minus1. If not present, it is inferred to be 0.

[0940] dadu_subgroup_id indicates an indicator of the subgroup of the layer group referenced by dadu_layer_group_id. The value range of dadu_subgroup_id is 0 to num_subgroups_minus1[dadu_layer_group_id]. dadu_subgroup_id can indicate the order of slices with the same dadu_layer_group_id. If not present, it is inferred to be 0.

[0941] The receiving method / device according to the embodiment may have the following effects.

[0942] The present disclosure describes a method for partitioning and transmitting compressed data based on specific criteria for point cloud data. Specifically, in one application of the present disclosure, when using layered codecs (or hierarchical codecs), compressed data can be partitioned according to layers for transmission, which can increase storage and transmission efficiency on the sending end. In particular, when using scalable attribute codecs, the entire encoded geometry data should be transmitted, which is a disadvantage. In the present disclosure, only the geometry layer that matches the tree level used in the scalable attribute codec can be delivered, which can further increase transmission efficiency.

[0943] Figure 42 FIG is an example of the geometry and attributes of compressed point cloud data for serving. In other words, in a PCC-based service, the compression rate or the amount of data sent can be adjusted depending on the receiver performance or the transmission environment. Figure 42 In the case of bundling point cloud data on a slice-by-slice basis, receiver performance or transmission environments may change. In this case, it is necessary to either: 1) pre-transcode the bitstream into a form suitable for each environment, store it separately, and select the bitstream at the time of transmission; or 2) perform transcoding before transmission. In this case, when the number of receiver environments to be supported increases or the transmission environment changes frequently, problems may arise related to storage space or latency caused by transcoding.

[0944] Figure 43 is another example illustrating the geometry and attributes of compressed point cloud data for serving according to an embodiment.

[0945] Dividing compressed data by layer for transmission, as proposed in this disclosure, enables selective delivery of only the necessary portions of pre-compressed data at the bitstream stage via a bitstream selector without a separate transcoding operation. This is highly efficient in terms of storage space, as only one storage space is required for each stream. It is also efficient in terms of bandwidth, as the bitstream selector selects only the necessary layers before transmission.

[0946] The effects of the features of the present disclosure are described below from the perspective of the receiving end. In one application of the present disclosure, when layered codecs (or hierarchical codecs) are used, the compressed data can be divided according to layers, which can increase efficiency on the receiving end. Specifically, in the case of scalable attribute codecs, receiving and decoding the geometry data of the entire codec may cause delays and increase the computational burden on the receiving end. According to the present disclosure, only the geometry layers that match the tree levels used in the scalable attribute codec can be decoded, which can reduce the delay factor and increase the efficiency of the decoder by saving the computational power required for decoding.

[0947] Figure 44 This diagram illustrates operations on both the transmitter and receiver sides when transmitting layered point cloud data. In this case, if information for reconstructing the entire point cloud data is delivered regardless of the receiver's performance, the receiver may need to reconstruct the point cloud data through decoding and then select only the point cloud data corresponding to the necessary layers (e.g., data selection or subsampling). In this case, because the delivered bitstream has already been decoded, a low-latency receiver may experience a delay, or decoding may not be possible depending on the receiver's performance.

[0948] However, as proposed herein, when a receiver receives compressed data of only necessary layers, it is possible to allow the receiver to selectively decode specific layers, thereby increasing decoder efficiency and supporting decoders of different performances.

[0949] Figure 45 is a flowchart of a method for transmitting point cloud data according to an embodiment.

[0950] According to an embodiment, a method for transmitting point cloud data may include: acquiring point cloud data (71001), encoding the point cloud data (71002), and transmitting the encoded point cloud data and signaling information (71003). In this case, a bit stream containing the encoded point cloud data and signaling information may be encapsulated in a file and transmitted.

[0951] When acquiring point cloud data (71001), you can execute Figure 1 Part or all of the operations of the point cloud video acquisition unit 10001, or Figure 8 Partial or complete operation of the data input unit 8000.

[0952] When encoding point cloud data (71002), you can execute Figure 1 Part or all of the operations of the point cloud video encoder 10002, Figure 2 Code 20001, Figure 3 Point cloud video encoder, Figure 8 Point cloud video encoder, Figure 25 Attribute encoding, Figure 30 The geometry encoder and attribute encoder and Figure 31 Attribute encoding is used to encode geometric information and attribute information.

[0953] The encoding of the point cloud data (71002) according to an embodiment may include compressing geometric information associated with the input point cloud data and compressing attribute information.

[0954] According to an embodiment, the compression of the geometric information may include compressing the geometric information on a per-layer group and / or per-subgroup basis. In this case, the geometric information may be compressed by distinguishing between IDCM nodes and non-IDCM nodes.

[0955] The compressed geometry information can be entropy coded and output in the form of a geometry bitstream.

[0956] According to an embodiment, compression of the attribute information includes compressing the attribute information based on position and / or reconstructed geometry information for which geometry encoding has not yet been performed. In one embodiment, the attribute information may be encoded and decoded using a combination of one or more of a RAHT codec, a LOD-based predictive transform codec, and a lifting transform codec.

[0957] According to an embodiment, compression of attribute information includes performing a reference Figures 11 to 31 In one embodiment, attribute information compression may include detecting at least one missing node of a parent subgroup using the corresponding geometry subgroup (or geometry FGS) based on the presence of missing subgroups (or FGSs or slices) associated with the attribute information. According to an embodiment, after comparing nodes from a subsample of the current subgroup and an intermediate geometry slice output node, missing nodes that are present in the geometry output but not in the subsample output are appended to the LoD of the current subgroup. In other words, the LoD N is generated using nodes from the subsample of the current attribute's subgroup and / or nodes from the geometry parent subgroup corresponding to the current attribute's parent subgroup. Then, during a NN search for the current point (or node), neighbors may be selected based on distance, which may include neighbor nodes outside the subgroup to which the current point (or node) belongs.

[0958] According to an embodiment, when compressing attribute information, when a neighboring node belongs to a sibling subgroup of a subgroup, that is, a subgroup at the same level, neighboring nodes outside the subgroup to which the current point belongs are not considered neighbors of the current point. In other words, based on the sub-level subgroup boundary, nodes outside the boundary are not selected as neighbors. Therefore, by imposing the restriction that only nodes within the same subgroup boundary box should be used when generating the predictor based on the aforementioned criteria, accurate decoding can be performed even when partially decoding segmented slices. In this case, it is possible to allow reference to the root layer group in all cases.

[0959] According to an embodiment, when compressing attribute information, when a neighbor node outside the subgroup to which the current point belongs belongs to a parent subgroup of the subgroup, the neighbor node is regarded as a neighbor node of the current point.

[0960] According to an embodiment, when performing the NN search during the compression of attribute information, when the neighbor candidate is located within the subgroup bounding box, the attribute encoder 51004 considers the geometric position details down to the finest level (e.g., the bottom level) of the child subgroup. On the other hand, when the neighbor candidate is located outside the subgroup bounding box, the geometric position details are considered down to the finest level (e.g., the bottom level) of the parent subgroup.

[0961] According to an embodiment, when layer group slicing is enabled, the operation of compressing attribute information may include: generating a Location of Descriptors (LoDs) based on subgroups, deriving weights based on the LoDs generated based on the subgroups; and performing an attribute transformation (e.g., a prediction / lifting transformation) on points in the LoDs. Furthermore, in one embodiment, when input points include points belonging to a parent subgroup related to the current subgroup, only points not included in the current subgroup's range may be added. Following this process, a LoD for the current subgroup is generated based on the determined input points, and an NN search is performed based on the generated LoD. In this disclosure, the terms "node" and "point" may be used interchangeably.

[0962] According to an embodiment, as described above, when performing LoD generation and NN search based on a subgroup, an operation may be performed such that at least one point in the current subgroup inherits the attributes of at least one point in the parent subgroup. In other words, some points in the top LoD of a subgroup may match points in the bottom LoD of a parent subgroup. In this case, the attributes of the points in the top LoD of the subgroup may inherit the attributes of the corresponding points in the bottom LoD of the parent subgroup.

[0963] The attribute information compressed through the above process is entropy encoded and output in the form of an attribute bitstream.

[0964] As described herein, signaling information may include slice-related information based on layer groups, for encoding geometry information and attribute information based on layer groups and / or subgroups and decoding them at a receiving end.

[0965] The slice related information based on the layer group may be included in at least one of the SPS, APS and / or attribute data unit header (or dependency attribute data unit header) and transmitted to the receiving end. For information included in the slice related information based on the layer group not described below, see Figure 40 and Figure 41 .

[0966] Figure 46 is a flowchart illustrating a method of receiving point cloud data according to an embodiment.

[0967] The method of receiving point cloud data according to an embodiment may include: receiving encoded point cloud data and signaling information (operation 81001); decoding the point cloud data based on the signaling information (operation 81002); and rendering the decoded point cloud data (operation 81003).

[0968] Operation 81001 of receiving point cloud data and signaling information according to an embodiment may be performed by Figure 1 Receiver 10005, Figure 2 Send 20002 or decode 20003, or Figure 9 The receiver 9000 or the receiving processor 9001 executes.

[0969] Operation 81002 of decoding point cloud data according to this embodiment may include decoding geometric information and attribute information. Figure 1 Point cloud video decoder 10006, Figure 2 Decoding 20003, Figure 8 Point cloud video decoder, Figure 9 Point cloud video decoder, Figure 32 Geometry decoder and attribute decoder, Figure 33 Attribute decoding or Figure 34 Some or all of the operations in the attribute decoding of .

[0970] Operation 81002 of decoding point cloud data according to an embodiment includes decoding geometric information and decoding attribute information.

[0971] The operation of decoding the geometric information may include decoding (ie, reconstructing) the geometric information based on layer groups and / or subgroups based on the layer group-based slice related information included in the signaling information.

[0972] The operation of decoding the attribute information may include decoding (i.e., decompressing) the attribute information based on the layer group and / or subgroup based on the reconstructed geometric information and the slice-related information based on the layer group included in the signaling information. In one embodiment, the attribute information may be decoded using a combination of one or more of RAHT codec, LoD-based prediction transform codec, and lifting transform codec.

[0973] When layer group slicing is enabled, decoding attribute information may include: generating a LoD based on a subgroup, taking into account IDCM nodes; deriving weights based on the LoD generated based on the subgroup; and performing an attribute transform (e.g., inverse prediction / lifting transform) on points (or nodes) in the LoD to restore (or derive) attributes. In this case, the input points used to generate the LoD for the current subgroup may include points of the current subgroup and points of the parent subgroup. Specifically, because decoding can be performed without reaching the full geometric tree depth, input point generation and LoD generation can be performed after compensating the tree depth by an amount equal to the difference between the final geometric tree depth and the geometric tree depth of the current subgroup. Furthermore, when including points from the parent subgroup associated with the current subgroup as input for LoD generation for the current subgroup, only IDCM points and / or points not included in the current subgroup's range may be added. In other words, points selected from the parent subgroup's points used for LoD generation and NN search for the current subgroup do not overlap with points belonging to the current subgroup. Following this process, the LoD for the current subgroup is generated based on the determined input points, and an NN search is performed based on the generated LoD. Here, in order to take into account the IDCM nodes to perform LoD generation and NN search, the input buffer stores the nodes of the current subgroup and the parent subgroup respectively, as Figure 37 As shown in . In addition, the nodes of the current subgroup are divided into non-IDCM nodes and IDCM nodes and stored. The nodes of the parent subgroup are also divided into non-IDCM nodes and IDCM nodes and stored. The IDCM nodes of the current subgroup are stored sequentially by layer, separated by layer, and the IDCM nodes of the parent subgroup are also stored sequentially, separated by layer. Whether the node stored in the input buffer belongs to the current subgroup or the parent subgroup, and whether it is a non-IDCM node or an IDCM node can be identified based on the index assigned to each node. In one embodiment, the nodes stored in the input buffer based on the index are used for LoD generation and neighbor search of the current subgroup.

[0974] As described above, when performing LoD generation and NN search based on a subgroup, the operation of decoding attribute information may include an operation in which at least one point in the current subgroup inherits the attribute of at least one point in the parent subgroup. In other words, some points belonging to the top LoD of the subgroup match points in the bottom LoD of the parent subgroup. In this case, the attributes of the points belonging to the top LoD of the subgroup may inherit the attributes of the corresponding points belonging to the bottom LoD of the parent subgroup. For details not described herein, please refer to Figures 11 to 38 Description.

[0975] In the rendering (81003) according to the embodiment, the point cloud data can be reconstructed based on the recovered (or reconstructed or derived) geometric information and attribute information, and rendered according to various rendering schemes. For example, the points in the point cloud content can be rendered as vertices with a specific thickness, as cubes with a specific minimum size centered at the corresponding vertex position, as circles centered at the vertex position, etc. All or part of the rendered point cloud content can be presented to the user through a display (e.g., a VR / AR display, a general display, etc.). The rendering of the point cloud data (81003) according to the embodiment can be performed by Figure 1 Renderer 10007, Figure 2 Rendering 20004 or Figure 9 The renderer 9011 executes.

[0976] Therefore, by performing LoD generation and NN search in consideration of IDCM nodes during attribute encoding / decoding, compression efficiency of attribute information can be improved.

[0977] Each of the components, modules, or units described above may be software, a processor, or a hardware component that executes a continuous process stored in a memory (or storage unit). Each of the steps described in the above embodiments may be performed by a processor, a software component, or a hardware component. Each module / block / unit described in the above embodiments may operate as a processor, software, or hardware. Furthermore, the methods presented in the embodiments may be executed as code. This code may be written to a processor-readable storage medium and thus read by a processor provided by the device.

[0978] In this specification, when a component "includes" or "comprising" an element, unless otherwise specified, it means that the component also includes or comprises another element. Furthermore, the term "module (or unit)" disclosed in this specification refers to a unit for processing at least one function or operation and can be implemented using hardware, software, or a combination of hardware and software.

[0979] Although the embodiments have been explained with reference to each of the accompanying drawings for simplicity, new embodiments can be designed by combining the embodiments illustrated in the accompanying drawings. If a person skilled in the art designs a computer-readable recording medium having a program for executing the embodiments mentioned in the foregoing description recorded thereon, it can fall within the scope of the appended claims and their equivalents.

[0980] The device and method may not be limited to the configuration and method of the above-mentioned embodiments. The above-mentioned embodiments may be configured by selectively combining them in whole or in part to achieve various modifications.

[0981] Although preferred embodiments of the embodiments have been shown and described, the embodiments are not limited to the specific embodiments described above, and various modifications may be made by a person of ordinary skill in the art without departing from the spirit of the embodiments claimed for protection in the claims, and these modifications should not be understood separately from the technical concepts or views of the embodiments.

[0982] The various components of the devices of the embodiments may be implemented using hardware, software, firmware, or a combination thereof. Various components of the embodiments may be implemented using a single chip (e.g., a single hardware circuit). Depending on the embodiment, each component of the device may be implemented as a separate chip. Depending on the embodiment, at least one or more components of the device may include one or more processors capable of executing one or more programs. These programs may execute any one or more operations / methods according to the embodiments or include instructions for executing them. The executable instructions for executing the methods / operations of the devices of the embodiments may be stored in a non-transitory CRM or other computer program product configured for execution by one or more processors, or in a transient CRM or other computer program product configured for execution by one or more processors. Furthermore, the term "memory" according to the embodiments may encompass not only volatile memory (e.g., RAM) but also non-volatile memory, flash memory, and PROM. Furthermore, implementation may also be in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, processor-readable recording media may be distributed to computer systems connected via a network, allowing processor-readable code to be stored and executed in a distributed manner.

[0983] In this document, the terms " / " and "", should be interpreted as indicating "and / or". For example, the expression "A / B" may mean "A and / or B". In addition, "A, B" may mean "A and / or B". In addition, "A / B / C" may mean "at least one of A, B and / or C". In addition, "A / B / C" may mean "at least one of A, B and / or C". In addition, in this document, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" may include 1) only A, 2) only B and / or 3) both A and B. In other words, the term "or" in this document should be interpreted as indicating "in addition or alternatively".

[0984] The various elements of the embodiment can be implemented by hardware, software, firmware or a combination thereof. The various elements of the embodiment can be executed by a single chip such as a single hardware circuit. According to the embodiment, the elements can be se...

Claims

1. A method for receiving point cloud data, the method comprising: Receive geometric data, attribute data and signaling information; decoding the geometry data based on the signaling information; decoding the attribute data based on the signaling information and the decoded geometry data; as well as Rendering point cloud data including the decoded geometry data and the decoded attribute data.

2. The method according to claim 1, wherein The decoding of the attribute data includes: receiving the attribute data based on a subgroup; storing, in a buffer, nodes of a current subgroup and nodes of a parent subgroup of the current subgroup based on the signaling information, the decoded geometric data, and the attribute data; generating at least one level of detail (LoD) of the current subgroup based on the nodes of the current subgroup and the nodes of the parent subgroup stored in the buffer; searching for nearest neighbors of a node in the current subgroup to be subjected to attribute decoding based on the at least one LoD of the current subgroup and the at least one LoD of the parent subgroup; and Derive attribute data based on the nearest neighbors found.

3. The method according to claim 2, wherein: The nodes of the parent subgroup used to generate the at least one LoD of the current subgroup are nodes that do not overlap with the nodes of the current subgroup.

4. The method according to claim 3, wherein: The nodes of the parent subgroup used to generate the at least one LoD of the current subgroup include at least a node outside a boundary of the current subgroup or at least one direct compression (IDCM) node.

5. The method according to claim 4, wherein The storage includes: storing the nodes of the current subgroup and the nodes of the parent subgroup separately in the buffer; Based on at least one of the nodes of the current subgroup being an IDCM node, separately storing the non-IDCM nodes of the current subgroup and the at least one IDCM node in the buffer; and Based on at least one of the nodes of the parent-child group being an IDCM node, the non-IDCM nodes of the parent-child group and the at least one IDCM node are stored separately in the buffer, Therein, the nodes stored in the buffer are identified by an index assigned to each of the nodes.

6. The method according to claim 5, wherein: The storage further comprises: Based on the at least one IDCM node of the current subgroup being included in a plurality of layers, separating the at least one IDCM node of the current subgroup according to the layers and storing it in the buffer; and Based on that the at least one IDCM node of the parent-child group is included in a plurality of layers, the at least one IDCM node of the parent-child group is separated according to the layers and stored in the buffer.

7. The method according to claim 6, wherein: The export of attribute data includes: Based on the bottom layer of the parent subgroup having a node at the same position as the node to be reconstructed in the top layer of the current subgroup, attributes of the node in the parent subgroup are matched with attributes of the node in the top layer of the current subgroup.

8. The method according to claim 5, wherein At least one IDCM node of the parent subgroup is a node excluded from the top subsample of the current subgroup.

9. A device for receiving point cloud data, comprising: a receiver configured to receive geometric data, attribute data, and signaling information; a geometry decoder configured to decode the geometry data based on the signaling information; an attribute decoder configured to decode the attribute data based on the signaling information and the decoded geometry data; as well as A renderer is configured to render point cloud data including the decoded geometry data and the decoded attribute data.

10. The device according to claim 9, wherein The attribute decoder comprises: a buffer configured to store a node of a current subgroup and a node of a parent subgroup of the current subgroup based on the signaling information, the decoded geometric data, and subgroup-based attribute data; a level of detail (LoD) generator configured to generate at least one LoD for the current subgroup based on the nodes of the current subgroup and the nodes of the parent subgroup stored in the buffer; a neighbor searcher configured to search for nearest neighbors of a node to be subjected to attribute decoding in the current subgroup based on the at least one LoD of the current subgroup and the at least one LoD of the parent subgroup; and An exporter is configured to export attribute data based on the searched nearest neighbors.

11. The device according to claim 10, wherein The nodes of the parent subgroup used to generate the at least one LoD of the current subgroup are nodes that do not overlap with the nodes of the current subgroup.

12. The device according to claim 11, wherein The nodes of the parent subgroup used to generate the at least one LoD of the current subgroup include at least a node outside a boundary of the current subgroup or at least one direct compression (IDCM) node.

13. The device according to claim 12, wherein The buffer is configured as: storing the nodes of the current child group and the nodes of the parent child group in separate storage areas; Based on the at least one node of the current subgroup being an IDCM node, storing the non-IDCM nodes of the current subgroup and the at least one IDCM node in separate storage areas; as well as Based on the fact that at least one node of the parent-child group is an IDCM node, the non-IDCM nodes of the parent-child group and the at least one IDCM node are stored in separate storage areas, Therein, the nodes stored in the buffer are identified by an index assigned to each of the nodes.

14. The device according to claim 13, wherein The buffer is configured as: Based on the at least one IDCM node of the current subgroup being included in a plurality of layers, separating the at least one IDCM node of the current subgroup according to the layers and storing the same; as well as Based on the at least one IDCM node of the parent-child group being included in a plurality of layers, the at least one IDCM node of the parent-child group is divided according to the layers and stored.

15. The device according to claim 14, wherein Based on the bottom layer of the parent subgroup having a node at the same position as the node to be reconstructed in the top layer of the current subgroup, the exporter matches the attributes of the node in the parent subgroup with the attributes of the node in the top layer of the current subgroup.

Images