Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

Abstract

The invention discloses a point cloud data decoding method and device. A point cloud data decoding method according to an embodiment may comprise the steps of: receiving geometric data, attribute data, and signaling information; decoding the geometric data based on the signaling information; and decoding the attribute data based on the signaling information.

Description

Technical Field

[0001] The embodiments relate to methods and devices for processing point cloud content. Background Technology

[0002] Point cloud content is content represented by a point cloud, which is a collection of points belonging to a coordinate system representing a three-dimensional space (or volume). Point cloud content can represent media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), XR (scaled reality), and autonomous driving services. However, tens of thousands to hundreds of thousands of points are needed to represent point cloud content. Therefore, methods for efficiently processing large amounts of point data are required.

[0003] In other words, high throughput is required to send and receive point cloud data. Therefore, the process of sending and receiving point cloud data, which involves encoding for compression and decoding for decompression, is computationally complex and time-consuming due to the large amount of point cloud data. Summary of the Invention

[0004] Technical issues

[0005] Designed to solve the above problems, the purpose of this disclosure is to provide an apparatus and method for efficiently transmitting / receiving point clouds.

[0006] Another object of this disclosure is to provide an apparatus and method for addressing latency and encoding / decoding complexity.

[0007] Another objective of the embodiments is to provide an apparatus and method for efficiently providing scalable services by performing neighbor searches on a subgroup basis.

[0008] The embodiments are not limited to the objectives described above, and the scope of the embodiments can be extended to other objectives that can be inferred by those skilled in the art based on the entire contents of this disclosure.

[0009] Technical solution

[0010] To achieve these and other advantages and in accordance with the purposes of this disclosure, as embodied and broadly described herein, a method for decoding point cloud data may include: receiving geometric data, attribute data, and signaling information; decoding the geometric data based on the signaling information; and decoding the attribute data based on the signaling information.

[0011] According to an embodiment, geometric data can be partitioned and included in data units of a layer group structure, wherein the geometric data of the current data unit of the current layer group can be decoded by referring to context information related to the data units of the upper layer group stored in memory.

[0012] According to an embodiment, for each management memory in a data unit, the memory referenced to decode the geometric data of the current data unit can be divided into a context state storage area, a reference count information storage area, and a counter information storage area. The context state storage area can store context information related to the data units of the upper-level group. The reference count information storage area can store the number of data units referencing the data units of the upper-level group for each layer in the layer group constituting the layer group structure. The counter information storage area can store counter information updated each time a data unit of the upper-level group is used for each layer in the layer group.

[0013] According to an embodiment, decoding of geometric data may include: skipping storing context information related to the current data unit of the current layer group being decoded in memory, based on the absence of subsequent data units that reference context information associated with the current data unit.

[0014] According to an embodiment, decoding of geometric data may include skipping the storage of context information related to the current data unit of the current layer group in memory, based on the fact that the current layer group is the last layer group in the layer group structure.

[0015] According to an embodiment, decoding of geometric data may include: skipping the storage of context information related to the current data unit of the current layer group in memory, based on the fact that the current layer group is the last layer group in the layer group structure that excludes the layer group for which decoding is skipped.

[0016] According to an embodiment, decoding of geometric data may include determining the release of context information associated with the data unit of the upper-level group based on values ​​stored in a reference count information storage area and values ​​stored in a counter information storage area.

[0017] According to an embodiment, the signaling information may include information for identifying data units of an upper-level group of geometric data referenced to decode the current data unit.

[0018] According to an embodiment, the signaling information may include information relating to the number of reference data units in each of the layer groups in the reference count information storage area for storage in the memory.

[0019] According to an embodiment, an apparatus for decoding point cloud data may include: a receiver configured to receive geometric data, attribute data, and signaling information; a geometric decoder configured to decode geometric data based on the signaling information; and an attribute decoder configured to decode attribute data based on the signaling information.

[0020] According to an embodiment, geometric data can be partitioned and included in data units of a layer group structure, wherein the geometric decoder can decode the geometric data of the current data unit of the current layer group by referring to context information related to the data units of the upper layer group stored in memory.

[0021] According to an embodiment, for each management memory in a data unit, the memory referenced to decode the geometric data of the current data unit can be divided into a context state storage area, a reference count information storage area, and a counter information storage area. The context state storage area can store context information related to the data units of the upper-level group. The reference count information storage area can store the number of data units referencing the data units of the upper-level group for each layer in the layer group constituting the layer group structure. The counter information storage area can store counter information updated each time a data unit of the upper-level group is used for each layer in the layer group.

[0022] According to an embodiment, the geometry decoder can determine the release of context information related to data units of the upper group stored in the context state storage area based on values ​​stored in the reference count information storage area and values ​​stored in the counter information storage area.

[0023] According to an embodiment, the method for encoding point cloud data may include: encoding geometric data; encoding attribute data; and transmitting the encoded geometric data, encoded attribute data, and signaling information.

[0024] According to an embodiment, an apparatus for encoding point cloud data may include: a geometric encoder configured to encode geometric data; an attribute encoder configured to encode attribute data; and a transmitter configured to transmit the encoded geometric data, the encoded attribute data, and signaling information.

[0025] Beneficial effects

[0026] The apparatus and method according to the embodiments can provide high-quality point cloud services.

[0027] The apparatus and method according to the embodiments can implement various video codec methods.

[0028] The apparatus and method according to the embodiments can provide general point cloud content, such as autonomous driving services.

[0029] The apparatus and method according to the embodiments can perform spatial adaptive partitioning of point cloud data for independent encoding and decoding of point cloud data, thereby improving parallel processing and providing scalability.

[0030] The apparatus and method according to the embodiments can improve the encoding and decoding performance of point clouds by partitioning point cloud data into units of tiles and / or slices and transmitting its necessary data with signals.

[0031] The apparatus and method according to the embodiments can divide and transmit compressed point cloud data according to predetermined criteria. Furthermore, when using layered encoding, compressed data can be divided and transmitted according to layers. Therefore, the storage and transmission efficiency of the transmitting apparatus can be increased.

[0032] The apparatus and method according to the embodiments can efficiently manage the context buffer in decoding a bitstream in a layered structure consisting of multiple data units.

[0033] The apparatus and method according to the embodiments can effectively manage the context buffer during partial decoding by managing the context buffer for each layer group of each data unit in a layer group structure consisting of multiple data units. Attached Figure Description

[0034] The accompanying drawings are included to provide a further understanding of this disclosure and are incorporated in and constitute a part of this application. The drawings illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure. In the drawings: Figure 1 An exemplary point cloud content providing system according to an embodiment is shown; Figure 2 This is a block diagram illustrating the operation provided by point cloud content according to an embodiment; Figure 3 The figure illustrates an exemplary point cloud encoder according to an embodiment; Figure 4 An example of an octree and occupancy code according to an embodiment is shown; Figure 5 The illustration shows an example of point configuration in each LOD according to an embodiment; Figure 6 The illustration shows an example of point configuration in each LOD according to an embodiment; Figure 7 The diagram illustrates a point cloud decoder according to an embodiment; Figure 8 The figure shows a transmitting apparatus according to an embodiment; Figure 9 The figure shows a receiving device according to an embodiment; Figure 10 The diagram illustrates an exemplary structure operable with respect to a point cloud data transmission / reception method / apparatus according to an embodiment; Figure 11 and Figure 12 This is a diagram illustrating the process of encoding, transmitting, and decoding point cloud data according to an embodiment; Figure 13 The diagram illustrates a layer-based configuration of point cloud data according to an embodiment. Figure 14 (a) The figure illustrates a bitstream structure for geometric data according to an embodiment, and Figure 14 (b) The diagram illustrates the bitstream structure for attribute data according to an embodiment; Figure 15 This is a diagram illustrating an example of a configuration for dividing a bitstream into layers for delivery, according to an embodiment. Figure 16 The illustration shows an exemplary bitstream sorting method used according to an embodiment when a geometric bitstream and an attribute bitstream are multiplexed into a single bitstream; Figure 17 The illustration shows another exemplary bitstream sorting method used according to an embodiment when the geometric bitstream and the attribute bitstream are multiplexed into a single bitstream; Figure 18 (a) to Figure 18 (c) An example of selecting symmetry geometric properties according to the embodiment; Figure 19 (a) to Figure 19 (c) An example of selecting asymmetric geometric properties according to the embodiment; Figure 20 (a) to Figure 20 (c) The illustration shows an exemplary method of slicing point cloud data according to an embodiment; Figure 21 (a) and Figure 21 (b) The illustration shows the geometric tree structure based on a single slice and a segmented slice according to the embodiments; Figure 22 (a) and Figure 22 (b) The illustration shows the hierarchical structure of the geometric coding tree and the aligned hierarchical structure of the attribute coding tree according to the embodiment; Figure 23 The diagram illustrates the layer and subgroup structure according to an embodiment; Figure 24 The illustration shows an example of context reference between layer groups according to an embodiment; Figure 25 The illustration shows an example of contextual references between groups according to an embodiment; Figure 26 (a) to Figure 26 (c) An example of context buffer management according to an embodiment; Figure 27 (a) to Figure 27 (c) An example of context buffer management according to an embodiment; Figure 28 (a) to Figure 28(c) An example of context buffer management according to an embodiment; Figure 29 (a) to Figure 29 (c) The illustration shows a context buffer release method according to an embodiment; Figure 30 The illustration shows a context buffer release method according to an embodiment; Figure 31 The illustration shows a context memory management method according to an embodiment; Figure 32 The illustration shows a bitstream containing point cloud data according to an embodiment; Figure 33 The illustration shows an example of the syntax structure of the Sequence Parameter Set (SPS) in a bitstream according to an embodiment; Figure 34 The illustration shows an example of the syntax structure of the geometry-dependent data unit header in a bitstream according to an embodiment; Figure 35 The illustration shows an example of the syntax structure of the Dependency Attribute Data Unit header in a bitstream according to an embodiment; Figures 36A and 36B illustrate examples of the syntax structure of the Layer Group Structure List (LGSI) according to an embodiment; Figure 37 The figure illustrates a point cloud data transmission apparatus / method according to an embodiment; Figure 38 The figure illustrates a point cloud data receiving apparatus / method according to an embodiment; Figure 39 The figure illustrates a point cloud data receiving method according to an embodiment; Figure 40 The illustration shows a point cloud data encoding method based on layer groups according to an embodiment; Figure 41 The figure illustrates a point cloud data decoding method based on layer groups according to an embodiment; Figure 42 (a) to Figure 42 (c) The illustration shows a context buffer management method according to an embodiment; Figure 43 (a) to Figure 43 (c) The illustration shows a context buffer management method according to an embodiment; Figure 44 (a) to Figure 44 (c) The illustration shows a context buffer management method according to an embodiment; Figure 45 The illustration shows an example of the syntax structure of the geometric data unit header according to an embodiment; Figure 46 The illustration shows an example of the syntax structure of the dependent geometry data unit header according to an embodiment; Figure 47 The illustration shows an example of the syntax structure of the attribute data unit header according to an embodiment; Figure 48 The illustration shows an example of the syntax structure of the Dependency Attribute Data Unit header according to an embodiment; Figure 49 This is a diagram illustrating an example of a layer group structure considering partial decoding according to an embodiment; Figure 50 (a) to Figure 50 (d) The illustration shows a context buffer management method according to an embodiment; Figure 51 The illustration shows an example of the syntax structure of the sequence parameter set according to an embodiment; Figure 52 The illustration shows an example of the syntax structure of the geometric data unit header according to an embodiment; Figure 53 The illustration shows an example of the syntax structure of the dependent geometry data unit header according to an embodiment; Figure 54 The illustration shows an example of the syntax structure of the attribute data unit header according to an embodiment; Figure 55 The illustration shows an example of the syntax structure of the Dependency Attribute Data Unit header according to an embodiment; Figure 56 This is a diagram illustrating an example of providing services by compressing the geometry and attributes of point cloud data; Figure 57 This is a diagram illustrating another example of providing services by compressing the geometry and attributes of point cloud data according to an embodiment; Figure 58 This diagram illustrates the operations on the sending and receiving sides when sending point cloud data composed of layers. Figure 59 The diagram illustrates a point cloud data transmission / reception apparatus / method according to an embodiment; Figure 60 This is a flowchart illustrating a point cloud data transmission method according to an embodiment; and Figure 61 This is a flowchart illustrating a point cloud data receiving method according to an embodiment. Detailed Implementation

[0035] Now, reference will be made in detail to preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The following detailed description, given with reference to the accompanying drawings, is intended to explain exemplary embodiments of the present disclosure and not to show only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0036] While most of the terms used in this disclosure are selected from commonly used terms in the art, the applicant has arbitrarily chosen some terms, and their meanings will be explained in detail in the following description as needed. Therefore, this disclosure should be understood based on the literal meaning of the terms rather than their simple names or connotations.

[0037] Figure 1 An exemplary point cloud content delivery system according to an embodiment is shown.

[0038] Figure 1 The point cloud content providing system shown may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 are capable of transmitting and receiving point cloud data via wired or wireless communication.

[0039] The point cloud data transmission device 10000 according to an embodiment can acquire and process point cloud video (or point cloud content) and transmit it. According to an embodiment, the transmission device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device, and / or a server. According to an embodiment, the transmission device 10000 may include devices configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers.

[0040] According to an embodiment, the transmitting device 10000 includes a point cloud video acquisition unit 10001, a point cloud video encoder 10002, and / or a transmitter (or communication module) 10003.

[0041] The point cloud video acquisition unit 10001 according to an embodiment acquires point cloud video through processing procedures such as capture, synthesis, or generation. Point cloud video is point cloud content represented by a point cloud, which is a set of points located in 3D space, and may be referred to as point cloud video data, point cloud data, etc. The point cloud video according to an embodiment may include one or more frames. A frame represents a still image / picture. Therefore, point cloud video may include point cloud images / frames / pictures, and may be referred to as point cloud images, frames, or pictures.

[0042] The point cloud video encoder 10002 according to an embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 can encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to an embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next-generation coding. The point cloud compression coding according to an embodiment is not limited to the above embodiments. The point cloud video encoder 10002 can output a bitstream containing the encoded point cloud video data. The bitstream may contain not only the encoded point cloud video data but also signaling information related to the encoding of the point cloud video data.

[0043] According to an embodiment, transmitter 10003 transmits a bitstream containing encoded point cloud video data. The bitstream, according to an embodiment, is encapsulated in a file or segment (e.g., a streaming segment) and transmitted via various networks such as broadcast networks and / or broadband networks. Although not shown in the figures, transmitting device 10000 may include an encapsulator (or encapsulation module) configured to perform encapsulation operations. According to an embodiment, the encapsulator may be included in transmitter 10003. According to an embodiment, the file or segment may be transmitted via a network to receiving device 10004 or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). Transmitter 10003 according to an embodiment is capable of wired / wireless communication with receiving device 10004 (or receiver 10005) via networks such as 4G, 5G, and 6G. Additionally, the transmitter may perform necessary data processing operations depending on the network system (e.g., a 4G, 5G, or 6G communication network system). Transmitting device 10000 may transmit encapsulated data on demand.

[0044] According to an embodiment, the receiving device 10004 includes a receiver 10005, a point cloud video decoder 10006, and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include devices, robots, vehicles, AR / VR / XR devices, portable devices, home appliances, Internet of Things (IoT) devices, and AI devices / servers configured to communicate with base stations and / or other wireless devices using radio access technologies (e.g., 5G New RAT (NR), Long Term Evolution (LTE)).

[0045] According to an embodiment, receiver 10005 receives a bitstream containing point cloud video data or a file / segment encapsulated with a bitstream from a network or storage medium. Receiver 10005 may perform necessary data processing according to the network system (e.g., a communication network system such as 4G, 5G, 6G, etc.). According to an embodiment, receiver 10005 may decapsulate the received file / segment and output a bitstream. According to an embodiment, receiver 10005 may include a decapsulator (or decapsulator module) configured to perform a decapsulation operation. The decapsulator may be implemented as a separate element (or component) from receiver 10005.

[0046] The point cloud video decoder 10006 decodes the bitstream containing point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to the method in which the point cloud video data is encoded (e.g., the reverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (the reverse process of point cloud compression). Point cloud decompression encoding includes G-PCC encoding.

[0047] Renderer 10007 renders decoded point cloud video data. In one embodiment, renderer 10007 can render decoded point cloud video data according to a viewport, etc. Renderer 10007 can render not only point cloud video data but also audio data to output point cloud content. According to an embodiment, renderer 10007 may include a display configured to display point cloud content. According to an embodiment, the display may be implemented as a separate device or component rather than included in renderer 10007.

[0048] The arrows indicated by dashed lines in the diagram represent the transmission path of the feedback information acquired by the receiving device 10004. The feedback information reflects the interactivity of the user consuming the point cloud content and includes information about the user (e.g., header orientation information, viewport information, etc.). Specifically, when the point cloud content is for a service requiring user interaction (e.g., autonomous driving services, etc.), the feedback information may be provided to the content sender (e.g., the sending device 10000) and / or the service provider. According to embodiments, the feedback information may be used in both the receiving device 10004 and the sending device 10000, or it may not be provided.

[0049] According to the embodiment, the header orientation information can represent information about the position, orientation, angle, and movement of the user's header. The receiving device 10004 according to the embodiment can calculate viewport information based on the header orientation information. Viewport information is information about the region of the point cloud video that the user is viewing (that is, the region the user is currently viewing). In other words, viewport information is information about the region the user is currently viewing in the point cloud video. In other words, the viewport or viewport region can represent the region the user is viewing in the point cloud video. The viewpoint is the point the user is viewing in the point cloud video and can represent the center point of the viewport region. That is, the viewport is the region centered on the viewpoint, and the size and shape of the region can be determined by the field of view (FOV). Therefore, the receiving device 10004 can extract viewport information based on the vertical or horizontal FOV supported by the device and the header orientation information. Furthermore, the receiving device 10004 can perform gaze analysis, etc., based on the header orientation information and / or viewport information to determine how the user consumes the point cloud video, the region the user is gazing at in the point cloud video, and the gaze duration. According to an embodiment, the receiving device 10004 can send feedback information, including gaze analysis results, to the transmitting device 10000. According to an embodiment, a device such as a VR / XR / AR / MR display can extract the viewport region based on the user's header position / orientation and a vertical or horizontal FOV supported by the device. According to an embodiment, header orientation information and viewport information can be referred to as feedback information, signaling information, or metadata.

[0050] Feedback information according to the embodiments can be acquired during rendering and / or display. The feedback information can be acquired by one or more sensors included in the receiving device 10004. According to the embodiments, the feedback information can be ensured by the renderer 10007 or by separate external components (or devices, components, etc.). Figure 1 The dashed lines in the diagram represent the process of sending feedback information ensured by renderer 10007. This feedback information can be sent not only to the sending side but also consumed at the receiving side. In other words, the point cloud content providing system can process (encode / decode / render) point cloud data based on the feedback information. For example, point cloud video decoder 10006 and renderer 10007 can prioritize decoding and rendering point cloud video only for the area currently being viewed by the user, based on the feedback information (i.e., header orientation information and / or viewport information).

[0051] Furthermore, the receiving device 10004 can send feedback information to the transmitting device 10000. The transmitting device 10000 (or the point cloud video data encoder 10002) can perform encoding operations based on the feedback information. Therefore, the point cloud content providing system can effectively process the necessary data (e.g., point cloud data corresponding to the user's header position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide the point cloud content to the user.

[0052] According to the embodiments, the transmitting device 10000 can be referred to as an encoder, transmitting device, transmitter, transmitting system, etc., and the receiving device 10004 can be referred to as a decoder, receiving device, receiver, receiving system, etc.

[0053] According to the embodiments Figure 1 Point cloud data processed in a point cloud content provision system (through a series of processes including acquisition, encoding, transmission, decoding, and rendering) can be referred to as point cloud content data or point cloud video data. According to embodiments, point cloud content data can be used as a concept encompassing metadata or signaling information related to point cloud data.

[0054] Figure 1 The components of the point cloud content provided by the system can be implemented by hardware, software, processors, and / or combinations thereof.

[0055] Figure 2 This is a block diagram illustrating the point cloud content provisioning operation according to an embodiment.

[0056] Figure 2 The block diagram shows Figure 1 The operation of the point cloud content providing system described herein. As mentioned above, the point cloud content providing system can process point cloud data based on point cloud compression encoding (e.g., G-PCC).

[0057] A point cloud content providing system (e.g., point cloud sending device 10000 or point cloud video acquisition unit 10001) according to an embodiment can acquire point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system used to represent 3D space. The point cloud video according to an embodiment may include Ply (Polygon file format or Stanford Triangle format) files. When the point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. A Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of the points. The position of each point may be represented by parameters (e.g., values ​​of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include the attributes of the points (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc., of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes: color and reflectivity. According to embodiments, geometry can be referred to as location, geometric information, geometric data, location information, location data, etc., and attributes can be referred to as attributes, attribute information, attribute data, etc. A point cloud content providing system (e.g., point cloud sending device 10000 or point cloud video acquisition unit 10001) can obtain point cloud data from information related to the point cloud video acquisition process (e.g., depth information, color information, etc.).

[0058] A point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) according to an embodiment can encode point cloud data (20001). The point cloud content providing system can encode point cloud data based on point cloud compression encoding. As described above, point cloud data can include geometric information and attribute information about points. Therefore, the point cloud content providing system can perform geometric encoding to encode geometry and output a geometric bitstream. The point cloud content providing system can perform attribute encoding to encode attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system can perform attribute encoding based on geometric encoding. The geometric bitstream and attribute bitstream according to an embodiment can be multiplexed and output as a single bitstream. The bitstream according to an embodiment may also contain signaling information related to geometric encoding and attribute encoding.

[0059] A point cloud content providing system (e.g., transmitting device 10000 or transmitter 10003) according to an embodiment can transmit encoded point cloud data (20002). Figure 1 As shown, encoded point cloud data can be represented by geometric bitstreams and attribute bitstreams. Additionally, the encoded point cloud data can be transmitted as a bitstream along with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometric encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and transmit it as a file or fragment.

[0060] The point cloud content providing system (e.g., receiving device 10004 or receiver 10005) according to the embodiment can receive a bitstream containing encoded point cloud data. Additionally, the point cloud content providing system (e.g., receiving device 10004 or receiver 10005) can demultiplex the bitstream.

[0061] A point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode encoded point cloud data (e.g., geometric bitstream, attribute bitstream) transmitted in a bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode point cloud video data based on signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can decode the geometric bitstream to reconstruct the location (geometry) of the points. The point cloud content providing system can reconstruct the attributes of the points by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., receiving device 10004 or point cloud video decoder 10005) can reconstruct point cloud video based on location according to the reconstructed geometry and the decoded attributes.

[0062] A point cloud content providing system (e.g., receiving device 10004 or renderer 10007) according to an embodiment can render decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) can use various rendering methods to render the geometry and attributes decoded through the decoding process. Points in the point cloud content can be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user through a display (e.g., a VR / AR display, a general display, etc.).

[0063] The point cloud content providing system (e.g., receiving device 10004) according to an embodiment can obtain feedback information (20005). The point cloud content providing system can encode and / or decode point cloud data based on the feedback information. The feedback information and operation of the point cloud content providing system according to an embodiment are related to reference. Figure 1 The feedback information and operation described are the same, so their detailed description is omitted.

[0064] Figure 3 An exemplary point cloud encoder according to an embodiment is shown.

[0065] Figure 3 Show Figure 1 An example of a point cloud video encoder 10002. The point cloud encoder reconstructs and encodes point cloud data (e.g., point locations and / or attributes) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) based on network conditions or applications. When the total size of the point cloud content is large (e.g., providing 60 Gbps of point cloud content for 30 fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system can reconstruct the point cloud content based on a maximum target bitrate to provide the point cloud content according to network conditions, etc.

[0066] For reference Figure 1 and Figure 2 As described, the point cloud encoder can perform geometric encoding and attribute encoding. Geometric encoding is performed before attribute encoding.

[0067] The point cloud video encoder according to an embodiment includes a coordinate transformer (transform coordinates) 30000, a quantizer (quantizes and removes points (voxarization)) 30001, an octree analyzer (analyzes octrees) 30002, a surface approximation analyzer (analyzes surface approximations) 30003, an arithmetic encoder (arithmetic encoding) 30004, a geometry reconstructor (reconstructs geometry) 30005, a color transformer (transforms colors) 30006, an attribute transformer (transforms attributes) 30007, a RAHT transformer 30008, a LOD generator (generates LODs) 30009, a lift transformer (lifts) 30010, a coefficient quantizer (quantizes coefficients) 30011, and / or an arithmetic encoder (arithmetic encoding) 30012. Figure 3 In the point cloud encoder, coordinate transformer 30000, quantizer 30001, octree analyzer 30002, surface approximation analyzer 30003, arithmetic encoder 30004, and geometric reconstructor 30005 can be grouped together and called the geometric encoder. Color transformer 30006, attribute transformer 30007, RAHT transformer 30008, LOD generator 30009, lift transformer 30010, coefficient quantizer 30011, and / or arithmetic encoder 30012 can be grouped together and called the attribute encoder.

[0068] Coordinate transformer 30000, quantizer 30001, octree analyzer 30002, surface approximation analyzer 30003, arithmetic encoder 30004, and geometric reconstructor 30005 are capable of performing geometric coding. Geometric coding according to embodiments may include octree geometric coding, prediction tree geometric coding, direct coding, triplet geometric coding, and entropy coding. Direct coding and triplet geometric coding are applied selectively or in combination. Geometric coding is not limited to the examples described above.

[0069] As shown in the figure, the coordinate transformer 30000 according to an embodiment receives a position and transforms it into coordinates. For example, the position can be transformed into position information in three-dimensional space (e.g., three-dimensional space represented by the XYZ coordinate system). The position information in three-dimensional space according to an embodiment can be referred to as geometric information.

[0070] According to an embodiment, quantizer 30001 performs geometric quantization. For example, quantizer 30001 may quantize points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). Quantizer 30001 performs a quantization operation: multiplying the difference between the minimum position value and the position value of each point by a preset quantization scaling value, and then finding the nearest integer value by rounding the value obtained through multiplication. Thus, one or more points may have the same quantized position (or position value). According to an embodiment, quantizer 30001 performs voxelization based on the quantized position to reconstruct the quantized points. As in the case of pixels (the smallest unit containing 2D image / video information), points in point cloud content (or 3D point cloud video) according to an embodiment may be included in one or more voxels. As a combination of volume and pixel, the term voxel refers to a 3D cubic space generated when 3D space is divided into units (unit = 1.0) based on axes representing 3D space (e.g., X-axis, Y-axis, and Z-axis). The quantizer 30001 allows a group of points in 3D space to be matched with voxels. According to one embodiment, a voxel may include only one point. According to another embodiment, a voxel may include one or more points. To represent a voxel as a point, the location of the voxel's center can be set based on the locations of one or more points included in the voxel. In this case, attributes of all locations included in a voxel can be combined and assigned to the voxel.

[0071] According to the embodiment, the octree analyzer 30002 performs octree geometric encoding (or octree coding) to represent voxels in an octree structure. The octree structure represents points based on the matching of octree structures with voxels.

[0072] The surface approximation analyzer 30003 according to the embodiment can analyze and approximate an octree. The octree analysis and approximation according to the embodiment is a process of analyzing a region containing multiple points to efficiently provide an octree and voxelization.

[0073] According to an embodiment, the arithmetic encoder 30004 performs entropy encoding on octrees and / or approximate octrees. For example, the encoding scheme includes arithmetic encoding. As a result of the encoding, a geometric bitstream is generated.

[0074] The attribute encoding is performed by a color transformer 30006, an attribute transformer 30007, a RAHT transformer 30008, a LOD generator 30009, a boosting transformer 30010, a coefficient quantizer 30011, and / or an arithmetic encoder 30012. As described above, a point may have one or more attributes. The attribute encoding according to the embodiment is also applied to the attributes that a point has. However, when an attribute (e.g., color) includes one or more elements, the attribute encoding is applied independently to each element. The attribute encoding according to the embodiment includes color transformation encoding, attribute transformation encoding, region adaptive hierarchical transformation (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transformation) encoding, and interpolation-based hierarchical nearest neighbor prediction (boosting transformation) encoding with update / boosting steps. Depending on the point cloud content, the above-described RAHT encoding, prediction transformation encoding, and boosting transformation encoding may be used selectively, or a combination of one or more encoding schemes may be used. The attribute encoding according to the embodiment is not limited to the examples described above.

[0075] The color converter 30006 according to an embodiment performs color transformation encoding that transforms the color values ​​(or textures) included in the attributes. For example, the color converter 30006 can transform the format of color information (e.g., from RGB to YCbCr). Optionally, the operation of the color converter 30006 according to an embodiment can be applied based on the color values ​​included in the attributes.

[0076] According to the embodiment, the geometry reconstructor 30005 reconstructs (decompresses) octrees and / or approximate octrees. The geometry reconstructor 30005 reconstructs the octree / voxel based on the results of analyzing the point distribution. The reconstructed octree / voxel may be referred to as the reconstructed geometry (recovered geometry).

[0077] According to an embodiment, the attribute transformer 30007 performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometric encoding is not performed. As described above, since attributes depend on geometry, the attribute transformer 30007 can transform attributes based on reconstructed geometric information. For example, based on the position value of a point included in a voxel, the attribute transformer 30007 can transform the attributes of the point at that location. As described above, when the center position of a voxel is set based on the positions of one or more points included in the voxel, the attribute transformer 30007 transforms the attributes of one or more points. When performing triadic geometric encoding, the attribute transformer 30007 can transform attributes based on the triadic geometric encoding.

[0078] The attribute transformer 30007 performs attribute transformation by calculating the average of the attributes or attribute values ​​(e.g., color or reflectivity of each point) of neighboring points within a specific location / radius from the center (or location value) of each voxel. The attribute transformer 30007 can apply weights based on the distance from the center to each point when calculating the average. Therefore, each voxel has a location and a calculated attribute (or attribute value).

[0079] The attribute transformer 30007 can search for nearest neighbors within a specific location / radius of the center of each voxel based on a KD-tree or Morton code. A KD-tree is a binary search tree and supports a data structure that allows points to be managed based on location, enabling fast nearest neighbor search (NNS). Morton codes are generated by representing the coordinates (e.g., (x, y, z)) of the 3D location of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the point location are (5, 9, 1), the bit values ​​are (0101, 1001, 0001). Mixing the bit values ​​according to the bit index in the order of z, y, and x produces 010001000111. This value is represented as the decimal number 1095. That is, the Morton code value for the point with coordinates (5, 9, 1) is 1095. The attribute transformer 30007 can sort the points based on their Morton code values ​​and perform NNS using a depth-first traversal process. After an attribute transformation operation, use a KD tree or Morton code when an NNS is needed in another transformation process used for attribute encoding.

[0080] As shown in the figure, the transformation properties are input to the RAHT transformer 30008 and / or the LOD generator 30009.

[0081] According to an embodiment, the RAHT transformer 30008 performs RAHT encoding for predicting attribute information based on reconstructed geometric information. For example, the RAHT transformer 30008 can predict the attribute information of higher-level nodes in an octree based on attribute information associated with lower-level nodes in the octree.

[0082] The LOD generator 30009 according to the embodiment generates a Level of Detail (LOD) to perform predictive transform coding. The LOD according to the embodiment represents the level of detail of the point cloud content. As the LOD value decreases, it indicates a deterioration in the detail of the point cloud content. As the LOD value increases, it indicates an enhancement in the detail of the point cloud content. Points can be classified by LOD.

[0083] The lift transformer 30010 according to the embodiment performs lift transform coding to transform point cloud attributes based on weights. As described above, lift transform coding may optionally be applied.

[0084] According to the embodiment, the coefficient quantizer 30011 quantizes the attribute encoded by the attribute based on the coefficient.

[0085] According to the embodiment, the arithmetic encoder 30012 encodes quantized attributes based on arithmetic coding.

[0086] Although not shown in the figure, Figure 3 The elements of the point cloud encoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above-described... Figure 3 At least one of the operation and / or functions of the elements of the point cloud encoder. Additionally, one or more processors are operable or perform operations for executing... Figure 3 The software program and / or instruction set for the operation and / or function of the elements of the point cloud encoder. One or more memories according to the embodiments may include high-speed random access memory, or include non-volatile memory (e.g., one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices).

[0087] Figure 4 An example of an octree and occupancy code according to an embodiment is shown.

[0088] For reference Figures 1 to 3 As described, the point cloud content delivery system (point cloud video encoder 10002) or point cloud encoder (e.g., octree analyzer 30002) performs octree geometric encoding (or octree encoding) based on an octree structure to efficiently manage the regions and / or locations of voxels.

[0089] Figure 4 The upper part shows an octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes of a coordinate system (e.g., the X, Y, and Z axes). This is achieved by two poles (0, 0, 0) and (2... d , 2 d , 2 d An octree structure is created by recursively subdividing the bounding box aligned to the cubic axis. Here, 2d can be set as the value of the minimum bounding box that constitutes all points surrounding the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following equation. In the following equation, (x int n , y int n , z int n ) indicates the position (or position value) of the quantized point.

[0090]

[0091] like Figure 4 As shown in the upper center, the entire 3D space can be divided into eight spaces according to partitions. Each partitioned space is represented by a cube with six faces. For example... Figure 4 As shown in the upper right, each of the eight spaces is further subdivided based on a coordinate system axis (e.g., the X, Y, and Z axes). Thus, each space is divided into eight smaller spaces. These smaller spaces are also represented by cubes with six faces. This partitioning scheme is applied until the leaf nodes of the octree become voxels.

[0092] Figure 4 The lower part shows the octree occupancy code. The occupancy code generates the octree to indicate whether each of the eight partitions generated by dividing a space contains at least one point. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of a partitioned space, and each child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to a child node contains at least one point, the node is assigned a value of 1. When the space corresponding to a child node does not contain a point (the space is empty), the node is assigned a value of 0. Since... Figure 4 The occupancy code shown is 00100001, therefore, it indicates that the space corresponding to the third and eighth child nodes among the eight child nodes each contains at least one point. As shown, each of the third and eighth child nodes has eight child nodes, and the child nodes are represented by an 8-bit occupancy code. The figure shows that the occupancy code for the third child node is 10000111, and the occupancy code for the eighth child node is 01001111. A point cloud encoder (e.g., an arithmetic encoder 30004) according to an embodiment can perform entropy coding on the occupancy code. To increase compression efficiency, the point cloud encoder can perform intra-frame / inter-frame coding on the occupancy code. A receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to an embodiment reconstructs the octree based on the occupancy code.

[0093] A point cloud encoder according to an embodiment (e.g., Figure 4 A point cloud encoder or octree analyzer (30002) can perform voxelization and octree encoding to store point locations. However, points are not always uniformly distributed in 3D space, so there may be specific regions with fewer points. Therefore, performing voxelization over the entire 3D space is inefficient. For example, when a specific region contains very few points, voxelization is not necessary in that specific region.

[0094] Therefore, for the aforementioned specific region (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment can skip voxelization and perform direct encoding to directly encode the point positions included in the specific region. The coordinates of the directly encoded points according to the embodiment are called the Direct Encoding Mode (DCM). The point cloud encoder according to the embodiment can also perform triadic geometry encoding based on the surface model, which reconstructs the point positions in the specific region (or node) based on voxels. Triadic geometry encoding is a geometric encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and triadic geometry encoding according to the embodiment can be performed selectively. In addition, direct encoding and triadic geometry encoding according to the embodiment can be performed in combination with octree geometry encoding (or octree encoding).

[0095] To perform direct encoding, the option to apply direct encoding using direct mode should be enabled. The node to which direct encoding is applied must not be a leaf node, and there should be fewer than a threshold number of points within that node. Furthermore, the total number of points to which direct encoding is applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 30004) according to the embodiment can perform entropy encoding on the point locations (or location values).

[0096] A point cloud encoder according to an embodiment (e.g., a surface approximation analyzer 30003) can determine a specific level of an octree (a level less than the depth d of the octree) and can begin using a surface model to perform triadic geometry encoding to reconstruct point locations in a node region based on voxels (triadic mode). The point cloud encoder according to an embodiment can specify the level to which triadic geometry encoding is applied. For example, the point cloud encoder does not operate in triadic mode when the specific level is equal to the depth of the octree. In other words, the point cloud encoder according to an embodiment can operate in triadic mode only when the specified level is less than the depth value of the octree. A 3D cubic region of a node at a specified level according to an embodiment is called a block. A block may include one or more voxels. A block or voxel may correspond to a cube. Geometry is represented as surfaces within each block. A surface according to an embodiment may intersect each edge of a block at most once.

[0097] A block has 12 edges, therefore a block contains at least 12 intersections. Each intersection is called a vertex. Vertices along an edge are detected when there is at least one occupied voxel adjacent to the edge in all blocks sharing the edge. An occupied voxel, according to an embodiment, refers to a voxel containing a point. The vertex position detected along an edge is the average position of the edges of all voxels adjacent to the edge in all blocks sharing the edge.

[0098] Once a vertex is detected, the point cloud encoder according to the embodiment can perform entropy encoding on the edge's origin (x, y, z), the edge's direction vector (Δx, Δy, Δz), and the vertex position value (relative position value within the edge). When applying triad geometry encoding, the point cloud encoder according to the embodiment (e.g., geometry reconstructor 30005) can generate the restored geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization processes.

[0099] Vertices located at the edges of a block determine the surface passing through the block. According to the embodiment, the surface is a non-planar polygon. During triangle reconstruction, the surface represented by the triangles is reconstructed based on the origin of the edges, the direction vectors of the edges, and the position values ​​of the vertices. The triangle reconstruction process is performed as follows: i) calculating the centroid value of each vertex, ii) subtracting the centroid value from each vertex value, and iii) estimating the sum of squares of the values ​​obtained through the subtraction.

[0100]

[0101] The minimum value of the sum is estimated, and a projection process is performed based on the axis with the minimum value. For example, when element x is minimum, each vertex is projected onto the x-axis relative to the center of the block, and onto the (y, z) plane. When the value obtained by projection onto the (y, z) plane is (ai, bi), the value of θ is estimated by atan2(bi, ai), and the vertices are sorted based on the value of θ. Table 1 below shows the vertex combinations for creating triangles based on the number of vertices. Vertices are sorted from 1 to n. Table 1 below shows that for four vertices, two triangles can be constructed based on vertex combinations. The first triangle can be composed of vertices 1, 2, and 3 from the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 from the sorted vertices.

[0102] [Table 1]

[0103] [Table 1] Triangles formed from vertices sorted by 1, ..., n

[0104] An upsampling process is performed to add points along the edges of the triangle at the center, and voxelization is then performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called thinned vertices. The point cloud encoder according to an embodiment can voxelize the thinned vertices. Additionally, the point cloud encoder can perform attribute encoding based on the voxelized positions (or position values).

[0105] Figure 5 An example of point configuration in each LOD according to an embodiment is shown.

[0106] For reference Figures 1 to 4The described approach involves reconstructing (decompressing) the encoded geometry before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation may include altering the placement of directly encoded points (e.g., placing directly encoded points in front of the point cloud data). When triadic geometry encoding is applied, the geometry reconstruction process is performed through triangle reconstruction, upsampling, and voxelization. Since attributes depend on geometry, attribute encoding is performed based on the reconstructed geometry.

[0107] A point cloud encoder (e.g., LOD generator 30009) can classify (or reorganize) points according to LOD. The figure shows the point cloud content corresponding to LOD. The leftmost image in the figure represents the original point cloud content. The second image from the left in the figure represents the point distribution in the lowest LOD, and the rightmost image represents the point distribution in the highest LOD. That is, points are sparsely distributed in the lowest LOD and densely distributed in the highest LOD. In other words, as the LOD increases in the direction indicated by the arrow at the bottom of the figure, the space (or distance) between points narrows.

[0108] Figure 6 An example of point configuration for each LOD according to an embodiment is shown.

[0109] For reference Figures 1 to 5 As described, a point cloud content providing system or point cloud encoder (e.g., point cloud video encoder 10002, Figure 3 A point cloud encoder or LOD generator (30009) can generate LODs. LODs are generated by reorganizing points into a set of refined levels based on a set of LOD distance values ​​(or a set of Euclidean distances). The LOD generation process is performed not only by the point cloud encoder but also by the point cloud decoder.

[0110] Figure 6 The upper part shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 6 In this context, the original order represents the order of points P0 to P9 before LOD generation. Figure 6 In this context, LOD-based order represents the order in which points are generated according to their LOD. Points are reorganized by LOD. Additionally, higher LODs include points belonging to lower LODs. For example... Figure 6 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0111] For reference Figure 3 As described, the point cloud encoder according to the embodiments may selectively or in combination perform predictive transform coding, lifting transform coding, and RAHT transform coding.

[0112] The point cloud encoder according to an embodiment can generate predictors for points to perform predictive transformation coding for setting the predictive attributes (or predictive attribute values) of each point. That is, N predictors can be generated for N points. The predictors according to an embodiment can calculate weights (=1 / distance) based on the LOD value of each point, index information of neighboring points existing within a set distance of each LOD, and the distance to the neighboring points.

[0113] According to an embodiment, the predicted attribute (or attribute value) is set as the average of values ​​obtained by multiplying the attributes (or attribute values) of neighboring points (e.g., color, reflectivity, etc.) set in the predictor of each point by a weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 30011) according to an embodiment can quantize and inverse quantize the residual (which may be referred to as residual attribute, residual attribute value, attribute prediction residual, etc.) obtained by subtracting the predicted attribute (or attribute value) of each point from the attribute (attribute value) of each point. Tables 2 and 3 below illustrate the quantization process.

[0114] [Table 2]

[0115] [Table 3]

[0116] When the predictors of each point have neighboring points, the point cloud encoder (e.g., arithmetic encoder 30012) according to the embodiment can perform entropy encoding on the residual values ​​of quantization and inverse quantization as described above. When the predictors of each point do not have neighboring points, the point cloud encoder (e.g., arithmetic encoder 30012) according to the embodiment can perform entropy encoding on the attributes of the corresponding point without performing the above operations.

[0117] The point cloud encoder (e.g., lift transformer 30010) according to an embodiment can generate predictors for each point, set the calculated LOD and register neighboring points in the predictors, and set weights based on the distance to the neighboring points to perform lift transform coding. The lift transform coding according to the embodiment is similar to the predictive transform coding described above, but the difference is that weights are applied cumulatively to the attribute values. The process of cumulatively applying weights to the attribute values ​​according to the embodiment is configured as follows.

[0118] 1) Create an array quantized weights (QW) to store the weight values ​​of each point. The initial value of all elements of QW is 1.0. Multiply the QW value of the predictor index of the neighboring nodes registered in the predictor by the weight of the current point's predictor, and add the values ​​obtained by multiplication.

[0119] 2) Improve the prediction process: Subtract the value obtained by multiplying the attribute value of the point by the weight from the existing attribute value to calculate the predicted attribute value.

[0120] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0121] 4) The weights calculated by multiplying the weights computed for all predictors by the weights stored in the QW corresponding to the predictor index are summed with the updateweight array and used as the index of the neighbor node. The values ​​obtained by multiplying the attribute values ​​of the neighbor node indexes by the calculated weights are summed with the update array.

[0122] 5) Improve the update process: Divide the attribute values ​​of the update array of all predictors by the weight values ​​of the updateweight array of the predictor index, and add the existing attribute values ​​to the values ​​obtained by division.

[0123] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated through the boosting update process by the weight updated through the boosting prediction process (stored in QW). The predicted attribute value is quantized by a point cloud encoder (e.g., coefficient quantizer 30011) according to the embodiment. Additionally, the point cloud encoder (e.g., arithmetic encoder 30012) performs entropy encoding on the quantized attribute value.

[0124] A point cloud encoder according to an embodiment (e.g., RAHT transform 30008) can perform RAHT transform coding, where attributes associated with lower-level nodes in an octree are used to predict attributes of higher-level nodes. RAHT transform coding is an example of intra-frame attribute coding performed by scanning backward through an octree. The point cloud encoder according to an embodiment scans the entire region starting from voxels and repeats a merging process at each step, merging voxels into larger blocks, until the root node is reached. The merging process according to an embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the node directly above an empty node.

[0125] The following equation represents the RAHT transformation matrix. In this equation, Indicates level The average attribute value of the voxels at that location. Based on and To be calculated. Used for and The weight is and .

[0126]

[0127] here, It is a low-pass value and is used in the next higher level of merging. This represents the high-pass coefficient. At each step, the high-pass coefficient is quantized and subjected to entropy encoding (e.g., encoded by an arithmetic encoder 30012). Weights are calculated as follows: .pass and Create the root node as follows.

[0128]

[0129] The value of gDC is also quantized and entropy-encoded, just like the high-pass coefficient.

[0130] Figure 7 A point cloud decoder according to an embodiment is shown.

[0131] Figure 7 The point cloud decoder shown is an example of a point cloud decoder and can perform decoding operations. Figures 1 to 6 The reverse process of the encoding operation of the point cloud encoder is shown.

[0132] For reference Figure 1 and Figure 6 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0133] The point cloud decoder according to the embodiment includes an arithmetic decoder (arithmetic decoding) 7000, an octree synthesizer (synthesized octree) 7001, a surface approximation synthesizer (synthesized surface approximation) 7002, a geometry reconstructor (reconstructed geometry) 7003, an inverse coordinate transformer (inverse coordinate transformation) 7004, an arithmetic decoder (arithmetic decoding) 7005, an inverse quantizer (inverse quantization) 7006, a RAHT transformer 7007, a LOD generator (generated LOD) 7008, an inverse lifter (inverse lift) 7009, and / or a color inverse transformer (inverse color transformation) 7010.

[0134] An arithmetic decoder 7000, an octree synthesizer 7001, a surface approximation synthesizer 7002, a geometry reconstructor 7003, and a coordinate inverse transformer 7004 can perform geometric decoding. Geometric decoding according to embodiments may include direct decoding and triplet geometric decoding. Direct encoding and triplet geometric decoding are selectively applied. Geometric decoding is not limited to the examples described above and is provided as a reference. Figures 1 to 6 The reverse process of the described geometric encoding is executed.

[0135] According to an embodiment, the arithmetic decoder 7000 decodes the received geometric bitstream based on arithmetic coding. The operation of the arithmetic decoder 7000 corresponds to the inverse process of the arithmetic encoder 30004.

[0136] The octree synthesizer 7001 according to an embodiment can generate an octree by obtaining a octet code (or information about the geometry obtained as a decoding result) from the decoded geometry bitstream. The octet code is as shown in the reference... Figures 1 to 6 Please describe that configuration in detail.

[0137] When applying triplet geometry encoding, the surface approximation synthesizer 7002 according to the embodiment can synthesize the surface based on the decoded geometry and / or the generated octree.

[0138] According to an embodiment, the geometry reconstructor 7003 can regenerate geometry based on surface and / or decoded geometry. See also... Figures 1 to 9 As described, direct encoding and triadic geometric encoding are selectively applied. Therefore, the geometry reconstructor 7003 directly imports the positional information of points for which direct encoding has been applied and sums them. When triadic geometric encoding is applied, the geometry reconstructor 7003 can reconstruct the geometry by performing the reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 30005. Details and references Figure 6 The descriptions are the same for all of them, so their descriptions are omitted. The reconstructed geometry may include point cloud images or frames that do not contain attributes.

[0139] According to the embodiment, the inverse coordinate transformer 7004 can obtain the point position based on the reconstructed geometric transformation coordinates.

[0140] Arithmetic decoder 7005, inverse quantizer 7006, RAHT transformer 7007, LOD generator 7008, inverse booster 7009, and / or color inverse transformer 7010 perform attribute decoding. Attribute decoding according to embodiments includes Region Adaptive Hierarchical Transformation (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) decoding, and interpolation-based hierarchical nearest neighbor prediction (boost transform) decoding with update / boost steps. The above three decoding schemes can be used selectively, or a combination of one or more decoding schemes can be used. Attribute decoding according to embodiments is not limited to the examples described above.

[0141] According to the embodiment, the arithmetic decoder 7005 decodes the attribute bitstream through arithmetic encoding.

[0142] According to the embodiment, the inverse quantizer 7006 inversely quantizes information about the decoded attribute bitstream or the attributes obtained as a decoding result, and outputs the inversely quantized attributes (or attribute values). Inverse quantization can be selectively applied based on the attribute encoding of the point cloud encoder.

[0143] According to an embodiment, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 can handle the reconstructed geometry and inverse quantization attributes. As described above, the RAHT transformer 7007, LOD generator 7008, and / or inverse lifter 7009 can selectively perform decoding operations corresponding to the encoding of the point cloud encoder.

[0144] According to the embodiment, the color inverse transformer 7010 performs inverse transformation encoding to inversely transform the color values ​​(or textures) included in the decoded attributes. The operation of the color inverse transformer 7010 can be selectively performed based on the operation of the color transformer 30006 of the point cloud encoder.

[0145] Although not shown in the figure, Figure 7 The elements of the point cloud decoder can be implemented by hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. One or more processors can perform the above-described... Figure 7 The point cloud decoder's components have at least one or more operations and / or functions. Additionally, one or more processors are operable or perform operations for executing... Figure 7 The software program and / or instruction set for the operation and / or function of the elements of the point cloud decoder.

[0146] Figure 8 A transmitting apparatus according to an embodiment is shown.

[0147] Figure 8 The transmitting device shown is Figure 1 The transmitting device 10000 (or Figure 3 Example of a point cloud encoder. Figure 8 The transmitting device shown can perform the same operation as the reference. Figures 1 to 6 The described point cloud encoder includes one or more of the same or similar operations and methods. The transmitting apparatus according to embodiments may include a data input unit 8000, a quantization processor 8001, a voxelization processor 8002, an octree occupancy code generator 8003, a surface model processor 8004, an intra / inter-frame coding processor 8005, an arithmetic encoder 8006, a metadata processor 8007, a color transformation processor 8008, an attribute transformation processor 8009, a prediction / boosting / RAHT transformation processor 8010, an arithmetic encoder 8011, and / or a transmission processor 8012.

[0148] According to an embodiment, the data input unit 8000 receives or acquires point cloud data. The data input unit 8000 can perform operations and / or acquisition methods similar to those of the point cloud video acquisition unit 10001 (or refer to...). Figure 2The described acquisition process (20000) is the same or similar operation and / or acquisition method.

[0149] The data input unit 8000, quantization processor 8001, voxelization processor 8002, octree occupancy code generator 8003, surface model processor 8004, intra / inter-frame coding processor 8005, and arithmetic encoder 8006 perform geometric coding. Geometric coding according to the embodiment and reference... Figures 1 to 9 The geometric codes described are the same or similar, so their detailed descriptions are omitted.

[0150] The quantization processor 8001 according to the embodiment quantizes geometry (e.g., point position values). The operation of the quantization processor 8001 and / or quantization with reference... Figure 3 The operation and / or quantization of the described quantizer 30001 are the same or similar. Details and references Figures 1 to 9 The descriptions are the same.

[0151] According to the embodiment, the voxelization processor 8002 voxels the quantized position values ​​of points. The voxelization processor 8002 can perform operations similar to those described above. Figure 3 The operation and / or voxelization process of the quantizer 30001 described are the same as or similar to the operation and / or process. Details and references Figures 1 to 6 The descriptions are the same.

[0152] According to an embodiment, the octree occupancy code generator 8003 performs octree encoding based on the voxelized positions of points in the octree structure. The octree occupancy code generator 8003 can generate occupancy codes. The octree occupancy code generator 8003 can perform operations similar to those referenced... Figure 3 and Figure 4 The operations and / or methods described are the same as or similar to those of the point cloud encoder (or octree analyzer 30002). Details and references Figures 1 to 6 The descriptions are the same.

[0153] According to an embodiment, the surface model processor 8004 can perform triadic geometry encoding based on a surface model to reconstruct point positions in a specific region (or node) on a voxel basis. The surface model processor 8004 can perform and reference... Figure 3 The operations and / or methods described are the same as or similar to those of the point cloud encoder (e.g., surface approximation analyzer 30003). Details and references are available. Figures 1 to 6 The descriptions are the same.

[0154] According to an embodiment, the intra / inter-frame coding processor 8005 can perform intra / inter-frame coding on point cloud data. The intra / inter-frame coding processor 8005 can perform coding that is the same as or similar to intra / inter-frame coding. According to an embodiment, the intra / inter-frame coding processor 8005 may be included in an arithmetic encoder 8006.

[0155] According to an embodiment, the arithmetic encoder 8006 performs entropy encoding on octrees and / or approximate octrees of point cloud data. For example, the encoding scheme includes arithmetic encoding. The arithmetic encoder 8006 performs the same or similar operations and / or methods as the arithmetic encoder 30004.

[0156] The metadata processor 8007 according to an embodiment processes metadata (e.g., set values) about point cloud data and provides it to necessary processing procedures such as geometric encoding and / or attribute encoding. Additionally, the metadata processor 8007 according to an embodiment can generate and / or process signaling information related to geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be encoded separately from the geometric encoding and / or attribute encoding. The signaling information according to an embodiment can be interleaved.

[0157] Color transformation processor 8008, attribute transformation processor 8009, prediction / boosting / RAHT transformation processor 8010, and arithmetic encoder 8011 perform attribute encoding. Attribute encoding and reference according to the embodiment. Figures 1 to 6 The attribute codes described are the same or similar, so their detailed descriptions are omitted.

[0158] According to an embodiment, a color transformation processor 8008 performs color transformation encoding to transform color values ​​included in attributes. The color transformation processor 8008 may perform color transformation encoding based on reconstructed geometry. The reconstructed geometry and reference... Figures 1 to 9 The description is the same. Furthermore, its execution is the same as the reference. Figure 3 The operation and / or methods of the described color converter 30006 are the same as or similar to those described. Detailed descriptions are omitted.

[0159] According to an embodiment, the attribute transformation processor 8009 performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometric encoding is not performed. The attribute transformation processor 8009 performs transformations with reference to... Figure 3 The operation and / or method of the described attribute transformer 30007 are the same as or similar to those described. Detailed descriptions thereof are omitted. The prediction / boosting / RAHT transformation processor 8010 according to the embodiment can encode the transformed attributes by any one or a combination of RAHT encoding, prediction transformation encoding, and boosting transformation encoding. The prediction / boosting / RAHT transformation processor 8010 performs and references... Figure 3The RAHT transformer 30008, LOD generator 30009, and boost transformer 30010 described herein operate at least one of the same or similar operations. Furthermore, the predictive transform coding, boost transform coding, and RAHT transform coding are similar to those of the reference transformer. Figures 1 to 9 The descriptions are the same, so their detailed descriptions are omitted.

[0160] The arithmetic encoder 8011 according to the embodiment can encode the attributes of the code based on arithmetic encoding. The arithmetic encoder 8011 performs the same or similar operations and / or methods as the arithmetic encoder 30012.

[0161] According to an embodiment, the transmission processor 8012 can transmit individual bitstreams containing encoded geometric and / or encoded attribute or metadata information, or transmit a single bitstream containing encoded geometric and / or encoded attribute and metadata information. When the encoded geometric and / or encoded attribute and metadata information according to an embodiment is configured as a single bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to an embodiment may include signaling information and slice data. The signaling information includes a sequence parameter set (SPS) for sequence-level signaling, a geometric parameter set (GPS) for signaling for geometric information encoding, an attribute parameter set (APS) for signaling for attribute information encoding, and a tile parameter set (TPS) for tile-level signaling. The slice data may include information about one or more slices. A slice according to an embodiment may include a geometric bitstream Geom0. 0 and one or more attribute bitstreams Attr0 0 and Attr1 0 .

[0162] A slice is a series of syntactic elements that represent a point cloud frame, either fully or partially encoded.

[0163] According to an embodiment, the TPS may include information about individual tiles in one or more tiles (e.g., coordinate information and height / size information about the bounding box). The geometric bitstream may include a header and a payload. The header of the geometric bitstream according to an embodiment may include a geom_parameter_set_id, a geom_tile_id, and a geom_slice_id included in the GPS, as well as information about the data contained in the payload. As described above, the metadata processor 8007 according to an embodiment may generate and / or process signaling information and transmit it to the transmission processor 8012. According to an embodiment, the element performing geometry encoding and the element performing attribute encoding may share data / information with each other, as indicated by the dashed lines. The transmission processor 8012 according to an embodiment may perform the same or similar operations and / or transmission methods as the transmitter 10003. Details and References Figure 1 and Figure 2 The descriptions are the same, so their descriptions are omitted.

[0164] Figure 9 An example of a receiving device according to an embodiment is shown.

[0165] Figure 9 The receiving device shown is Figure 1 Example of receiving device 10004. Figure 9 The receiving device shown can perform the same operation as the reference. Figures 1 to 8 The same or similar one or more operations and methods described in the point cloud decoder.

[0166] The receiving apparatus according to an embodiment may include a receiver 9000, a receiving processor 9001, an arithmetic decoder 9002, an octree reconstruction processor based on occupancy codes 9003, a surface model processor (triangle reconstruction, upsampling, voxelization) 9004, an inverse quantization processor 9005, a metadata parser 9006, an arithmetic decoder 9007, an inverse quantization processor 9008, a prediction / boost / RAHT inverse transform processor 9009, a color inverse transform processor 9010, and / or a renderer 9011. Each decoding element according to an embodiment can perform the inverse process of the operation of the corresponding encoding element according to the embodiment.

[0167] Receiver 9000 according to an embodiment receives point cloud data. Receiver 9000 can perform operations related to... Figure 1 The operation and / or receiving method of the receiver 10005 are the same as or similar to those of the receiver. Detailed description omitted.

[0168] According to an embodiment, the receiving processor 9001 can acquire a geometric bitstream and / or an attribute bitstream from the received data. The receiving processor 9001 may be included in the receiver 9000.

[0169] The arithmetic decoder 9002, the octet-based octree reconstruction processor 9003, the surface model processor 9004, and the inverse quantization processor 9005 are capable of performing geometric decoding. Geometric decoding and reference according to the embodiment... Figures 1 to 10 The described geometric decodings are the same or similar, so their detailed descriptions are omitted.

[0170] The arithmetic decoder 9002 according to the embodiment can decode a geometric bitstream based on arithmetic coding. The arithmetic decoder 9002 performs the same or similar operations and / or encodings as the arithmetic decoder 7000.

[0171] According to an embodiment, the octree reconstruction processor 9003 based on occupancy codes can reconstruct an octree by obtaining occupancy codes from the decoded geometric bitstream (or information about the geometry obtained as a decoding result). The octree reconstruction processor 9003 performs the same or similar operations and / or methods as the octree synthesizer 7001 and / or the octree generation method. When applying triad geometry encoding, the surface model processor 9004 according to an embodiment can perform triad geometry decoding and related geometric reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on surface model methods. The surface model processor 9004 performs the same or similar operations as the surface approximation synthesizer 7002 and / or the geometry reconstructor 7003.

[0172] According to the embodiment, the inverse quantization processor 9005 can inverse quantize the decoded geometry.

[0173] Metadata parser 9006 according to an embodiment can parse metadata (e.g., set values) contained in received point cloud data. Metadata parser 9006 can deliver the metadata to geometry decoder and / or attribute decoder. Metadata and reference Figure 8 The metadata described is the same, so its detailed description is omitted.

[0174] Arithmetic decoder 9007, inverse quantization processor 9008, prediction / boost / RAHT inverse transform processor 9009, and color inverse transform processor 9010 perform attribute decoding. Attribute decoding and reference Figures 1 to 8 At least one of the described attributes is decoded the same or similarly, therefore its detailed description is omitted.

[0175] The arithmetic decoder 9007 according to an embodiment can decode the attribute bitstream via arithmetic coding. The arithmetic decoder 9007 can decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 9007 performs the same or similar operations and / or encodings as the arithmetic decoder 7005.

[0176] According to an embodiment, the inverse quantization processor 9008 can reversibly quantize and decode attribute bitstreams. The inverse quantization processor 9008 performs the same or similar operations and / or methods as the inverse quantizer 7006 and / or the inverse quantization method.

[0177] According to an embodiment, the predict / boost / RAHT inverse transform processor 9009 can handle the reconstructed geometric and inverse quantization properties. The predict / boost / RAHT inverse transform processor 9009 performs operations related to... Figure 7 The operation and / or decoding of the RAHT transformer 7007, LOD generator 7008, and / or inverse booster 7009 are the same as or similar to one or more operations and / or decodings. The color inverse transform processor 9010 according to the embodiment performs inverse transform encoding to inverse transform the color values ​​(or textures) included in the decoded attributes. The color inverse transform processor 9010 performs operations similar to... Figure 7 The operation and / or inverse transform encoding of the color inverse transformer 7010 are the same as or similar to the operation and / or inverse transform encoding. The renderer 9011 according to the embodiment can render point cloud data.

[0178] Figure 10 An exemplary structure operable in conjunction with a point cloud data transmission / reception method / apparatus according to an embodiment is shown.

[0179] Figure 10 The structure represents a configuration in which at least one of the following components—server 1060, robot 1010, autonomous vehicle 1020, XR device 1030, smartphone 1040, home appliance 1050, and / or head-mounted display (HMD) 1070—is connected to cloud network 1000. Robot 1010, autonomous vehicle 1020, XR device 1030, smartphone 1040, or home appliance 1050 are referred to as devices. Furthermore, XR device 1030 may correspond to a point cloud data (PCC) device according to an embodiment or be operatively connected to a PCC device.

[0180] Cloud Network 1000 can refer to a network that forms part of or exists within a cloud computing infrastructure. Here, Cloud Network 1000 can be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0181] Server 1060 can be connected via cloud network 1000 to at least one of robot 1010, self-driving vehicle 1020, XR device 1030, smartphone 1040, home appliance 1050 and / or HMD 1070, and can assist at least a portion of the processing of connected devices 1010 to 1070.

[0182] HMD 1070 represents one of the implementation types of the XR device and / or PCC device according to the embodiments. The HMD-type device according to the embodiments includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0183] Hereinafter, various embodiments of the apparatus 1010 to 1050 that apply the above-described technology will be described. Figure 10 The devices 1010 to 1050 shown are operable to be connected to / coupled to the point cloud data transmitting and receiving devices according to the above embodiments.

[0184] <PCC + XR>

[0185] The XR / PCC device 1030 may employ PCC technology and / or XR (AR+VR) technology, and may be implemented as an HMD, a head-up display (HUD) installed in a vehicle, a television, a mobile phone, a smartphone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a stationary robot, or a mobile robot.

[0186] The XR / PCC device 1030 can analyze 3D point cloud data or image data acquired through various sensors or from external devices and generate positional and attribute data about 3D points. Thus, the XR / PCC device 1030 can acquire information about the surrounding space or real-world objects and render and output XR objects. For example, the XR / PCC device 1030 can match an XR object, including auxiliary information about the identified object, with the identified object and output a matched XR object.

[0187] <PCC + XR + Mobile Phone>

[0188] The XR / PCC device 1030 can be implemented as a smartphone 1040 by applying PCC technology.

[0189] The 1040 smartphone can decode and display point cloud content based on PCC technology.

[0190] <PCC + Self-driving + XR>

[0191] The self-driving vehicle 1020 can be realized as a mobile robot, vehicle, unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0192] The self-driving vehicle 1020 employing XR / PCC technology can refer to a self-driving vehicle equipped with means for providing XR images, or a self-driving vehicle serving as a control / interaction target in an XR image. Specifically, as a control / interaction target in an XR image, the self-driving vehicle 1020 can be distinguished from and operatively connected to the XR device 1030.

[0193] The autonomous vehicle 1020, equipped with means for providing XR / PCC images, can acquire sensor information from sensors including cameras and output generated XR / PCC images based on the acquired sensor information. For example, the autonomous vehicle 1020 may have a HUD and output XR / PCC images to it, thereby providing passengers with XR / PCC objects corresponding to real objects or objects presented on a screen.

[0194] When an XR / PCC object is output to a HUD, at least a portion of the XR / PCC object can be output to overlap with the actual object being pointed at by the passenger's eyes. Conversely, when an XR / PCC object is output to a display installed within the autonomous vehicle, at least a portion of the XR / PCC object can be output to overlap with objects on the screen. For example, the autonomous vehicle 1020 can output XR / PCC objects corresponding to objects such as roads, other vehicles, traffic lights, traffic signs, two-wheeled vehicles, pedestrians, and buildings.

[0195] Virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or point cloud compression (PCC) technologies according to the embodiments are applicable to various devices.

[0196] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. AR technology, on the other hand, refers to the technology of displaying virtually created CG images on top of images of real objects. MR technology is similar to AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology differs from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with the same characteristics as real objects. More specifically, an example of MR technology application is holographic services.

[0197] Recently, VR, AR, and MR technologies have often been referred to as scaled reality (XR) technologies rather than being clearly distinguished from each other. Therefore, embodiments of this disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are suitable for such technologies.

[0198] The PCC method / apparatus according to the embodiments can be applied to vehicles that provide autonomous driving services.

[0199] Vehicles providing autonomous driving services connect to the PCC device for wired / wireless communication.

[0200] When the point cloud data (PCC) transmitting / receiving device according to an embodiment is connected to a vehicle for wired / wireless communication, the device can receive / process content data related to AR / VR / PCC services (which may be provided together with autonomous driving services) and transmit it to the vehicle. When the PCC transmitting / receiving device is installed in the vehicle, it can receive / process content data related to AR / VR / PCC services based on user input signals input through a user interface device and provide it to the user. The vehicle or user interface device according to an embodiment can receive user input signals. User input signals according to an embodiment may include signals indicating autonomous driving services.

[0201] The point cloud data transmission method / apparatus according to the embodiments is interpreted as referring to... Figure 1 The transmitting device 10000, the point cloud video encoder 10002, and the transmitter 10003, Figure 2 Get 20000 - Encode 20001 - Send 20002 Figure 3 Point cloud video encoder, Figure 8 The transmitting device Figure 10 The device Figure 25 Encoding methods Figure 27 The transmitting device Figure 40 Terms related to sending methods, etc.

[0202] The point cloud data receiving method / apparatus according to the embodiments is to be interpreted as referring to... Figure 1 The receiving device 10004, receiver 10005, and point cloud video decoder 10006. Figure 2 Sending 20002 - Decoding 20003 - Rendering 20004 Figure 7 Point cloud video decoder, Figure 9 The receiving device Figure 10 The device Figure 26 Decoding methods Figure 28 The receiving device Figure 41 Terminology related to receiving methods, etc.

[0203] The point cloud data transmission and reception method / apparatus according to the embodiments can be simply referred to as the method / apparatus according to the embodiments.

[0204] According to the embodiments, geometric data, geometric information, location information, etc., that constitute point cloud data are interpreted as having the same meaning. Attribute data, attribute information, etc., that constitute point cloud data are interpreted as having the same meaning.

[0205] The method / apparatus according to the embodiments can be considered for scalable transmission to process point cloud data.

[0206] Regarding the method / apparatus according to embodiments, this document discloses a method for effectively supporting selective decoding of partial data based on receiver performance or transmission speed when transmitting / receiving point cloud data. Specifically, this disclosure proposes a technique for increasing the efficiency of scalable coding, wherein an encoder at the transmitting end can selectively deliver information required by a decoder at the receiving end for compressing data, and the decoder can decode it, wherein coding units can be configured as independent slices in the sense of tree level, LOD, layer group unit, etc.

[0207] Specifically, this disclosure proposes a method for increasing the efficiency of scalable coding in point cloud data compression methods. Here, scalable coding is a technique for gradually changing the data resolution based on receiver requests / processing speed / performance / transmission bandwidth, etc., enabling compressed data to be efficiently delivered from the sending side and decoded at the receiving side. To this end, layer-based grouping for efficient delivery of point cloud data can be applied in conjunction with the techniques of this disclosure. Specifically, this paper proposes a method for configuring slice fragments to be more suitable for scalable PCC services and sending / receiving them when direct compression mode is used for location compression. Additionally, compression methods for efficiently storing and transmitting large point cloud data with wide distribution and high point density can be used.

[0208] Reference Figure 3 and Figure 7 The point cloud data transmitting / receiving apparatus (which may be simply referred to as an encoder / decoder) shown in the embodiment comprises a set of points. Each point includes geometric information (or geometry or geometric data) and attribute information (or attributes or attribute data). The geometric information is the three-dimensional position information (x, y, z) of each point. That is, the position of each point is represented by parameters in a coordinate system representing three-dimensional space (e.g., parameters representing the three axes of space (x, y, z), such as the X-axis, Y-axis, and Z-axis). The attribute information represents the point's color (RGB, YUV, etc.), reflectivity, normal vector, transparency, etc. In point cloud compression (PCC), octree-based compression is performed to effectively compress non-uniform distributions in three-dimensional space, and the attribute information is compressed based on octree compression. Figure 3 and Figure 7 The point cloud video encoder and point cloud video decoder shown can perform operations through their respective components according to the embodiments.

[0209] According to an embodiment, the transmitting device compresses geometric information (e.g., location) and attribute information (e.g., color / brightness / reflectivity, etc.) about the point cloud data and sends the compressed information to the receiving device. The point cloud data can be configured according to an octree structure having layers based on the level of detail (LoD). Scalable point cloud data encoding and representation can then be performed based on this configuration. In this case, depending on the performance of the receiving device or the transmission rate, only a portion of the point cloud data can be decoded or represented. However, there is currently no method for pre-removing unnecessary data.

[0210] In other words, when only a portion of the scalable point cloud compressed bitstream needs to be transmitted (e.g., only a portion of the decoding layer in scalable decoding), it is impossible to select and transmit the necessary parts. Therefore, the transmitting device must re-encode the necessary parts after decoding, such as... Figure 11 As shown, or when the entire data is transmitted to the receiving device, the receiving device should selectively apply the necessary data after decoding, such as in Figure 12 As shown in the image.

[0211] However, in Figure 11 In such cases, delays may occur due to the time required for decoding and re-encoding. Figure 12 In such cases, bandwidth efficiency may deteriorate due to the transmission of unnecessary data to the receiving device. Furthermore, when using fixed bandwidth, it may be necessary to reduce data quality for data transmission.

[0212] Therefore, the method / apparatus according to the embodiments can provide slicing, such that the point cloud is divided into regions for processing.

[0213] Specifically, in the case of octree-based positional compression, entropy-based compression and direct compression can be used together. In this regard, this paper proposes a slice configuration for effectively utilizing scalability.

[0214] Furthermore, regarding the method / apparatus according to the embodiments, a sliced ​​structure for point cloud data can be defined, and scalable layers and sliced ​​structures for scalable transmission can be signaled.

[0215] The method / apparatus according to the embodiments can divide and process a bitstream into specific units for efficient bitstream delivery and decoding.

[0216] The method / apparatus according to the embodiments can realize selective transmission and decoding of hierarchical point cloud data in a bitstream unit.

[0217] According to embodiments, units can be referred to as Level of Detail (LOD), layers, slices, etc. LOD is the same term as LOD in attribute data encoding, but in another sense it can refer to a data unit used for a hierarchical structure of bitstreams. An LOD can be a layered structure based on point cloud data, for example, the depth (level) of an octree or multi-way tree corresponds to a concept of one depth or a bundle of two or more depths. Similarly, layers are units provided to generate sub-bitstreams. It is a concept corresponding to a bundle of one depth or two or more depths, and can correspond to one LOD or two or more LODs. Furthermore, a slice is a unit used to configure sub-bitstreams, and can correspond to one depth, a portion of a depth, or two or more depths. Additionally, a slice can correspond to one LOD, a portion of an LOD, or two or more LODs. According to embodiments, LODs, layers, and slices can correspond to each other, or one of LODs, layers, and slices can be included in another. Furthermore, units according to embodiments can include LODs, layers, slices, layer groups, or subgroups, and can be interchangeably referenced. According to embodiments, in an octree structure, layers, depths, levels, and depth levels can have the same meaning.

[0218] Figure 13 This is a diagram illustrating an example of layer-based point cloud data configuration according to an embodiment. Figure 13 The diagram illustrates an example of an octree structure, where the root node's depth level is set to 0 and the leaf nodes' depth level is set to 7.

[0219] The method / apparatus according to the embodiments can be configured as follows: Figure 13 The layer-based point cloud data shown is used for encoding and decoding point cloud data.

[0220] The layering of point cloud data according to the embodiments can depend on the application domain and have a layered structure in terms of SNR, spatial resolution, color, temporal frequency, bit depth, etc., and the layers can be configured in the direction of increasing data density based on octree structure or LOD structure.

[0221] In other words, when generating a Level of Detail (LOD) based on an octree structure, the LOD can be defined as an increase in detail in the direction of increasing detail, i.e., in the direction of increasing octree depth. In this disclosure, a layer can have the same meaning as level, depth, and depth level.

[0222] refer to Figure 13For example, in an octree structure with 7 depth levels other than the root node level (or root level), LOD 0 is configured to include levels from the root node level to octree depth level 4, LOD 1 is configured to include levels from the root node level to octree depth level 5, and LOD 2 is configured to include levels from the root node level to octree depth level 7.

[0223] Figure 13 (a) The diagram illustrates the bitstream structure of the geometric data according to an embodiment, and Figure 13 (b) The diagram illustrates the bitstream structure of the attribute data according to the embodiment.

[0224] The method / apparatus according to the embodiments can be based on, for example... Figure 13 The hierarchical generation of LOD using the octree structure shown is configured as follows: Figure 14 (a) and Figure 14 (b) shows the geometric bitstream and attribute bitstream.

[0225] According to this embodiment, the transmitting device can divide the bit stream obtained by point cloud compression into a geometric bit stream and an attribute bit stream based on the data type in order to transmit the bit stream.

[0226] In this scenario, each bitstream can consist of multiple slices and be transmitted. According to an embodiment, a geometric bitstream (e.g., Figure 14 (a) and attribute bitstreams (e.g., Figure 14 (b) can be configured as a slice and can be delivered regardless of layer information or LoD information. In this case, in order to use only some layers or LoD, operations such as decoding the bitstream, selecting only the parts to be used and removing unnecessary parts, and re-encoding only based on the necessary information should be performed.

[0227] This disclosure proposes a method for dividing a bitstream into layers (or LoD) to avoid such unnecessary intermediate operations.

[0228] Figure 15 This is a diagram illustrating an example of a bitstream configuration for delivering a bitstream divided into layers (or LoD) according to an embodiment.

[0229] For example, in the case of LoD-based PCC technology, lower LoDs are included in higher LoDs. That is, higher LoDs include all points of lower LoDs. Furthermore, when information about points included in the current LoD but not in previous LoDs—that is, new points added to each LoD—is defined as R (remaining or reserved), the transmitting device can divide the initial LoD information and the newly included information R in each LoD into independent units (e.g., slices) and transmit them, such as...Figure 15 As shown in the image.

[0230] In other words, the set of new points added by configuring each LoD compared to the previous LoD can be defined as information R. Figure 15 The illustration shows an example where LoD1 includes LoD0 and information R1, and LoD2 includes LoD1 and information R2.

[0231] According to an embodiment, points sampled for one or more octree depth levels can be identified as data belonging to information R. That is, the set of points sampled for one or more octree depth levels (i.e., points matching the occupying node) can be defined as information R. According to another embodiment, points sampled for one octree depth level can be divided into multiple information Rs according to predetermined criteria. In this case, various criteria for dividing an octree depth level into multiple information Rs can be considered. For example, when dividing an octree depth level into M information Rs, the M information Rs can be configured such that the data in information Rs have consecutive Morton codes, or they can be grouped such that the data in information Rs have the same remainder obtained by dividing the Morton code sequential index by M. Alternatively, when grouped as sibling nodes, the M information Rs can be configured by grouping data at the same position. According to another embodiment, if necessary, some sampled points from multiple octree depth levels can be identified as information R.

[0232] exist Figure 15 In this example, according to the embodiment, both the geometry bitstream and the attribute bitstream are divided into three slices. Each slice includes a header and a payload (also called a data unit) containing the actual data (e.g., geometry data, attribute data). The header may contain information about the corresponding slice. Additionally, the header may further contain reference information related to previous slices, previous LoDs, or previous layers used for LoD configuration.

[0233] For example, refer to Figure 15 The geometric bitstream is divided into slices carrying geometric data belonging to LoD0, slices carrying geometric data belonging to information R1, and slices carrying geometric data belonging to information R2. The attribute bitstream is divided into slices carrying attribute data belonging to LoD0, slices carrying attribute data belonging to information R1, and slices carrying attribute data belonging to information R2.

[0234] The receiving method / apparatus according to the embodiments can receive bit streams divided into LODs or layers, and can efficiently decode only the data to be used without complex intermediate processes.

[0235] In this regard, various implementation methods can be used to send bit streams.

[0236] For example, the geometry bitstream and the attribute bitstream can be delivered separately. Alternatively, the geometry bitstream and the attribute bitstream can be multiplexed into a single bitstream and delivered.

[0237] When each bitstream contains LoD0 and one or more messages R, the delivery order of LoD0 and one or more messages R can vary.

[0238] exist Figure 15 In the example, the geometric bitstream and the attribute bitstream are delivered separately. In this case, LoD0 including the geometric bitstream and two pieces of information R(R1, R2) are delivered sequentially, and LoD0 including the attribute bitstream and two pieces of information R(R1, R2) are delivered sequentially.

[0239] Figure 16 The illustration shows an exemplary bitstream ordering used when multiplexing a geometric bitstream and an attribute bitstream into a single bitstream, according to an embodiment.

[0240] The transmission method / apparatus according to the embodiments can transmit bit streams sequentially, such as... Figure 16 The diagram shows geometric data and attribute data. In this operation, depending on the data type, the entire geometric data (or geometric information) can be sent first, followed by the attribute data (or attribute information). In this case, the geometric data can be quickly reconstructed based on the sent bitstream information.

[0241] For example, refer to Figure 16 The layer containing geometric data (LOD) can be located first in the bitstream, and the layer containing attribute data (LOD) can be located after the geometric layer. Because the attribute data depends on the geometric data, the layer containing the geometric data (LOD) can be located before the layer containing the attribute data (LOD). Figure 16 The illustrated example shows a LoD0 containing geometric data and two pieces of information R(R1, R2) delivered sequentially, followed by a LoD0 containing attribute data and two pieces of information R(R1, R2) delivered sequentially. The positions can be changed in this regard depending on the embodiment. Furthermore, references can be made between geometric headers, and also between attribute headers and geometric headers.

[0242] Figure 17 The illustration shows another exemplary bitstream sorting method according to an embodiment, where the geometric bitstream and the attribute bitstream are multiplexed into a single bitstream.

[0243] When transmitting bit streams, the transmission method / apparatus according to the embodiments can transmit in series such as Figure 17The diagram shows geometric and attribute data. In this case, bitstreams containing geometric and attribute data that constitute the same layer can be bundled and sent. In this case, when compression techniques are used for parallel decoding of geometry and attributes, decoding execution time can be reduced. In this respect, information that requires priority can be processed first (low LoD and geometry should take precedence over attributes).

[0244] Figure 17 The illustration shows an example of sending LoD0 containing geometric data, LoD0 containing attribute data, information R1 containing geometric data, information R1 containing attribute data, information R2 containing geometric data, and information R2 containing attribute data in this order. In this case, these positions can be adjusted according to the embodiment. Furthermore, references can be made between the geometric headers, and also between the attribute header and the geometric header.

[0245] The transmitting / receiving method / apparatus according to the embodiments can effectively select the layer (or LoD) required by the application domain at the bitstream level when transmitting and receiving bitstreams. In the bitstream ordering method according to the embodiments, when such as Figure 1 When the geometric information shown is grouped and transmitted, blank sections may appear in the middle after selecting the bitstream level. In this case, the bitstream may need to be reprocessed.

[0246] By according to Figure 17 The layers shown are used to bundle and transmit geometric and attribute data, and can selectively deliver necessary information and / or selectively remove unnecessary information based on the application domain, such as... Figure 18 (a) to Figure 18 (c) or Figure 19 (a) to Figure 19 As shown in (c).

[0247] Figure 18 (a) to Figure 18 (c) is a diagram illustrating an example of the selection of symmetric geometric properties according to an embodiment.

[0248] For example, refer to Figure 18 (a) to Figure 18 (c) When it is necessary to select a portion of the bitstream according to the embodiment, the transmitting device selects and transmits only the data up to LoD1 (i.e., LoD0+R1), and removes the information R2 corresponding to the upper layer (i.e., the new portion of LoD2) from the bitstream and does not transmit it. In the case of symmetric geometric attribute selection, the geometric data and attribute data of the same layer are selected and transmitted simultaneously, or selected and removed simultaneously.

[0249] Figure 19 (a) to Figure 19(c) is a diagram illustrating an example of asymmetric geometric attribute selection according to an embodiment. In the case of asymmetric geometric attribute selection, only one of the geometric data and attribute data of the same layer is selected and sent or removed.

[0250] For example, refer to Figure 19 (a) to Figure 19 (c) When it is necessary to select a portion of the bitstream according to the embodiment, the transmitting device selects and transmits LoD1 (LoD0 + R1) containing geometric data and attribute data, LoD1 (LoD0 + R1) containing attribute data, and R2 containing geometric data, and removes R2 containing attribute data from the bitstream so that it is not transmitted. In other words, for attribute data, data other than the data in the upper layer (R2) is selected and transmitted. For geometric data, data from all layers (from level 0 (root level) to level 7 (leaf level) in the octree structure) are transmitted.

[0251] When it is necessary to select a portion of the bitstream according to an implementation example, the following can be used: Figure 18 (a) to Figure 18 (c) Method for selecting symmetric geometric attributes Figure 19 (a) to Figure 19 (c) an asymmetric geometric attribute selection method, or a combination of asymmetric and asymmetric geometric attribute selection methods, to select a portion of the bit stream.

[0252] The segmentation of the bitstream and the selection of bitstream portions mentioned above are designed to support the scalability of point cloud data.

[0253] When point cloud data is represented in an octree structure and divided into LODs (or layers), scalable encoding / decoding (scalability) can be supported.

[0254] The scalability features according to the embodiments may include slice-level scalability and / or octree-level scalability.

[0255] According to the embodiments, LoD can be used as a unit to indicate a set of one or more octree layers. Furthermore, LoD can refer to a bundle of octree layers to be configured on a slice-by-slice basis.

[0256] The LOD according to the embodiment can be used in a broad sense, such as for units that are used to divide data in detail, beyond the meaning of LOD in attribute encoding / decoding.

[0257] In other words, spatial scalability can be provided for each octree layer, which is the actual octree layer (or scalable attribute layer). However, when scalability is configured at the slice level before bitstream parsing, the choice can be made at the LoD level.

[0258] For example, refer to Figure 13In an octree structure, levels from the root level to level 4 correspond to LoD0, and levels from the root level to level 5 correspond to LoD1. Furthermore, levels from the root level to level 8 (i.e., the leaf level) correspond to LoD2.

[0259] In other words, in Figure 13 In the example, when scalability is used at the slice level, the provided scalable operations correspond to three steps: LoD0, LoD1, and LoD2, and the octree structure can provide scalable operations in the decoding operation corresponding to eight steps from the root level to the leaf level.

[0260] According to an embodiment, when LoD0 to LoD2 are composed of their respective slices, the transcoder of the receiver or transmitter (see [example]) [[ID= You can select only LoD0, only LoD1, or LoD2 for scalable processing. ​ In this context, LoD1 includes LoD0, and LOD2 includes both LoD1 and LoD2.

[0261] For example, when only LoD0 is selected, the maximum octree level is 4, and a scalable layer can be selected from octree levels 0 to 4 during the decoding operation. In this case, the receiving device can treat the node size obtainable through the maximum octree level (or depth) as a leaf node and send that node size via signaling information.

[0262] For example, when LoD1 is selected, layer 5 can be added. Therefore, the maximum octree level can be 5, and a scalable layer can be selected from octree layers 0 to 5 during the decoding operation. In this case, the receiving device can treat the node size obtainable through the maximum octree level (or depth) as a leaf node and can transmit that node size via signaling information. According to embodiments, octree depth, octree layers, and octree levels can be units where data is meticulously divided.

[0263] For example, when LoD2 is selected, layers 6 and 7 can be added. Therefore, the maximum octree level can be 7, and a scalable layer can be selected from octree layers 0 to 7 during the decoding operation. In this case, the receiving device can treat the node size available through the maximum octree level (or depth) as a leaf node and can transmit that node size via signaling information.

[0264] ​ (a) to ​ (c) The illustration shows an exemplary method of slicing point cloud data according to an embodiment.

[0265] According to the embodiments, the transmission method / apparatus / encoder can configure the G-PCC bitstream by segmenting the bitstream in a slice structure. The data unit used for detailed data representation can be a slice.

[0266] For example, one or more octree layers (or depths) can be matched with a slice.

[0267] According to the transmission method / apparatus of the embodiment, such as an encoder, a bit stream can be configured based on slice 21001 by scanning the nodes (points) included in the octree in the direction of scan sequence 21000. A slice may include nodes at one or more levels in the octree structure, may include only nodes at a specific level, or may include only some nodes at a specific level. Alternatively, it may include only some nodes at one or more levels.

[0268] ​ - (a) The diagram illustrates an exemplary octree structure consisting of 7 slices. In this example, slice 21002 may include nodes from level 0 to level 4, and slice 21003 may include some nodes from level 5. Slice 21004 may include some other nodes from level 5, and slice 21005 may include some other nodes from level 5. That is, in ​ In (a), level 5 is divided into three slices. Similarly, in ​ In (a), level 6 (i.e., the leaf level) is also divided into three slices. In other words, a slice can consist of some nodes at a specific level.

[0269] ​ (b) The diagram illustrates an exemplary octree structure consisting of four slices. In this example, one slice includes nodes from level 0 to level 3 and some nodes from level 4, and another slice includes other nodes from level 4 and some nodes from level 5. Additionally, another slice includes other nodes from level 5 and some nodes from level 6, and yet another slice includes other nodes from level 6.

[0270] ​ (c) The diagram illustrates an exemplary octree structure consisting of five slices. One slice consists of nodes from level 0 to level 3, and four slices consist of nodes from level 4 to level 6. That is, a slice includes some nodes of level 4, some nodes of level 5, and some nodes of level 6. In other words, at levels 4 to 6, a slice may include some data of level 4 and data of level 5 or level 6 corresponding to the child nodes of that data.

[0271] In other words, such as ​ (b) and ​As shown in (c), when multiple octree layers match a slice, only some nodes from each layer can be included in the slice. When multiple slices form a geometry / attribute frame in this manner, information necessary for the receiving device to configure the layers can be sent to the receiving device via signaling information. For example, the signaling information may include information about the layers included in each slice and information about the nodes included in each layer.

[0272] The encoder and the corresponding device according to this embodiment can encode point cloud data and generate and transmit a bit stream containing the encoded data and signaling information (or parameter information) related to the point cloud data.

[0273] Furthermore, when generating the bitstream, it can be based on the bitstream structure according to the embodiment (e.g., see...). ​ (etc.) generate a bitstream. Therefore, the receiving device, decoder, corresponding device, etc. according to the embodiment can receive and parse the bitstream configured to selectively decode some data, thereby decoding and effectively providing only a portion of the point cloud data.

[0274] Next, we will describe the scalable transfer of point cloud data.

[0275] The point cloud data transmission method / apparatus according to the embodiments can scalably transmit a bit stream containing point cloud data, and the point cloud data receiving method / apparatus according to the embodiments can scalably receive the bit stream and decode it.

[0276] When have ​ When the bitstream structure illustrated in the diagram is used for scalable transmission, signaling information for selecting the desired slice for the receiving device can be sent to the receiving device. Scalable transmission does not necessarily mean sending or decoding the entire bitstream, but rather sending or decoding only a portion of the bitstream. Therefore, low-resolution point cloud data can be provided.

[0277] When scalable transmission is applied to an octree-based geometric bitstream according to an embodiment, for each octree layer from the root node to the leaf node ( ​ The bitstream should allow configuration of point cloud data based solely on information about the layers up to a specific octree layer.

[0278] Therefore, the target octree layer should not have any dependency on information from lower octree layers. This can be a constraint that is applied jointly to geometric encoding / attribute encoding.

[0279] Additionally, in scalable transmission, the scalable structure used by the transmitting / receiving device to select scalable layers needs to be sent to the receiving device. Considering the octree structure according to the embodiment, all octree layers can support scalable transmission, or scalable transmission can be allowed only for specific octree layers and lower layers. For example, when some octree layers are included, signaling information can be delivered to the receiving device to indicate that a scalable layer including a slice is included. Therefore, the receiving device can determine whether the slice is necessary / unnecessary during the bitstream stage. ​ In the example of (a), levels 0 (i.e., the root level) through 21002 can constitute a scalable layer that does not support scalable transport, and lower octree levels can be matched with the scalable layer in a one-to-one correspondence. Typically, scalability can be supported for the portions corresponding to leaf nodes. ​ As shown in (c), when a slice includes multiple octree layers, these layers can be defined as being configured as a scalable layer.

[0280] In this context, scalable transmission and scalable decoding can be used individually, depending on the purpose. According to an embodiment, scalable transmission can be used to allow transmitting / receiving devices to select information up to a specific layer without involving a decoder. According to an embodiment, scalable decoding can be used to select a specific layer during encoding. That is, scalable transmission can support the selection of necessary information in a compressed state (i.e., during the bitstream stage) without involving a decoder, allowing the transmitting or receiving device to determine a specific layer. On the other hand, in the case of scalable decoding, encoding / decoding can be supported only for information up to the necessary portion during the encoding / decoding process. Therefore, scalable decoding can be used in operations such as scalable representation.

[0281] In this context, the layer configuration for scalable transport may differ from the layer configuration for scalable decoding. For example, for scalable transport, the lower three octree layers, including leaf nodes, can constitute a single layer. On the other hand, for scalable decoding, when all layer information is included, scalable decoding can be enabled for each of the leaf node layer, leaf node layer-1, and leaf node layer-2.

[0282] ​ (a) and ​ (b) The illustration shows the geometric tree structure based on a single slice and a segmented slice according to the embodiments.

[0283] The method / apparatus according to the embodiments can be configured for delivering slices of point cloud data, such as... ​ (a) and ​ The diagram in (b) is shown.

[0284] ​ (a) and ​(b) illustrates the geometric tree structure included in different slice structures. According to G-PCC technology, the entire encoded bitstream can be contained within a single slice. Furthermore, for multiple slices, each slice can contain sub-bitstreams. The order of the slices can be the same as the order of the sub-bitstreams. Bitstreams can be accumulated in width-first order of the geometric tree, and each slice can be matched with a set of tree layers ( ​ (a) and ​ (b)). Segmented slices can inherit the hierarchical structure of the G-PCC bitstream.

[0285] Just as the upper layers of a geometric tree do not affect the lower layers, subsequent slices can remain unaffected by previous slices.

[0286] The segmented slicing according to the embodiments is effective in terms of error robustness, efficient transmission, and support for regions of interest.

[0287] 1) Error Recovery

[0288] Compared to a single-slice structure, segmented slices may be more robust to errors. When a slice contains the entire bitstream of a frame, data loss can affect the entire frame. On the other hand, when the bitstream is segmented into multiple slices, even if some slices are lost, some slices that are unaffected by the loss can still be decoded.

[0289] 2) Scalable transmission

[0290] Supporting multiple decoders with different capabilities could be considered. When the encoded data is contained in a single slice, the LOD of the encoded point cloud can be determined before encoding. Therefore, multiple pre-coded bitstreams of point cloud data with different resolutions can be sent independently, which may be inefficient in terms of bandwidth or storage space.

[0291] When a PCC bitstream is generated and included in segmented slices, a single bitstream can support different levels of decoders. From the decoder's perspective, the receiver can select the target layer and deliver a portion of the selected bitstream to the decoder. Similarly, by using a single PCC bitstream without partitioning the entire bitstream, a partial PCC bitstream can be efficiently generated on the transmitter side.

[0292] 3) Region-based spatial scalability

[0293] Regarding G-PCC requirements, region-based spatial scalability can be defined as follows: A compressed bitstream can be configured to have one or more layers. Specific regions of interest can utilize additional layers with higher density, and layers can be predicted from lower layers.

[0294] To support this requirement, it is necessary to support different levels of detail representation for each region. For example, in VR / AR applications, it is desirable to represent distant objects with lower precision and nearby objects with higher precision. Alternatively, the decoder can increase the resolution of the region of interest upon request. This operation can be implemented using a scalable structure such as a geometric octree and a scalable attribute coding scheme like G-PCC. The decoder should access the entire bitstream based on the current slice structure containing the entire geometry or attributes, which may lead to inefficiencies in bandwidth, memory, and decoder performance. On the other hand, if the bitstream is segmented into multiple slices, and each slice contains a sub-bitstream according to a scalable layer, the decoder according to the embodiment can efficiently select slices as needed before parsing the bitstream.

[0295] ​ (a) and ​ (b) The illustration shows the layer group structure of the geometric coding tree and the aligned layer group structure of the attribute coding tree according to the embodiment.

[0296] ​ (a) and ​ (b) The illustration shows the layer group structure of the geometric coding tree and the aligned layer group structure of the attribute coding tree according to the embodiment.

[0297] The method / apparatus according to the embodiments can be used as follows: ​ (a) and ​ (b) uses the hierarchical or tree structure of the point cloud data to generate slice layer groups.

[0298] The method / apparatus according to the embodiments can apply segmentation of geometry and attribute bitstreams contained in different slices. Additionally, regarding tree depth, a encoded tree structure using slices included in partial tree information and geometry and attribute encoding can be used.

[0299] refer to ​ (a) illustrates the geometric tree structure and an example of the proposed slice fragment.

[0300] For example, an octree can have eight levels (levels 0 to 7), and five slices can be used to contain sub-bitstreams of one or more levels. A group represents a set of geometric tree levels. For example, group 1 can include levels 0 to 4, group 2 can include level 5, and group 3 can include levels 6 and 7. Additionally, a group can be divided into three subgroups. Parent and child pairs exist in each subgroup. Groups 3-1 to 3-3 are subgroups of group 3. When using scalable attribute encoding, the tree structure can be the same as the geometric tree structure. The same octree-slice mapping can be used to generate attribute slice fragments ( ​ (b)).

[0301] Layer group: refers to the grouping unit of the layer structure generated in G-PCC encoding, such as an octree layer or a LOD layer.

[0302] Subgroups: These can be represented as sets of neighboring nodes based on location information for a single layer group. Alternatively, they can be formed based on the lowest layer within the layer group (which may refer to the layer closest to the root, e.g., layer 6 in group 3), or they can be formed as a group of neighboring nodes according to Morton code order, distance, or coding order. Furthermore, nodes with parent-child relationships can be defined as existing within the same subgroup.

[0303] When defining subgroups, boundaries can appear in the middle of a layer. Whether continuity is maintained at such boundaries can be signaled using `sps_entropy_continuation_enabled_flag` and `gsh_entropy_continuation_flag` to indicate whether entropy is used continuously, and continuity with previous slices can be maintained by signaling `ref_slice_id`.

[0304] The tree structure of the geometry according to the embodiments may be an octree structure, and the attribute layer structure or attribute encoding tree according to the embodiments may include a level of detail (LOD) structure. In other words, the tree structure for point cloud data may include layers corresponding to depth or level, and these layers may be grouped.

[0305] The method / apparatus according to the embodiments (e.g., ​ Octree analyzer 30002 or LOD generator 30009 ​ The octree synthesizer 7001 or LOD generator 7008 can generate octree structures of geometry or LOD tree structures of attributes. Additionally, it can be based on, for example... ​ and ​ The tree structure shown in the figure groups the point cloud data into layers.

[0306] refer to ​ (a) and ​ (b) Multiple layers are grouped to form Group 1 through Group 3. Individual groups can be further subdivided to form subgroups. ​ (a) and ​ (b) The third group in the diagram is divided into three subgroups.

[0307] The method / apparatus according to the embodiments can generate geometry-based slices and attribute-based slice layers.

[0308] The attribute coding layer can have a different structure than the geometric coding tree.

[0309] To effectively utilize the hierarchical structure of G-PCC, segments of slices can be provided that are paired with the geometry and attribute hierarchical structure.

[0310] For a geometric slice fragment, each slice fragment can contain encoded data from a layer group. Here, a layer group is defined as a set of consecutive tree layers. The start depth and end depth of a tree layer can be specific numbers within the tree depth, where the number for the start is less than the number for the end.

[0311] For attribute slice fragments, each slice fragment can contain encoded data from the layer group. Here, depending on the attribute encoding scheme, the layer can be a tree depth or a Level of Detail (LOD).

[0312] The order of coded data within a slice fragment can be the same as the order of coded data within a single slice.

[0313] The following can be provided as a set of parameters contained in the bitstream.

[0314] ​ The diagram illustrates the layer and subgroup structure according to an embodiment.

[0315] refer to ​ Point cloud data and bit streams can be distinguished and represented by bounding boxes.

[0316] refer to ​ The diagram illustrates the subgroup structure and the corresponding bounding boxes for each subgroup. Layer group 2 is divided into two subgroups (group 2-1 and group 2-2), which are included in different slices, and layer group 3 is divided into four subgroups (group 3-1, group 3-2, group 3-3, and group 3-4), which are included in different slices. Given the layer groups and subgroup slices, as well as the bounding box information, spatial access can be performed by 1) comparing the bounding box of each slice with the ROI, 2) selecting slices where the subgroup bounding boxes overlap with the ROIs, and 3) decoding the selected slices.

[0317] When considering the ROI in region 3-3, slices 1, 3, and 6 are selected as layer group 1 covering the ROI, and the subgroup bounding boxes of subgroups 2-2 and 3-3. For efficient spatial access, it is assumed that there are no dependencies between subgroups from the same layer group. In live or low-latency use cases, time efficiency can be increased by performing selection and decoding upon receiving each slice fragment.

[0318] The method / apparatus according to the embodiments can present data as a tree 22000 consisting of layers (which may be referred to as depth, level, etc.) during geometric and / or attribute encoding. Point cloud data corresponding to each layer (depth / level) can be grouped into layer groups (or subgroups). For example, four layers can be grouped to form layer group 1 (22001). Layer group 2 (22002) can be further divided (segmented) into two subgroups, and layer group 3 (22003) can be further divided (segmented) into four subgroups. Each subgroup can be configured to slice to generate a bitstream.

[0319] The receiving apparatus according to an embodiment can receive a bitstream, select a specific slice from the bitstream, and decode the bounding boxes corresponding to the subgroups included in the selected slice. For example, when slice 1 is selected, the bounding box 22004 corresponding to layer group 1 can be decoded. Layer group 1 may be data corresponding to the largest region. When additionally displaying detailed regions for layer group 1, the method / apparatus according to an embodiment can select slice 3 and / or slice 6, and hierarchically and partially access the bounding boxes (point cloud data) of subgroups 2-2 and / or subgroups 3-3 for detailed regions included in the region of layer group 1.

[0320] use ​ Encoding and decoding of point cloud data of layers and subgroups can be performed by at least one of the following: ​ Transmitting / receiving devices ​ Encoding and decoding, ​ Transmission device / method ​ Receiving device / method ​ and ​ Transmitting / receiving device / method ​ The device ​ Encoding device, ​ or ​ Encoding methods ​ Decoding device or ​ Decoding method.

[0321] As described above, according to this disclosure, the geometric bitstream or attribute bitstream in a layer group and / or subgroup can be divided into slices, allowing for efficient compression and reconstruction of geometric or attribute information on a slice-by-slice basis. In such a case, the continuity of context reference can be used as a method to reduce coding efficiency loss.

[0322] In other words, the context table used in encoding one slice can also be used in encoding another slice. The context table is based on the correlations between nodes existing within a single geometry tree and can be used to improve encoding efficiency. In this regard, as a method for considering local correlations between layer groups, context reference relationships can be established only when the subgroup bounding box of the reference slice contains or is identical to the subgroup bounding box of the slice being referenced. In other words, encoding efficiency can be further improved by using the context table of slices representing parent-child / child-subgroup relationships or ancestor-child / child-subgroup relationships. Alternatively, as a method to reduce the burden on the buffer storing the context table, the context table of the initial slice can be used to encode subsequent slices.

[0323] Next, an embodiment of applying the continuity of context reference in the early-presented standard is described. To utilize the continuity of context reference information, a layer group index and a subgroup index can be determined for each node, and the reference context corresponding to each index can be used as the initial value of the context table at the beginning of each subgroup. During encoding, nodes can exist regardless of the order of the subgroups. By storing and loading the encoder context state in a buffer, it can be ensured that the context table is used continuously within a subgroup.

[0324] This process involves two operations: when the depth changes, determining whether the layer group changes, and performing different operations accordingly.

[0325] The relationships between parent and child nodes in a tree structure (e.g., an octree or a Level of Detail) according to an embodiment can be represented as connections between nodes at higher levels and nodes at lower levels. The transmitting / receiving apparatus / method according to an embodiment can generate a tree structure and group point cloud data into multiple groups based on layers of the tree structure. These groups can be layer groups, subgroups, etc., and can correspond to slices according to an embodiment.

[0326] When encoding / decoding point cloud data belonging to a layer group, subgroup, or slice, the transmitting / receiving apparatus / method according to the embodiment can load the context (or context information) stored in the buffer based on a reference layer group, reference subgroup, or reference slice, and perform encoding / decoding. In such a case, the reference layer group, reference subgroup, or reference slice may include a node corresponding to the parent of the node belonging to the layer group, subgroup, or slice to be encoded / decoded.

[0327] In other words, the point cloud data transmitting / receiving apparatus / method according to the embodiment can encode / decode the second group based on context information associated with the first group, and store the context information associated with the second group in a buffer. The stored context information associated with the second group can be used to encode / decode the third group. In this case, the first to third groups can be groups formed based on any layer. Furthermore, the first group can correspond to the parent layer of the second group.

[0328] Additionally, the point cloud data sending / receiving apparatus / method according to the embodiments can store the context of a parent layer group, parent-child group, or parent slice in a buffer, and use the stored context when encoding / decoding a sub-layer group, sub-sub-group, or sub-slice. The parent group or slice can be located at a higher level in the tree structure than the sub-group or slice, and nodes belonging to the parent group (layer group, sub-group) / slice can have a parent-child relationship with nodes belonging to the sub-group or slice.

[0329] ​ The illustration shows an example of contextual references between layer groups according to an embodiment. Specifically, ​ The illustration shows an example of a fixed context reference, where the current child group references the parent and child groups.

[0330] refer to ​ According to an embodiment, a fine-grained slice (FGS) can represent a subgroup or slice, and a subgroup or slice can be a grouping of point cloud data.

[0331] exist ​ In the diagram, FGS 1 (1,0) represents subgroup 0 of layer group 1, and FGS 2 (1,1) represents subgroup 1 of layer group 1. FGS N+1 (2,0) represents subgroup 0 of layer group 2, and FGS N+2 (2,1) represents subgroup 1 of layer group 2.

[0332] exist ​ In this context, "Save State" indicates the encoding or decoding of the corresponding subgroup and stores the context information, while "Refer to Context" indicates referencing the stored context information to encode or decode the corresponding subgroup.

[0333] Therefore, in ​ The diagram illustrates that FGS 1 references context information related to FGS 0 (23001), and FGS N+1 (23003) references context information related to FGS 1 (23002). For example... ​ As shown, a subgroup (or slice) belonging to layer group 2 can refer to the context of a subgroup (or slice) belonging to layer group 1. Additionally, a subgroup belonging to layer group 1 can be a parent-child group of a subgroup belonging to layer group 2.

[0334] ​

[0335] ​ The diagram illustrates a context reference structure for layer group slicing according to an embodiment. In the diagram, an FGS can correspond to a subgroup according to the embodiment, and subgroups arranged in the same row are considered to belong to the same layer group. A subgroup can correspond to a slice. In the diagram, arrows pointing from one slice to another indicate the context reference relationship between two slices. The context reference of the current slice can be one of the slices decoded before the current slice.

[0336] Considering embodiments of spatial random access, referencing parent-child groups can be a good choice to ensure independence within child groups. However, as the number of child groups or layer groups increases, the number of context buffers may also increase. When the number of context buffers is considered to be the number of referenced slices, the number of context buffers can be the sum of all child groups except those belonging to the first and last layer groups. This can be formulated as follows, where N represents the number.

[0337] N （上下文缓冲区） =1+N 子组 ×(N （层组） -2)

[0338] ​ The illustration shows an example of contextual references between groups according to an embodiment. Specifically, ​ The illustration shows an example of flexible contextual references, where all subgroup references are associated with contextual information related to the root subgroup (i.e., the root slice).

[0339] exist ​ In this context, FGS 1 (24002) to FGS N and FGS N+1 (24003) to FGS 2N+1 refer to the context information associated with FGS 0 (24001). That is, all subgroups (or slices) except FGS 0 (24001) refer to FGS 0 (24001).

[0340] One method for reducing the size of the context buffer, according to an embodiment, is to reduce the number of subgroups referenced by subsequent slices. An extreme case of this method is referencing, such as... ​ The root slice is shown. This method, based on the aforementioned number of context buffers, results in a single context buffer because the number of referenced subgroup slices is zero.

[0341] N （上下文缓冲区） =1

[0342] In other words, all slices reference the first slice. ​ Compared to the previous case, the number of stored context states is reduced to one, and all dependent slices reference the first slice.

[0343] ​ (a) to ​ (c) The illustration shows an example of context buffer management according to the embodiment, particularly in the case of a reference parent-child group.

[0344] refer to ​ (a) The transmitting / receiving device according to the embodiment processes FGS 0 (0,0) (25001) and stores the context information (25002) associated with the corresponding group (FGS 0) in the context buffer.

[0345] refer to ​ (b) In order to process (encode or decode) FGS 2 (1,1) (25003), the transmitting / receiving device according to the embodiment may load the context information 25004 (to the loading state) related to FGS 0 (0,0) stored in the context buffer, process FGS 2 (1,1), and store the context information related to FGS 2 (1,1) in the context buffer (to the saving state). In this case, the loading of the context information can be performed by referring to the parameters ref_layer_group_id and ref_subgroup_id.

[0346] refer to ​ (c) In order to process (encode or decode) FGS N+2(2,1) (25005), the transmitting / receiving device according to the embodiment may load context information (context state (1,1)) 25006 associated with FGS 2(1,1). Because FGS N+2(2,1) belongs to the last layer group, the context information associated with it is not stored in the context buffer.

[0347] As mentioned above, in the case of the parent-child group reference method, the reference... ​ (a) to ​ (c) Describe the changes in the context buffer. When the sliced ​​bitstream is decoded, the context state is stored in the context buffer, such as... ​ As shown in (a). When using context inheritance, the context state of the subsequent slice is initialized by one of the stored context states of the previous slice, indicated by ref_layer_group_id and ref_subgroup_id. ​ (b) Therefore, the context state of a slice belonging to the last layer group is initialized by the stored context state of the previous slice, such as ​ As shown in (c). However, when there is a restriction that subgroups belonging to the same layer group should not be referenced, the receiving device (or decoder) according to the embodiment can determine not to store the context of FGS N+1 to FGS 2N. Based on prior information, the smart decoder can save context buffer space.

[0348] ​ (a) to ​ (c) The illustration shows an example of context buffer management according to the embodiment, particularly in the case of the reference root group.

[0349] refer to ​ (a) The transmitting / receiving device according to the embodiment processes FGS 0 (0,0) (26001) and stores the context information (26002) associated with the corresponding group (FGS0) in the context buffer.

[0350] refer to ​ (b) In order to process (encode or decode) FGS 2 (1,1) (26003), the transmitting / receiving device according to the embodiment may load the context information 26004 related to FGS 0 stored in the context buffer (to a loaded state), process FGS 2 (1,1), and store the context information related to FGS 2 (1,1) in the context buffer (to a saved state). In this case, the loading of the context information can be performed by referring to the parameters ref_layer_group_id and ref_subgroup_id.

[0351] refer to ​ (c) In order to process (encode or decode) FGS N+2 (2,1) (26005), the transmitting / receiving device according to the embodiment may load context information (context state (0,0)) (26006) associated with FGS 0 (0,0) (26001). Because FGS N+2 (2,1) (26005) belongs to the last layer group, the context information associated with it is not stored in the context buffer.

[0352] When flexible context referencing is allowed, there may be context states that are not used by subsequent slices. For example, consider a root layer group reference where all context states dependent on a slice are initialized based on the context states stored in the first slice. The decoder may be unaware of the entire reference structure and may potentially use the current context of subsequent slices. However, as expected, the stored context states (1,0) to (1,N-1) are not used in any slice. In this example, the inefficiency is caused by the lack of information on the decoder side.

[0353] ​ (a) to ​ (c) Illustration of an example of context buffer management according to an embodiment.

[0354] refer to ​(a) The transmitting / receiving device according to the embodiment processes FGS 0 (0,0) (27001) and stores the context information (27002) associated with the corresponding group (FGS 0) in the context buffer.

[0355] refer to ​ (b) In order to process (encode or decode) FGS 2 (1,1) (27003), the transmitting / receiving apparatus / method according to the embodiment can load context information (context state (0,0)) (27004) related to FGS 0 (0,0) (27001) from the context buffer. Then, based on the context_reference_indication_flag information, the context information related to FGS 2 (1,1) can be stored in the context buffer. That is, depending on the context_reference_indication_flag information, the context information related to FGS 2 (1,1) may or may not be stored in the buffer. The context_reference_indication_flag information can be generated and transmitted by the transmitting apparatus / method according to the embodiment, and used by the receiving apparatus / method according to the embodiment.

[0356] refer to ​ (c) In order to process (encode or decode) FGS N+2 (2,1) (27005), the transmitting / receiving device / method according to the embodiment may load context information (context state (0,0)) (27006) related to FGS 0 (0,0) (27001) from the context buffer.

[0357] Because the context buffer always stores only context information related to FGS 0, the transmitting / receiving apparatus according to the embodiment can effectively use the buffer memory.

[0358] To increase the efficiency of context buffer management, the transmitting / receiving device according to the embodiment proposes a new signal to indicate whether to use the current context of the following slice, and assists the decoder in determining whether to store the context.

[0359] According to an embodiment, a followed slice may represent a slice referenced by other slices. A subsequent following slice according to an embodiment may represent a slice referencing another slice.

[0360] To improve context buffer management on the decoder side, context reference information from the perspective of the current slice is proposed. That is, an indication of the context state reference is proposed by the following slice reference.

[0361] ​ (b) Illustration of an example of buffer management using the proposed signal. Compared to the previous example which did not provide additional information, the context state is determined to be stored in the context buffer via the context_reference_indication_flag information. When the context reference indicator is on, the decoder stores the current context state in the context buffer to allow the context state to be used by the following slice. On the other hand, when the context reference indicator is off, the decoder does not store the current context state in the context buffer to preserve context memory. Comparison ​ (c) and ​ (c) Compared to the context buffer in the parent-child group reference case, the context buffer in the root reference case has only one context. The size of the stored context memory is N units, where N represents the number of child groups.

[0362] To prevent the degradation in encoding efficiency caused by slicing, the receiver can use inter-slice context, phiBuffer, planar context, and buffer continuity. Additionally, the receiver can determine whether to store the context, phiBuffer, and planar context based on whether the context needs to be reused.

[0363] if (_dep_gbh.context_reuse_flag) {

[0364] _refIdxToSavedArrayIdx[curLayerGroup][_dep_gbh.subgroup_id] = _ctxtMemSaved.size();

[0365] _ctxtMemSaved.push_back(cur_ctxtMem);

[0366] if(_gps->geom_angular_mode_enabled_flag)

[0367] _phiBufferSaved.push_back(cur_phiBuffer);

[0368] if(_gps->geom_planar_mode_enabled_flag)

[0369] _planarSaved.push_back(cur_planar);

[0370] int idx = _refIdxToSavedArrayIdx[curLayerGroup][_dep_gbh.subgroup_id];

[0371] }

[0372] ​ (a) to ​ (c) Illustration of a context buffer release method according to an embodiment.

[0373] The method according to an embodiment includes a method for determining the decoder context release time.

[0374] Indicating whether a context is reused can be done by specifying whether the context used in a particular slice is stored in the context buffer. Therefore, only the context needed to encode subsequent slices can be stored, and the context buffer memory can be used efficiently. However, in such a case, the stored context remains in the context buffer until the encoding of the corresponding frame is completed, and the burden on the context buffer can increase as the number of stored contexts increases. To use the context buffer more efficiently, the stored context can be removed (i.e., released) from the context buffer when it is no longer needed.

[0375] 1) List-based context memory management

[0376] ​ (a) through 29(c) illustrate methods for managing a list of slices / subgroups of context information stored in a buffer as an efficient way to manage the context memory (i.e., the buffer). ​ (a) through 29(c) illustrate a list-based context memory management approach.

[0377] ​(a): The `context_reference_indication_flag` can indicate whether the context used to encode the current slice is stored. That is, a `context_reference_indication_flag` equal to 1 indicates that the context of the current slice / subgroup can be used in subsequent slices / subgroups. For this purpose, the context of the current slice / subgroup can be stored in a context buffer. To use the context later, the context index can be set to the same as the subgroup index. In this case, when information about slices / subgroups using the current context is provided as a list, a target list can be stored and compared with the list actually used. The list of slices / subgroups using the context can be pre-searched and sent by the encoder, or the decoder can estimate the context reference relationships based on a predetermined layer group structure to generate the list. For example, when the method of referencing the context of parent-child groups is fixed, a list of subgroups configured as children of the current subgroup can be searched. In this embodiment, this list can be referred to as list A.

[0378] ​ (b): When delivering a new slice / subgroup, the context to be used for decoding the slice / subgroup can be specified based on the context reference id. In the following example, the context corresponding to (0,0) is used, and (1,1), which is the index of the current slice / subgroup, can be added to the list of subgroups used. The list of slices / subgroups that have used a particular context during the encoding process is called list B. When context_reference_indication_flag is equal to 1, it indicates that the context of the current slice / subgroup will be used later. Therefore, the context can be stored in a context buffer, and list B(1,1) in the context buffer (1,1) can be initialized.

[0379] ​ (c): When lists A and B used for a specific context buffer are the same, memory can be managed by releasing the context buffer. When the context reference id is (1,1), encoding can be performed based on the context state (1,1) from the context buffer. At this time, the index (2,1) of the current slice / subgroup can be added to list B. Because list B and list A are the same as (2,1), it can be determined that the context state (1,1) will no longer be used in the future. In this case, the memory storing the context state (1,1) can be released. Thus, the context buffer memory can be managed effectively.

[0380] Implementations include methods for specifying context indices based on layer group indices and subgroup indices. When a unique slice index is assigned to each slice, the context buffer list can be managed based on the slice index.

[0381] ​ The illustration shows a context buffer release method according to an embodiment.

[0382] Examples include a method for managing context memory based on reference counts.

[0383] Context buffers can be used efficiently by managing context states that are no longer used in real time based on the number of times each context state is used. Implementations can manage context buffers based on a target number of context states and a counter within the context buffer.

[0384] ​ (a): The target number represents the number of times the corresponding context state is used, and the counter can be updated to show the actual number of times the context state has been used. That is, the counter value increases by 1 each time the context state is used. ​ (a) Illustration of the first slice / subgroup. When context_reference_indication_flag equals 1 or when using a layer group slice, the context state (0,0) can be stored in the context buffer. Additionally, the number of times the context state is used, N, can be stored in the target number. Since the parent-child group context states are used as an example, the target number can be specified as the same value as the number N of subgroups belonging to layer group 1. In other words, the list of sub-subgroups of FGS 0 can be derived as FGS 1 (1,0), FGS 2 (1,1), ..., FGS N (1, N-1), and the number N of elements in the list can be specified as the target number.

[0385] ​ (b): When encoding a new slice / subgroup, the context reference for the slice / subgroup can be found in the context buffer via the context reference id. The context state (0,0) can be used, and the counter can be incremented by 1. Since the context state (0,0) has already been used for encoding FGS 1 (1,0), the counter can have a value of 2. Additionally, when context_reference_indication_flag equals 1, it indicates that the context can be used as a reference for subsequent slices / subgroups, and therefore the context state for FGS 2 (1,1) can be stored in the context buffer. At this point, the number of subslices / subgroups using context state (1,1) can be stored as the target number.

[0386] ​(c): For context states where the number of targets and the counter are the same in the context buffer, memory usage of the context buffer can be managed through memory release. The context state (1,1) corresponding to the context reference ID can be used, and the counter (1,1) can be incremented by 1. In this case, since the number of targets and the counter value of the context state (1,1) become the same, it means that the context state (1,1) will no longer be used. Therefore, removing the context state (1,1) from the context buffer does not affect the encoding of subsequent slices / subgroups. By removing unused context states from the context buffer, the memory usage of the context buffer can be minimized.

[0387] The following describes an example of the decoder code implementation.

[0388] if (_dep_gbh.context_reference_indication_flag) {

[0389] _refIdxToSavedArrayIdx[curLayerGroup][_dep_gbh.subgroup_id] = _ctxtMemSaved.size();

[0390] _ctxtMemSaved.push_back(cur_ctxtMem);

[0391] _numSubsequentSubgroups.push_back(_dep_gbh.numSubsequentSubgroups);

[0392] if (_gps->geom_angular_mode_enabled_flag)

[0393] _phiBufferSaved.push_back(cur_phiBuffer);

[0394] if (_gps->geom_planar_mode_enabled_flag)

[0395] _planarSaved.push_back(cur_planar);

[0396] }

[0397] _numSubsequentSubgroups[refArrayIdx]--;

[0398] if (_numSubsequentSubgroups[refArrayIdx] == 0) {

[0399] _ctxtMemSaved[refArrayIdx].resetMap();

[0400] _ctxtMemSaved[refArrayIdx].reset();

[0401] }

[0402] The target number for each context state can be determined by the encoder by checking the number of subsequent subgroups, and then sent. If necessary, a list of slices / subgroups used for context reference can be sent separately to verify that the number of subsequent subgroups is accurate and to determine whether to delete a context state.

[0403] ​ The illustration shows a context memory management method according to an embodiment.

[0404] The embodiments include a method for managing context memory based on a data unit encoding structure.

[0405] Slices / subgroups can be sent in a specific order. Representative methods include breadth-first search and depth-first search. Breadth-first search encodes subgroups belonging to the same layer group first, and then encodes subgroups belonging to sub-layer groups. Conversely, depth-first search encodes subgroups until the subgroup corresponding to the maximum depth is reached, and then encodes subgroups belonging to the same parent first.

[0406] When each node is considered an FGS slice index, it can be assumed that slice 0 belongs to layer group 0, slices 1 and 2 belong to layer group 1, and slices 3, 4, 5, and 6 belong to layer group 2. Additionally, it can be assumed that slices connected by solid lines represent parent-child slice pairs. When encoding is performed based on breadth-first search, slices can be encoded in the order 0, 1, 2, 3, 4, 5, and 6. When encoding is performed based on depth-first search, slices can be encoded in the order 0, 1, 3, 4, 2, 5, and 6.

[0407] When managing the context buffer memory based on slice order, the following operations can be performed. In this case, it can be assumed that the parent context state is used as the reference context. When encoding is performed based on breadth-first search, the context state of the parent group is no longer used when the encoding of a subgroup belonging to each layer group is completed. Specifically, context state 0 is used when encoding slices 1 and 2, and context state 1 is no longer used when encoding slice 2. In this case, at the time of switching layer groups (i.e., at slice 2), the context states of the parent and child groups (i.e., the context state of slice 0) can be deleted from the context buffer.

[0408] When encoding is performed based on depth-first search, the context state can be deleted when encoding a subgroup is completed or when a layer group is switched (from leaf node to root). After encoding slices 3 and 4 is completed, encoding proceeds to slice 2. At this point, slices 3 and 4 belong to layer group 2, and slice 2 belongs to layer group 1. Slices 3 and 4 are encoded based on the context state of slice 1, and context state 1 can be deleted from the context buffer because it is no longer used.

[0409] In such cases, context storage can be used efficiently without additional information such as the list of subsequent subgroups or the number of subsequent subgroups.

[0410] ​ The illustration shows a bitstream containing point cloud data according to an embodiment.

[0411] The encoder according to an embodiment can encode point cloud data and generate relevant parameter information (i.e., signaling information) to generate a bitstream. The decoder according to an embodiment can receive the bitstream and decode the point cloud data based on the parameter information (i.e., signaling information).

[0412] Information about the separated slices can be defined in the parameter set and SEI message as follows. It can be defined in the Sequence Parameter Set (SPS), Geometry Parameter Set (GPS), Attribute Parameter Set (APS), Geometry Slice Header (GSH), and Attribute Slice Header (ASH). Depending on the application or system, it can be defined in corresponding or individual locations, resulting in different scopes and application methods. That is, depending on where the signaling is delivered, it can have different meanings. When defined in the SPS, it can be consistently applied to the entire sequence. When defined in the GPS, it can be used for geometry reconstruction. When defined in the APS, it can be used for attribute reconstruction. When defined in the Tile Parameter Set (TPS), the signaling can be applied only to points within a tile. When each slice is delivered, the signaling can be applied only to that slice. Depending on the application or system, it can be defined in corresponding or individual locations, resulting in different scopes and application methods. Furthermore, the syntax elements (also referred to as fields) defined below can be applied not only to the current point cloud data stream but also to multiple point cloud data streams and can be sent through higher-level parameter sets.

[0413] Each abbreviation has the following meanings and can be referred to by other terms within the same range: SPS: Sequence Parameter Set; GPS: Geometric Parameter Set; APS: Attribute Parameter Set; TPS: Tile Parameter Set; Geom: Geometric Bitstream = Geometric Slice Header + Geometric Slice Data; Attr: Attribute Bitstream = Attribute Patch Header + Attribute Patch Data.

[0414] According to embodiments, information can be generated independently of or in combination with encoding techniques. The tile parameter set can be defined to support different scalability across regions. Alternatively, the bitstream can be selected at the system level by defining Network Abstraction Layer (NAL) units and sending information related to selecting layers such as layer_id.

[0415] In the following text, parameters according to the embodiments (which may be referred to as metadata, signaling information, etc.) may be generated during the process of the transmitter according to the embodiments described below, and may be sent to the receiver according to the embodiments for use in the reconstruction process.

[0416] For example, the parameters according to the embodiments can be generated by the metadata processor (or metadata generator) of the transmitting device according to the embodiments described below, and obtained by the metadata parser of the receiving device according to the embodiments.

[0417] ​ The illustration shows an example of the syntax structure of the Sequence Parameter Set (SPS) in a bitstream according to an embodiment.

[0418] Figure 34The illustration shows an example of the syntax structure of the dependent geometry data unit header in a bitstream according to an embodiment.

[0419] Figure 35 The illustration shows an example of the syntax structure of the dependency attribute data unit header in a bitstream according to an embodiment.

[0420] Figures 36A and 36B illustrate examples of the syntax structure of the Layer Group Structure List (LGSI) according to an embodiment.

[0421] Figures 33 to 3 The definitions of the syntax elements in Figures 6A and 36B are as follows.

[0422] A layer_group_enabled_flag value of 1 indicates that the geometric and / or attribute bitstream of a frame or tile is contained in multiple slices that match the coding layer group or its subgroups. A layer_group_enabled_flag value of 0 indicates that the geometric bitstream of a frame or tile is contained in a single slice.

[0423] `layer_group_slice_order_type` indicates the sorting type of slices in a layer group. `layer_group_slice_order_type` equal to 0 indicates a width-first search order for slices. `layer_group_slice_order_type` equal to 1 indicates a depth-first search order. `layer_group_slice_order_type` equal to 2 indicates that no sorting type is specified.

[0424] A context_reference_indication_flag value of 1 indicates that the context state of the current dependency slice will be inherited by one or more of the following dependency slices. A context_reference_indication_flag value of 0 indicates that the context state of the current dependency slice will not be inherited by the following dependency slices.

[0425] The decoder can use `context_reference_indication_flag` to manage the context buffer. When `context_reference_indication_flag` equals 1, the context state of the current dependent slice is saved in the context buffer at the end of decoding. When `context_reference_indication_flag` equals 0, the context state of the current dependent slice is not saved in the context buffer.

[0426] num_subsequent_data_units indicates the number of subsequent dependent data units that use the context state of the current data unit.

[0427] A subsequent_data_unit_list_present_flag value of 1 indicates that a list of subsequent data units exists. A subsequent_data_unit_list_present_flag value of 0 indicates that a list of subsequent data units does not exist.

[0428] number_of_layer_groups indicates the number of layer groups in the list of subsequent data units.

[0429] The subsequent_layer_group_id indicates the layer group index of the subsequent data unit.

[0430] The number_of_subgroup indicates the number of subgroups in the hierarchy of the list of subsequent data units.

[0431] The subsequent_subgroup_id indicates the subgroup index in the layer group of the subsequent data unit.

[0432] Layered structure list syntax

[0433] `lgsi_seq_parameter_set_id` indicates the value of `sps_seq_parameter_set_id`. Bitstream consistency requires `lgsi_seq_parameter_set_id` to be equal to 0.

[0434] lgsi_frame_ctr_lsb_bits indicates the length in bits of the lgsi_frame_ctr_lsb syntax element (or field).

[0435] The `lgsi_frame_ctr_lsb` indicates the least significant bits (`lgsi_frame_ctr_lsb_bits`) of the valid FrameCtr in the layer group structure list. The layer group structure list remains valid until it is replaced by another layer group structure list.

[0436] lgsi_num_slice_ids_minus1 plus 1 indicates the number of slices present in the layer group structure list.

[0437] `lgsi_slice_id` indicates the slice ID of the `sid`-th slice within the layer group structure list. The requirement for bitstream consistency is that all values ​​of `lgsi_slice_id` are unique within the layer group structure list.

[0438] lgsi_num_layer_groups_minus1+1 indicates the number of layer groups.

[0439] lgsi_layer_group_id specifies the indicator for the layer group. lgsi_layer_group_id should be in the range of 0 to lgsi_num_layer_groups_minus1.

[0440] lgsi_num_layers_minus1+1 indicates the number of encoded layers in the slice of the i-th layer group within the sid-th slice. For i from 0 to n, the total number of encoded layers required to decode the n-th layer group is equal to the sum of lgsi_num_layers_minus1[sid][i]+1.

[0441] lgsi_num_subgroup_minus1+1 indicates the number of subgroups in the i-th layer group of the sid-th slice.

[0442] lgsi_subgroup_id specifies the indicator for the subgroup. lgsi_subgroup_id should be in the range of 0 to lgsi_num_subgroups_minus1.

[0443] `lgsi_parent_subgroup_id` specifies the indicator of the subgroup within the layer group indicated by `lgsi_subgroup_id`. `lgsi_parent_subgroup_id` should be in the range of 0 to `gi_num_subgroup_minus1` within the layer group indicated by `lgsi_subgroup_id`.

[0444] lgsi_subgroup_bbox_origin and lgsi_subgroup_bbox_size indicate the bounding box of the current subgroup.

[0445] lgsi_subgroup_bbox_origin indicates the origin of the subgroup bounding box of the subgroup indicated by lgsi_subgroup_id within the layer group indicated by lgsi_layer_group_id.

[0446] lgsi_subgroup_bbox_size indicates the size of the bounding box of the subgroup of the subgroup indicated by lgsi_subgroup_id within the layer group indicated by lgsi_subgroup_id.

[0447] lgsi_origin_bits_minus+1 indicates the length of the lgsi_origin_xyz syntax element in bits.

[0448] lgsi_origin_xyz indicates the origin of all partitions. The value of lgsi_origin_xyz[k] should be equal to sps_bounding_box_offset[k].

[0449] lgsi_origin_log2_scale indicates the scaling factor used to scale the components of lgsi_origin_xyz. The value of lgsi_origin_log2_scale should be equal to sps_bounding_box_offset_log2_scale.

[0450] Figure 37 The illustration shows a point cloud data transmission apparatus / method according to an embodiment.

[0451] Figure 37 Each component of the device corresponds to hardware, software, processor, and / or a combination thereof.

[0452] refer to Figure 37 The illustration shows an embodiment of a detailed functional configuration for encoding / transmitting point cloud data. When point cloud data is input, the encoder can process the geometric data (e.g., XYZ coordinates, ...) separately. The compressed data is encoded using the -θ coordinates and attribute data (e.g., color, reflectivity, intensity, grayscale, opacity, medium, material, gloss). The compressed data is divided into units for transmission. The sub-bitstream generator 40010 can divide the data into appropriate units to select the necessary information at the bitstream level for packaging based on the hierarchical structure information.

[0453] According to an embodiment, when different types of bitstreams are included in a single slice, the encoder can separate the generated bitstreams (e.g., AEC bitstream or DC bitstream) according to the purpose. Subsequently, based on layer group information, each slice or adjacent information can be included in a single slice according to the layer group information. Then, the metadata generator 40006 can deliver information such as layer group information, the layers included in the layer group, the number of nodes, the layer depth, the number of nodes included in the subgroup, the bitstream type, bitstream_offset, bitstream_length, and bitstream direction according to each slice ID.

[0454] When point cloud data is input to the transmitting device according to the embodiment, the geometric encoder 40002 encodes position information (geometric data, such as XYZ coordinates, ...). -θ coordinate), and the attribute encoder 40004 encodes attribute information (attribute data, such as color, reflectivity, intensity, grayscale, opacity, medium, material, gloss).

[0455] Compressed (encoded) data is divided into units for transmission. The sub-bitstream generator 40010 can divide the data into appropriate units to select the necessary information at the bitstream level for packaging based on the hierarchical structure information.

[0456] According to an embodiment, the octree-encoded geometric bitstream is input to the octree-encoded geometric bitstream segmentation unit 40011, and the directly encoded geometric bitstream is input to the directly encoded geometric bitstream segmentation unit 40012.

[0457] Octree-coded geometric bitstream segmentation unit 40011 divides the octree-coded geometric bitstream into one or more groups and / or subgroups based on information about the slices of segments (splitting) generated by the layer group structure generator 40014 and / or information related to direct encoding.

[0458] Furthermore, the direct-encoded geometric bitstream segmentation unit 40012 divides the directly encoded geometric bitstream into one or more groups and / or subgroups based on information about the slices of segments (splitting) generated by the layer group structure generator 40014 and / or information related to direct encoding.

[0459] The outputs of the octree-encoded geometric bitstream segmentation unit 40011 and the directly encoded geometric bitstream segmentation unit 40012 are input to the geometric bitstream binding unit 40013.

[0460] The geometric bitstream binding unit 40013 performs geometric bitstream binding based on information about the segmented (split) slices generated by the layer group structure generator 40014 and / or information related to direct encoding, and outputs sub-bitstreams to the segmented slice generator 40016 on a layer group basis. For example, the geometric bitstream binding unit 40013 concatenates the AEC bitstream and the DC bitstream within a single slice. The final slice is generated by the geometric bitstream binding unit.

[0461] The encoded attribute bitstream segmentation unit 40015 divides the encoded attribute bitstream into one or more groups and / or subgroups based on information about the slices (splitting) generated by the layer group structure generator 40014 and / or information related to direct encoding. One or more groups and / or subgroups of attribute information may be linked with one or more groups and / or subgroups used for geometric information or may be generated independently.

[0462] The segmentation generator 40016 receives input from the geometric bitstream binding unit 40013 and / or the encoding attribute bitstream segmentation unit 40015 based on information about the segments (splitting) generated by the metadata generator 40006 and / or information related to direct encoding, and segments one slice into multiple slices. Each sub-bitstream is sent through each slice fragment. At this time, the AEC bitstream and DC bitstream can be sent through a single slice or through different slices.

[0463] Multiplexer 40008 multiplexes the outputs from segmentation slice generator 40016 and metadata generator 40006 layer by layer, and outputs the multiplexed results to transmitter 40009.

[0464] When different types of bitstreams (e.g., AEC and DC bitstreams) are included in a single slice, the geometric encoder 40002 can separate the generated bitstreams (e.g., AEC and DC bitstreams) according to the purpose. Subsequently, based on information about the segmented (split) slices generated by the layer group structure generator 40014 and / or the metadata generator 40006 and / or information related to direct encoding (i.e., layer group information), each slice or adjacent information can be included in a single slice. According to an embodiment, information about the segmented (split) slices and / or information related to direct encoding (e.g., layer group information based on each slice ID, the layers included in the layer group, the number of nodes, the layer depth, the number of nodes included in the subgroup, the bitstream type, bitstream_offset, bitstream_length, and bitstream direction) can be sent through the metadata generator 40006. Information about the segmented (split) slices and / or information related to direct encoding (e.g., layer group information based on each slice ID, the layers included in the layer group, the number of nodes, the layer depth, the number of nodes included in the subgroup, the bitstream type, bitstream_offset, bitstream_length, and bitstream direction) can be sent by signal in SPS, APS, GPS, geometric data unit header, attribute data unit header, or SEI message.

[0465] Figure 38 The illustration shows a point cloud data receiving apparatus / method according to an embodiment.

[0466] Figure 38 The receiving method can follow Figure 37 The reverse process of the sending method. Figure 38 Each component of the device corresponds to hardware, software, processor, and / or a combination thereof.

[0467] Figure 38The illustration shows an embodiment of a detailed functional configuration for receiving / decoding point cloud data (i.e., PCC data). When an input bitstream is received, the receiving apparatus according to the embodiment can distinguish between a bitstream for location information and a bitstream for attribute information. A sub-bitstream classifier 41010 can forward the bitstream to the appropriate decoder based on information in the bitstream header. Alternatively, the layer required by the receiver can be selected at this stage. Geometric data and attribute data can be recovered from the classified bitstream according to data characteristics by the geometry decoder 41006 and attribute decoder 41008, respectively, and can then be converted into a format for the final output by the renderer 41009.

[0468] When including different types of geometric bitstreams, each bitstream can be decoded individually using a bitstream splitter 41014. In embodiments of this disclosure, octree-coded arithmetic entropy-encoded bitstreams and directly encoded bitstreams can be distinguished and processed by a geometric decoder 41006. Bitstreams can be split based on information such as bitstream type, bitstream_offset, bitstream_length, and bitstream direction. For split bitstreams, bitstream segments of the same type can be concatenated (connected) by a bitstream segment concatenator 41016. This operation can be included to process bitstreams separated by layer groups into continuous bitstreams, and the bitstreams can be ordered according to layer group information. When the bitstreams are parallelizable, they can be decoded by the decoder without concatenation.

[0469] Receiver 41002 can receive bit streams.

[0470] The demultiplexer 41004 can output point cloud data and metadata (signaling information) contained in the bitstream.

[0471] The sub-bitstream classifier 41010 can select slices, split bitstreams, and can concatenate octree-coded geometric bitstreams with bitstream segments of directly encoded geometric bitstreams.

[0472] Metadata parser 41005 can provide information about slices and / or layer groups.

[0473] The slice selector 41012 can select one or more slices included in the bitstream.

[0474] The bitstream splitter 41014 can split geometric bitstreams. Geometric data can be encoded based on an octree and / or can be directly encoded.

[0475] The bitstream segment concatenator 41016 can concatenate octree-coded geometric bitstreams and directly encoded geometric bitstreams according to the encoding type. For layer-based geometric bitstreams, bitstream segments including multiple groups / subgroups associated with the decoding region can be concatenated.

[0476] The geometry decoder 41006 can decode geometry bitstreams to output geometry data.

[0477] The attribute decoder 41008 can decode the attribute data contained in the selected slice.

[0478] Renderer 41009 can render point cloud data based on geometric data and / or attribute data.

[0479] Figure 39 The illustration shows a point cloud data receiving method according to an embodiment.

[0480] Figure 39 More detailed illustrations Figure 38 Operation of the sub-bit stream classifier 41010.

[0481] The receiving device receives data on a per-slice basis, and the metadata parser delivers parameter set information such as SPS, GPS, APS, or TPS (e.g., information about segmented (split) slices and / or information related to direct encoding). Based on the delivered information, it can be determined whether scalability is available. When scalability is available, the slice structure for scalable transmission is identified, such as... Figure 39 As shown in (42011). Based on information delivered in GPS such as num_scalable_layers, scalable_layer_id, tree_depth_start, tree_depth_end, node_size, num_nodes, num_slices_in_scalable_layer, and slice_id, the geometric slice structure can be identified.

[0482] When the value of aligned_slice_structure_enabled_flag is 1 (42017), the attribute slice structure can be similarly identified (e.g., when the geometry is encoded based on an octree, the attributes are encoded based on a scalable LoD or a scalable RAHT, and the geometry / attribute slices generated by the same slice partition have the same number of nodes for the same octree layer).

[0483] When the structures are the same, the range of geometry slice IDs is determined based on the target scalable layer, the range of attribute slice IDs is determined by slice_id_offset, and geometry / attribute slices are selected based on the determined ranges (42012 to 42014, 42018 and 42019).

[0484] When `aligned_slice_structure_enabled_flag` = 0, the attribute slice structure is determined based on information delivered in the APS, such as `num_scalable_layers`, `scalable_layer_id`, `tree_depth_start`, `tree_depth_end`, `node_size`, `num_nodes`, `num_slice_in_scalable_layer`, and `slice_id`. Depending on scalability requirements, the desired range of attribute slice IDs may be limited, and the required slice for each slice ID can be selected based on the range (42020-42021 and 42019) before reconstruction. The selected geometry / attribute slice is then provided as input to the receiving device.

[0485] In the above discussion, the decoding process based on the slice structure has been described based on scalable transmission or scalable selection at the receiver. However, when `scalable_transmission_enabled_flag` equals 0, the process for the ranging geometry / attribute slice ID can be skipped, and the entire slice can be selected, making it applicable to the non-scalable process as well. In such cases, information about previous slices (e.g., slices belonging to higher layers or slices specified by `ref_slice_id`) can still be used based on the slice structure information delivered in parameter sets such as SPS, GPS, APS, and TPS (e.g., information about segmented (split) slices and / or information related to direct encoding).

[0486] Therefore, when different types of geometric bitstreams exist, all slices within a range can be selected for each type of bitstream during slice selection. If different types of bitstreams are included in a single slice, each bitstream can be separated based on offset and length information, and the separated bitstreams can be reordered according to the layer group order for decoding.

[0487] Figure 40 The illustration shows a point cloud data encoding method based on layer groups according to an embodiment.

[0488] The encoder according to an embodiment includes Figure 40The flowchart is as follows. When inputting point cloud data, a layer group structure is constructed, and relevant parameters are obtained. Reference relationships between subgroups are established based on the layer group structure (or already provided by external input). Encoding can then be performed based on the layer group structure and the reference structure. Each subgroup / slice is checked to see if it is used as a reference. When it is used as a reference, `context_reference_indication_flag=1` can be set. When it is not used as a reference, `context_reference_indication_flag=0` can be set. Whether a slice / subgroup is used as a reference can be determined after encoding or directly through the reference structure. If it is used as a reference, the number of times the slice / subgroup is used as a reference can be signaled as the number of subsequent data units (`num_subsequent_data_units`). Furthermore, when delivering a specific list of slices / subgroups used as references, `subsequent_subgroup_list_present_flag` can be set to 1, and the list of subsequent data units can be delivered as layer group indices and subgroup indices. The parameters required for decoding are included in the data unit header, and the encoded compressed bitstream is included in the data unit to generate a bitstream for each slice. This process can be performed for each data unit / slice / subgroup.

[0489] Figure 41 The illustration shows a point cloud data decoding method based on layer groups according to an embodiment.

[0490] Figure 41 The decoding method can follow Figure 40 The reverse process of the encoding method.

[0491] For each slice, the decoder can analyze the data unit header to prepare for decoding. The context state used for decoding can be obtained from the context buffer based on `ref_layer_group_id` and `ref_subgroup_id`. The decoder is initialized based on the obtained context state, and then decoding is performed. Whether to store new context states generated during decoding can be determined based on the `context_reference_indication_flag` contained in the data unit header. When a context state is used for a subsequent slice / subgroup / data unit, `context_reference_indication_flag` is signaled as 1. In this case, the target number of subsequent data units can be determined as `num_subsequent_data_units` for memory management related to the context state. Furthermore, when `subsequent_subgroup_list_present_flag=1`, the layer group and subgroup indices of the subsequent data units can be stored in the list of subsequent data units.

[0492] For the context state in use, the context state counter and the list of used data units can be updated. Then, the context state can be released from the counter buffer's memory when the target number of subsequent data units of the context state matches the counter, or when the list of subsequent data units (list_given) matches the updated list of used data units (list_updated).

[0493] The context buffer management method can be applied to both the decoder and encoder in the same way. This method can also be applied when referencing the context buffer not only across slices based on layer groups but also within general frames or between slices of general frames.

[0494] Figure 42 (a) to Figure 42 (c) The illustration shows a context buffer management method according to an embodiment.

[0495] Fine-grained slicing (FGS) uses context inheritance across slices to mitigate coding loss caused by discontinuities between adjacent nodes or coding layers. However, as the number of slices or layer groups increases, the amount of context states in memory (also known as the context buffer) also increases. To help the decoder manage the context buffer, signals are used to indicate future use of the current context by subsequent slices.

[0496] refer to Figure 42 (a) to Figure 42(c) illustrates the context buffer control scheme for the current layer group slicing method. When the sliced ​​bitstream is decoded and context_reference_indication_flag is enabled, the context state of the decoder output is stored in the context buffer, such as... Figure 42 As shown in (a). When using context inheritance, the context state of a subsequent slice can be initialized by one of the stored context states of the previous subgroup, indicated by ref_layer_group_id and ref_subgroup_id (see [reference]). Figure 42 (b)). In Figure 42 In (c), the context state of the slice belonging to the last layer group is initialized by the stored context state of the upper slice. However, because context_reference_indication_flag is disabled, the output context state is not stored in the context buffer.

[0497] Using the `context_reference_indication_flag`, the overall size of the context buffer can be reduced by selecting context states known to be used in subsequent slices. However, the decoder cannot identify when to release each stored context. Therefore, all context states must be stored in the context buffer until all subgroups are decoded.

[0498] refer to Figure 42 (a) The first slice FGS 0 (0, 0) represents subgroup 0 of layer group 0 of the point cloud data. When encoding (or decoding) starts from FGS 0 (0, 0), the context state (0, 0) for FGS 0 is stored in the context buffer for subsequent FGS.

[0499] refer to Figure 42 (b) The third slice FGS 2 (1, 1) can be encoded (or decoded) sequentially. FGS 2 (1, 1) represents subgroup 1 of layer group 1 of the point cloud data. FGS 2 (1, 1) can be a subgroup belonging to FGS 0 (0, 0) (in a parent-child relationship). Therefore, FGS 2 (1, 1) can be efficiently encoded (or decoded) by referencing the context state (0, 0) for FGS 0 (0, 0) from the context buffer. The context state (1, 1) is then stored in the context buffer for subsequent FGS slices.

[0500] According to an embodiment, this method can solve this problem through a mechanism for releasing the context state as follows.

[0501] Figure 43 (a) to Figure 43(c) The illustration shows a context buffer management method according to an embodiment.

[0502] According to an embodiment, the method includes signaling the number of subgroups of the current subgroup so that the decoder can identify the timing for releasing the stored context state.

[0503] Figure 43 Figures (a) to (c) illustrate the method of releasing the context buffer using the proposed signal. Figure 42 Compared to 42(c), the context buffer has two additional columns: one for indicating the number of subsequent subgroups (i.e., following subgroups) referencing the current subgroup, and another for counting the number of subgroups whose context state has already been used in subgroup decoding. When context_reference_indication_flag is enabled, the number of subsequent subgroups (i.e., following subgroups) is signaled (num_subsequent_subgroups) and stored along with the context state, as shown below. Figure 43 As shown in (a). Figure 43 As shown in (b), three context states (i.e., (0, 0), (1, 0), and (1, 1)) that are known to be referenced N times or once are stored in the context buffer. For example, when there are N subsequent subgroups (i.e., the following subgroups) of the context state of FGS 0 (0, 0), and FGS 2 (1, 1) is encoded (or decoded) by referencing the context state of FGS 0 (0, 0), the counter value for the context state (0, 0) is 2 because the context state of FGS 0 (0, 0) is referenced (i.e., twice) by each of FGS 1 (1, 0) and FGS 2 (1, 1). Figure 43 As shown in (c), the context state of FGS 2 (1, 1) is released after it has been used in FGS N+2 (2, 1). Context states (0, 0) and (1, 0) are released beforehand because there are no subsequent subgroups referencing them. Releasing the context state means removing it from the context buffer (or memory).

[0504] Figure 44 (a) to Figure 44 (c) The illustration shows a context buffer management method according to an embodiment.

[0505] The proposed method is equally effective when different orders are used for FGS (simply referred to as slicing or fine slicing). Figure 44 (a) to Figure 44 As shown in (c), the method according to the embodiment is also applicable to depth-first sorting. In order to... Figure 43 (a) to Figure 43Compare this to the width-first sorting example in (c), where the slice names remain the same, but the delivery order changes from FGS 0, FGS 1, FGS 2 and FGS 3, ... to FGS 0, FGS 1, FGS N+1 and FGS 2. Figure 44 As shown in (b), when FGS 2 decoding is complete, the output context state (1, 1) is stored in the context buffer, and the number of subsequent subgroups (e.g., 1) is stored. Figure 44 As shown in (b), when decoding the subsequent slice FGS N+2, the context state is initialized with the context state (1, 1), and the counter in the context buffer is incremented by 1. Since the count in the counter equals the number of subsequent subgroups stored, the context state (1, 1) can be released at this point. In this example, the maximum number of context states in the context buffer is 2, which is equal to the number of layer groups minus 1.

[0506] In this respect, the size of the context buffer can be predicted by the receiver. For example, assuming there are N layers and the size of the subgroup within the nth layer is S[n], the case of referencing the parent-child group (or the upper subgroup) can be considered.

[0507] In this case, the number of context states that should be stored in memory (or context buffer) can be estimated as follows. Since the (N-1)th subgroup will not be referenced subsequently, the addition operation is performed only up to N.

[0508]

[0509] In the extreme opposite case of referencing the initial slice / subgroup, the amount of context state that should be stored in memory can be estimated as follows.

[0510] (Number of subgroups in the root layer - number of groups) = 1

[0511] For example, the number of subgroups in the root group can be 1.

[0512] Referencing a parent-child group can represent the case where the maximum number of context states is used, while referring to the first slice / subgroup (e.g., the root subgroup) can represent the case where the minimum number of context states is used. Therefore, when using layer group slices with different referencing relationships, the number of context states can fall between the maximum and minimum values.

[0513] Using the method according to the embodiments, less context state may need to be stored compared to dynamically releasing memory. For example, when generating and / or delivering FGS generated by layer group slices in width-first order and referring only to parent and child groups, the context state belonging to the parent layer group can be released at the end of the encoding of the current layer group. Therefore, the amount of context state that should be stored in memory can be estimated as follows:

[0514] When generating / delivering the FGS generated from layer group slices in depth-first order, the context memory of the parent-child group can be released (deleted) when the encoding of the sub-subgroup ends according to the method of the embodiment. In this case, only the context of the parent-child group with remaining children needs to be stored. Because leaf layer groups are excluded, the number of context states that should be stored in memory can be estimated as follows.

[0515]

[0516] When the receiver needs to estimate the memory size required to store the context buffer (or context state), relevant information (slice encoding order type, depth-first / width-first, number of layer groups, number of subgroups belonging to each layer group, context reference method, parent reference / root reference, etc.) can be delivered, and the above methods can be used to predict the number of context states.

[0517] Based on the context memory control mechanism according to the embodiment, signaling information (num_subsequent_data_units) indicating the number of subsequent data units is sent in the geometric data unit header and the dependent geometric data unit header, which is new signaling information.

[0518] The encoder, according to the embodiment, encodes the point cloud data and generates relevant signaling information. Then, it generates and transmits a bitstream containing the encoded point cloud data and parameter information. In the reverse process, the decoder, according to the embodiment, receives the bitstream, parses the parameter information contained in the bitstream, and decodes the point cloud data based on the parameter information. Figures 45 to 47 The diagram illustrates the syntax of the parameter information contained in the bitstream.

[0519] Figure 45 The illustration shows an example of the syntax structure of the geometric data unit header according to an embodiment.

[0520] Figure 46 The illustration shows an example of the syntax structure of the dependent geometry data unit header according to an embodiment.

[0521] Figure 47 The illustration shows an example of the syntax structure of the attribute data unit header according to an embodiment.

[0522] Figure 48 The illustration shows an example of the syntax structure of the dependency attribute data unit header according to an embodiment.

[0523] exist Figures 45 to 48 In this context, `num_subsequent_data_units` indicates the number of subsequent dependent data units that reference the current data unit or dependent data unit. A data unit can be a slice. According to an embodiment, a slice can be an FGS for a subgroup within a layer group.

[0524] `dgsh_geometry_parameter_set_id` is a geometry parameter set identifier. It can be information used to identify the set of parameters used for dependent geometric data cells.

[0525] dgsh_slice_id is the slice identifier. It can be a slice identifier associated with a dependent geometric data unit.

[0526] `layer_group_id` is the layer group identifier. It can be the identifier of the layer group associated with the dependent geometric data cell.

[0527] `subgroup_id` is the subgroup identifier. It can be the identifier of a subgroup associated with a dependent geometric data unit.

[0528] subgroup_bbox_origin[i] indicates the origin of the subgroup bounding box.

[0529] subgroup_bbox_size[i] indicates the size of the subgroup bounding box.

[0530] ref_layer_group_id is the reference layer group identifier. It can be the identifier of a layer group referenced by a dependent geometric data cell.

[0531] ref_subgroup_id is the reference subgroup identifier. It can be the identifier of the subgroup referenced by the layer group used for layer_group_id.

[0532] The context_reference_indication_flag is a flag that indicates whether the context is being referenced.

[0533] num_reference indicates the number of times the reference is made.

[0534] The following section describes a method for controlling the context buffer (also known as memory) by considering partial decoding.

[0535] refer to Figure 43 and Figure 44The described context memory control method can efficiently manage the context buffer when a bitstream comprising multiple slices is received and all received slices are decoded. However, in the case of partial decoding (i.e., decoding only some slices when a region of interest or resolution of interest exists), the reference count (num_subsequent_data_units) sent by the encoder may not be reached. In this disclosure, the terms slice, subgroup, and data unit are used interchangeably.

[0536] For example, in a layered structure divided into three layers, the receiver's decoder can determine that the last layer group is not used. In this case, when the number of references in the second layer group is known, it is not necessary to store the context (also known as the context state or context information) in the context buffer to decode the last layer group.

[0537] Therefore, in this disclosure, the number of times the context is used (or referenced) can be signaled within a predetermined unit (e.g., a data unit) so that the decoder can perform a context memory release after a certain number of references. Additionally, by signaling the number of times the context is used (or referenced) within a predetermined unit (e.g., a data unit) on a layer group basis, the decoder can perform a context memory release after a certain number of references. For example, when two or more layer groups reference the current data unit, the number of subsequent data units (i.e., data units referencing the current data unit) can be signaled for each layer group. In this disclosure, a data unit can be a subgroup or a slice.

[0538] In this case, the relationship between the number of times the context is used (i.e., referenced) for each data unit and the total number of times the context is used (i.e., referenced) can be expressed by the following equation.

[0539]

[0540] In the above equation, "subsequent layer group" can refer to a layer group that includes a subgroup (i.e., a data unit) that references the context of the current data unit, and num_sdu_per_layer_group can represent the number of times a subgroup belonging to each layer group references the context of the current subgroup. In other words, num_sdu_per_layer_group can represent the number of data units (also called subgroups) belonging to each layer group that reference the context of the current data unit of the current layer group.

[0541] assumed Figure 49In layer group #1, FGS1(1,0) is the current data unit, and the subsequent layer groups are layer group #2 and layer group #3. Therefore, the number of data units in layer group #2 that reference the context of FGS1(1,0) as the current data unit is 1. Additionally, the number of data units in layer group #3 that reference the context of the context of FGS1(1,0) as the current data unit is also 1.

[0542] Figure 49 This is a diagram illustrating an example of a layer group structure considering partial decoding according to an embodiment. That is, from the perspective of layer groups, structures such as... Figure 49 The context reference structure shown in the diagram is used for partial decoding. More specifically, a subgroup belonging to layer group #1 can refer to the context state of a subgroup belonging to layer group #0 (i.e., the root subgroup), a subgroup belonging to layer group #2 can refer to the context state of the parent-child group belonging to layer group #1, and a subgroup belonging to layer group #3 can refer to the context state of the grandparent-child group belonging to layer group #1.

[0543] In this case, when divided into such Figure 49 In the layer group structure of the four layers shown in the diagram, the decoder of the receiving device can perform partial decoding by skipping the last layer group (i.e., layer group #3).

[0544] According to an embodiment, when performing partial decoding by skipping a specific layer group (e.g., layer group #3), the context buffer can store num_subsequent_subgroups (also referred to as num_subsequent_data_units) for each layer group and determine whether the context reference count is satisfied for each subsequent layer group. Figure 49 In the example, layer group #3 is skipped, and therefore num_subsequent_subgroups (or num_subsequent_data_units) for layer group #3 are neither used nor stored in the context buffer counter.

[0545] Figure 50 (a) to Figure 50 (d) Illustration of a context buffer management method according to an embodiment.

[0546] exist Figure 50In steps (a) to 50(d), the context buffer is managed on a per-data-unit basis. According to an embodiment, for each data unit, the context buffer can be divided into three storage areas, for example, a context state storage area 51010, a reference count information storage area 51020, and a counter information storage area 51030. In this case, the reference count information storage area 51020 and the counter information storage area 51030 can both be further divided into sub-storage areas corresponding to the number of layers in the layer group structure.

[0547] For example, in a system that includes four layers (i.e., layer #0 to layer #3) Figure 49 In the layered structure, the reference count information storage area 51020 and the counter information storage area 51030 of the context buffer can both be divided into four sub-storage areas.

[0548] refer to Figure 50 (a) After FGS0(0, 0) is encoded or decoded, the context state of FGS0(0, 0) (i.e., the context state (0, 0)) is stored in the context state storage area 51010 of the context buffer. Additionally, in the reference count information storage area 51020, the number of data units (i.e., subgroups) referencing the context state (0, 0) in each layer group, i.e., the four layer groups (layer group #0 to layer group #3), is stored. In this case, because the context state (0, 0) is referenced only by the N data units of layer group #1 (i.e., FGS1(1, 0) to FGSN(1, N-1)) and not by other layer groups, the value {-|N|0|0} is stored in the reference count information storage area 51020. Here, "-" indicates no reference. In other words, for the context state (0, 0), this value indicates that there are no references in layer group #0, no N references in layer group #1, and no zero references in layer groups #2 and #3. Figure 50 In (a), since the context state (0, 0) has not been referenced, the value of {-|0|0|0} is stored in the counter information storage area 51030.

[0549] In other words, Figure 50 In (a), when decoding FGS0 / subgroup (0, 0) (FGS0(0, 0)), the context state storage area 51010 of the context buffer can store the context state (0, 0), and the reference count information storage area 51020 can store num_subsequent_subgroups (and) Figures 51 to 55The signaling information corresponds to num_subsequent_data_units, which is delivered for each subsequent layer group via the data unit header. In this example, the value of {-|N|0|0} can be stored.

[0550] In other words, when context_reference_indication_flag is enabled, the number of subsequent (i.e., below) subgroups (num_subsequent_subgroups or num_subsequent_data_units) is signaled. Figure 50 As illustrated in (a), the context state (0, 0) is stored in the context state storage area 51010, and the corresponding number is stored in the reference count information storage area 51020 for each layer group. At this time, the counter information storage area 51030 is set to {-|0|0|0}.

[0551] The same rules are applied to other data units.

[0552] For example, for Figure 50 In (b), after FGS1(1, 0) is encoded or decoded, its context state (i.e., context state (1, 0)) is stored in the context state storage area 51010 of the context buffer. Because the context state (1, 0) is referenced by FGS N+1(2, 0) of layer group #2 and FGS 2N+1(3, 0) of layer group #3, the value of {-|-|1|1} is stored in the reference count information storage area 51020. That is, for the context state (1, 0), this value indicates that there is no reference in layer groups #0 and #1, one reference in layer group #2, and one reference in layer group #3. Figure 50 In (b), since the context state (1, 0) has not yet been referenced, the value of {-|0|0|0} is stored in the counter information storage area 51030. Because the context state (0, 0) is referenced by FGS1 (1,0), the counter information storage area 51030 corresponding to FGS0 (0, 0) stores the value of {-|1|0|0}.

[0553] In other words, Figure 50In (b), when FGS1 / subgroup (1, 0) is decoded (FGS1(1, 0)), context_reference_indication_flag is set to 1, and accordingly, context state storage area 51010 stores context state (1, 0), and reference count information storage area 51020 stores num_subsequent_subgroups for each subsequent layer group (and... Figures 51 to 55 The value of {-|-|1|1} can be stored. (This corresponds to num_subsequent_data_units in the signaling information.)

[0554] In this disclosure, the process of loading the context state (i.e., context information) can be performed with reference to the parameters ref_layer_group_id and ref_subgroup_id. Figure 50 In (b), to process (encode or decode) FGS1(1, 0), the context information (context state (0, 0)) associated with FGS0(0, 0) can be loaded. Additionally, the context state (0, 0) is used to process FGS1(1, 0), and the corresponding counter value is incremented by 1. That is, the counter information storage area 51030 corresponding to FGS0(0, 0) stores the value of {-|1|0|0}.

[0555] When the data unit processing order corresponds to breadth-first search, FGS1(1, 0) is processed, and then the next data unit in the same layer group (i.e., layer group #1) (i.e., FGS2(1, 1)) is processed. Conversely, when the data unit processing order is depth-first search, FGS1(1, 0) is processed, and then the first data unit of another layer group (i.e., layer group #2) (FGSN+1(2, 0)) is processed. Figure 50 (a) to Figure 50 (d) An example of depth-first search is illustrated. This is merely one embodiment, and this disclosure can also be applied to breadth-first search.

[0556] Here, breadth-first search refers to the method of encoding / decoding subgroups belonging to the same layer group and then encoding / decoding subgroups belonging to the sub-layer group. Conversely, depth-first search refers to the method of reaching the subgroup corresponding to the maximum depth and then first encoding / decoding children belonging to the same parent.

[0557] For each data unit (i.e., subgroup or slice), the decoder can analyze the data unit header to prepare for decoding. The context state to be used for decoding can be loaded from the context buffer using `ref_layer_group_id` and `ref_subgroup_id`. The decoder initializes the context buffer for the current data unit based on the loaded context state and then performs decoding.

[0558] exist Figure 50 In (b), when decoding FGS N+1 / subgroup (2, 0), the encoder / decoder can initialize the context state of FGS N+1 / subgroup (2, 0) based on the context state (1, 0) corresponding to ref_layer_group_id=0 and ref_subgroup_id=0. That is, the context state can be initialized based on the context state (1, 0). In this case, because the context state (1, 0) is referenced by FGS N+1 (2, 0) of layer group #2, the value in the counter information storage area 51030 changes from {-|0|0|0} to {-|0|1|0}. In addition, considering the skipped layer group, since no reference will be made in layer group #3, the context state (1, 0) can be released (i.e., deleted) because the number of references for each layer group stored in the context buffer matches the reference count of the counter. In other words, because the value (=1) corresponding to layer group #2 stored in the reference count information storage area 51020 corresponding to FGS(1,0) is equal to the value (=1) corresponding to layer group #2 stored in the counter information storage area 51030, the context state (1,0), reference count information, and counter information for FGS(1,0) are deleted from the context buffer. Thus, the context state and related information (reference count information and counter information) of FGS 1 (1,0) are released after being used by FGS N+1 (2,0). Releasing the context state means deleting the context state from the context buffer (or memory).

[0559] Furthermore, since it is assumed that decoding of layer group #3 is skipped, FGS N+2(2, 1) belongs to the last layer group, and therefore the context state of FGS N+2(2, 1) is not stored in the context buffer. As another example, even when it is assumed that layer group #3 is not skipped, the context state of FGS N+2(2, 1) is still not stored because... Figure 49 There is no reference layer group #2 data unit in the current data unit. That is, if the current data unit belongs to the last layer group due to skipped layer groups and / or there is no reference data unit for the current data unit, the context state of the currently encoded or decoded data unit is not stored in the context buffer.

[0560] When FGS2 / subgroup (1, 1) is as follows Figure 50 When the data is decoded (FGS 2 (1, 1)) as shown in (c), the context_reference_indication_flag is set to 1, the context state (1, 1) can be stored in the context state storage area 51010 of the context buffer, and the num_subsequent_subgroups (also known as num_subsequent_data_units) delivered via the data unit header can be stored in the reference count information storage area 51020 for each subsequent subgroup. In this example, {-|-|1|1} can be stored. Regarding... Figure 50 For details of (c), please refer to [reference needed]. Figure 50 (b) Description.

[0561] When decoding is as follows Figure 50 When the FGS 2N / subgroup (2, N-1) (FGS 2N(2, N-1)) is shown in (d), the encoder / decoder can initialize the context state of the FGS 2N / subgroup (2, N-1) based on the context state (1, N-1), which corresponds to ref_layer_group_id=1 and ref_subgroup_id=N-1. In other words, the context state of the FGS 2N / subgroup (2, N-1) can be initialized based on the context state (1, N-1) identified by ref_layer_group_id and ref_subgroup_id.

[0562] At this point, because the context state (1, N-1) is referenced via FGS 2N(2, N-1) of layer group #2, the value stored in the counter information storage area 51030 changes from {-|0|0|0} to {-|0|1|0}. Furthermore, considering the skipped layer group, since no reference will be made in layer group #3, the context state (1, N-1) can be released (i.e., deleted) because the number of references for each layer group stored in the context buffer matches the reference count of the counter. In other words, because the value (=1) corresponding to layer group #2 stored in the reference count information storage area 51020 corresponding to FGS (1, N-1) is equal to the value (=1) corresponding to layer group #2 stored in the counter information storage area 51030, the context state (1, N-1), reference count information, and counter information for FGS (1, N-1) are deleted from the context buffer. Thus, the context state and related information (reference count information and counter information) of FGS N(1, N-1) are released after being used by FGS2N(2, N-1).

[0563] Furthermore, since it is assumed that decoding of layer group #3 is skipped, FGS 2N(2, N-1) belongs to the last layer group, and therefore the context state of FGS 2N(2, N-1) is not stored in the context buffer. As another example, even when it is assumed that layer group #3 is not skipped, the context state of FGS 2N(2, N-1) is still not stored because... Figure 49 There is no reference layer group #2 data unit in the current data unit. That is, if the current data unit belongs to the last layer group due to skipped layer groups and / or there is no reference data unit for the current data unit, the context state of the currently encoded or decoded data unit is not stored in the context buffer.

[0564] In this disclosure, the layer group to skip decoding can be determined by the decoder according to the application.

[0565] As mentioned above, the context memory (i.e., the context buffer) can be released early in the case of partial decoding.

[0566] The following section describes the signaling information required to release the context memory (i.e., the context buffer) in a partially decoded scenario.

[0567] According to embodiments of this disclosure, the entire context reference structure can be signaled and transmitted. In one embodiment, the encoder of the transmitting device can signal the entire context reference structure via SPS and send it to the decoder of the receiving device. In this case, information such as the layer group structure and the number of reference subgroup IDs / subgroups can be delivered, enabling the decoder to identify the actual relationships of the reference structure. Then, based on the identified entire reference structure, the decoder can release the context (or context state) early in partial decoding.

[0568] According to embodiments of this disclosure, subsequent subgroup identification information (i.e., subsequent subgroup ID) can be signaled and sent directly to the decoder of the receiving device. In one embodiment, the encoder of the transmitting device can directly deliver the subsequent subgroup identification information (i.e., subsequent subgroup ID) in the data unit header. In this case, the context reference structure is delivered on a per-data-unit basis. The decoder can then check whether a request has been received from the corresponding subgroup ID in the buffer, and can release the context early when the context is fully used by the subgroup used for partial decoding.

[0569] According to embodiments of this disclosure, the number of subsequent subgroups can be divided and delivered. In one embodiment, the encoder can divide and signal the number of subsequent subgroups via a data unit header and send it to the decoder. For example, for the number N of subsequent subgroups, N can be divided into N1 (the number of referenced subgroups in layer group 1) and N2 (the number of referenced subgroups in layer group 2) for delivery. Then, when the context is used N1 times, the decoder of the receiving device that has not decoded layer group #2 can release the context early. In full decoding, the context can be released after being used N1+N2 times. Here, the subsequent subgroup can be a subsequent data unit.

[0570] Figures 51 to 55 The diagram illustrates the syntax for parameter information contained in the bitstream. That is, Figures 51 to 55 The diagram illustrates a method for delivering the number of references and subgroup identification information (subgroup id) within each layer group for each data unit. In this disclosure, when a single parent-child group has multiple sub-subgroups, bounding box information associated with the referenced subgroup can be additionally delivered, allowing for more accurate delivery of the number of context references in the case of partial decoding based on ROI, and enabling the decoder to release the context memory (i.e., the context buffer) at precise timing.

[0571] Figure 51 The illustration shows an example of the syntax structure of the Sequence Parameter Set (SPS) according to an embodiment.

[0572] exist Figure 51In this context, `layer_group_enabled_flag` equal to 1 specifies that the geometric bitstream of a slice is included in multiple slices that match a set of coding layers or their subgroups. `layer_group_enabled_flag` equal to 0 specifies that the geometric bitstream is included in a single slice.

[0573] `num_layer_groups_minus1` incremented by 1 specifies the number of layer groups, where a layer group represents a group of consecutive tree layers that form part of the geometric coding tree structure. `num_layer_groups_minus1` ranges from 0 to the number of coding trees.

[0574] `layer_group_id` is an indicator of the layer group of the slice. `layer_group_id` is in the range of 0 to `num_layer_groups_minus1`.

[0575] The increment of 1 in num_layers_minus1 specifies the number of coding layers contained in the i-th layer group. The total number of layer groups can be derived by adding (num_layers_minus1[i]+1) for all i equal to 0 to num_layer_groups_minus1.

[0576] `subgroup_enabled_flag` equal to 1 indicates that the i-th layer group is divided into two or more subgroups, where the set of points in the subgroups of the layer group is the same as the set of points in the layer group. When `subgroup_enabled_flag` of the i-th layer group is equal to 1, `subgroup_enabled_flag` of the j-th subgroup is equal to 1, where j is greater than or equal to i. `Subgroupenabled_flag=0` indicates that the current layer group is not subdivided into multiple subgroups and is contained in a single slice.

[0577] The increment of 1 in subgroup_bbox_origin_bits_minus1 specifies the length of the syntax element subgroup_bbox_origin in bits.

[0578] The increment of 1 in subgroup_bbox_size_bits_minus1 specifies the length in bits of the syntax element subgroup_bbox_size.

[0579] The increment of 1 in num_subgroup_minus1 specifies the number of subgroups in the i-th subgroup.

[0580] num_subsequent_data_units specifies the number of subsequent dependent data units of the j-th data unit or the j-th dependent data unit in the i-th layer group.

[0581] The subsequent_data_unit_id specifies the index of the k-th subsequent data unit in the i-th layer group, or the j-th dependent data unit.

[0582] Figure 52 The illustration shows an example of the syntax structure of the geometric data unit header according to an embodiment.

[0583] Figure 53 The illustration shows an example of the syntax structure of the dependent geometry data unit header according to an embodiment.

[0584] Figure 54 The illustration shows an example of the syntax structure of the attribute data unit header according to an embodiment.

[0585] Figure 55 The illustration shows an example of the syntax structure of the dependency attribute data unit header according to an embodiment.

[0586] exist Figures 52 to 55 In this context, dgsh_geometry_parameter_set_id is an identifier used to identify a set of geometric parameters. In one embodiment, dgsh_geometry_parameter_set_id may be information used to identify a set of parameters used for dependent geometric data units.

[0587] dgsh_slice_id is a slice identifier. In one embodiment, dgsh_slice_id may be a slice identifier associated with a dependent geometric data unit.

[0588] `layer_group_id` is a layer group identifier. In one embodiment, `layer_group_id` can be an identifier of a layer group associated with a dependent geometric data unit.

[0589] `subgroup_id` is a subgroup identifier. In one embodiment, `subgroup_id` can be an identifier of a subgroup associated with a dependent geometric data unit.

[0590] subgroup_bbox_origin[i] indicates the origin of the subgroup bounding box.

[0591] subgroup_bbox_size[i] indicates the size of the subgroup bounding box.

[0592] ref_layer_group_id is a reference layer group identifier. In one embodiment, ref_layer_group_id can be an identifier of a layer group referenced by a dependent geometric data unit.

[0593] ref_subgroup_id is the reference subgroup identifier. In one embodiment, ref_subgroup_id may be the identifier of the subgroup referenced by the layer group corresponding to the layer group identifier.

[0594] `context_reference_indication_flag` is a flag indicating whether a context is referenced. In one embodiment, the value of `context_reference_indication_flag` can be used to determine whether new context state generated during the decoding process of the corresponding data unit will be stored in the context buffer.

[0595] `num_subsequent_data_units` specifies the number of subsequent dependent data units that reference the current data unit or dependent data unit. Data units can be subgroups or slices. According to an embodiment, a slice can be an FGS of a subgroup within a layer group.

[0596] `num_sdu_per_layer_group_present_flag` equal to 1 indicates that the number of subsequent data units in each layer group exists. `num_sdu_per_layer_group_present_flag` equal to 0 indicates that the number of subsequent data units in each layer group does not exist. In other words, `num_sdu_per_layer_group_present_flag` is a flag indicating whether to deliver the number of references and `data_unit_id` for each layer group. When `num_sdu_per_layer_group_present_flag` equals 0, only `num_subsequent_data_units` are delivered.

[0597] `sdu_present_flag` equal to 1 indicates that the subsequent data units of the current data unit exist in the i-th layer group. `sdu_present_flag` equal to 0 indicates that the subsequent data units of the current data unit do not exist in the i-th layer group.

[0598] num_sdu_per_layer_group specifies the number of subsequent data units in the i-th layer group referenced by the current data unit or dependent data units.

[0599] The subsequent_data_unit_id specifies the index of the subsequent data unit referenced by the current data unit or a dependent data unit.

[0600] for Figures 52 to 55 For syntax elements not described in the text, please refer to [reference]. Figures 45 to 48 The description.

[0601] refer to Figure 49 As an example, Figure 52 The geometry_data_unit_header is applied to the geometric data unit corresponding to the data unit included in layer group #0, and Figure 53 The dependent_geometry_data_unit_header is applied to the dependent data unit corresponding to the data unit included in layer group #1 to layer group #3.

[0602] Figure 56 This diagram illustrates an example of providing services by compressing the geometry and attributes of point cloud data. Specifically, in a point cloud compression (PCC) based service, the compression rate or data volume can be adjusted for transmission based on receiver performance or transmission environment. When such... Figure 56 As shown, when each slice groups point cloud data, changes in receiver performance or transmission environment may necessitate either 1) pre-transcoding and separate storage of the bitstream for each environment before transmission, allowing selection to be performed during transmission, or 2) transcoding the data before transmission. In this case, storage problems or potential latency issues may arise due to transcoding if the number of receiver environments to be supported increases or the transmission environment changes frequently.

[0603] Figure 57 This is a diagram illustrating another example of providing services by compressing the geometry and attributes of point cloud data according to an embodiment.

[0604] As proposed in this disclosure, by dividing compressed data according to the layers used for transmission, only the necessary portions of the pre-compressed data can be selectively sent at the bitstream stage via a bitstream selector, without the need for separate transcoding operations. This scheme is efficient in terms of storage space because only one storage space is required per stream. Furthermore, it is efficient in terms of bandwidth because only the necessary layers are selected by the bitstream selector before transmission.

[0605] Regarding the effects of the features of this disclosure from the receiver's perspective, when hierarchical coding (or tiered coding) is used as an application, compressed data can be partitioned and transmitted according to layers. In this case, efficiency at the receiver side can be increased. In particular, while applying scalable attribute coding results in latency and computational burden on the receiver because the entire encoded geometry must be received and decoded, the proposal of this disclosure only allows the decoding of geometry layers matching the tree level used in scalable attribute coding, thereby reducing the latency factor and increasing decoder efficiency by saving the computational power required for decoding.

[0606] Figure 58 This diagram illustrates the operations at the transmitting and receiving sides when transmitting point cloud data composed of layers. In this regard, if the information used to reconstruct the entire point cloud data is delivered regardless of the receiver's performance, the receiver, after reconstructing the point cloud data through decoding, needs to select only the point cloud data corresponding to the necessary layers (e.g., data selection or subsampling). Because decoding the delivered bitstream is performed in this case, it may result in delays in a receiver aiming for low latency, or decoding may be performed regardless of the receiver's performance.

[0607] However, as proposed in this paper, selective decoding of specific layers is allowed by the receiver when receiving compressed data for only the necessary layers. Therefore, decoder efficiency can be increased and decoders with various performance levels can be supported.

[0608] Figure 59 The illustration shows a point cloud data transmission / reception apparatus / method according to an embodiment.

[0609] When layer group slices are used to generate G-PCC bitstreams, multi-resolution ROIs can be supported.

[0610] refer to Figure 59 Multi-resolution ROIs can be supported through the scalability and spatial accessibility of hierarchical tiling. Figure 59 In the transmitting side, encoder 47001 can generate bitstream slices of spatial subgroups or octree layer groups for each layer group. Upon request, slices matching the ROI at each resolution are selected and sent to the receiving side. Compared to a tile-based method, the overall bitstream size is reduced because it does not include details beyond the requested ROI. At the receiving side, decoder 47004 can combine the slices to produce three outputs, for example: 1) a high-level view output from the layer group; 2) a mid-level view output from selected subgroups of layer groups 1 and 2; and 3) a low-level view output with high-quality details from layer groups 2 and 3, as well as selected subgroups of layer group 1. The outputs can be generated progressively, and thus the receiver can provide a viewing experience such as scaling, progressively increasing resolution from high-level to low-level views.

[0611] According to an embodiment, encoder 47001 may correspond to a geometric encoder and / or an attribute encoder as a point cloud encoder. The encoder may slice the point cloud data based on layer groups (or multiple groups). Layers may be referred to as the depth of a tree, the level of LOD, etc. As shown in section 47002, the depth of the geometric octree and / or the level of the attribute layer may be divided into layer groups (or subgroups).

[0612] The slice selector 47003, connected to the encoder 47001, can select segmented slices (or sub-slices) and selectively and partially send data, such as layer group 1 to layer group 3.

[0613] Decoder 47004 can decode point cloud data that is selectively and partially transmitted. For example, it can decode layer group 1 (which has a high depth / layer / level or an index of 0, or is close to the root) for high-level views. Furthermore, for mid-level views, decoding can be performed based on layer group 1 and layer group 2 by increasing the depth / level index above layer group 1. Additionally, for low-level views, decoding can be performed based on layer groups 1 through 3.

[0614] refer to Figure 59 According to an embodiment, encoder 47001 can receive point cloud data as input and slice it into layer groups. That is, the point cloud data can be hierarchically structured and divided into multiple layer groups. The hierarchical structure can refer to an octree structure or a level of detail (LoD). Part 47002 represents point cloud data divided into multiple layer groups. Slice selector 47003 can select a layer group (or a corresponding slice) and send the selected slice to decoder 47004 at the receiving side. Decoder 47004 can reconstruct only layer group 1, reconstruct layers 1 and 2, or reconstruct all received layer groups by combining the received slices as needed by the user. Layer groups are hierarchical and have different levels of detail. Reconstructing only layer group 1 can result in a wider reconstruction range and coarser detail. Reconstructing all layer groups 1 to 3 can result in a narrower reconstruction range and finer detail.

[0615] Figure 60 This is a flowchart illustrating a point cloud data transmission method according to an embodiment.

[0616] The point cloud data transmission method according to the embodiment may include: acquiring point cloud data (71001), encoding point cloud data (71002), and transmitting the encoded point cloud data and signaling information (71003). In this case, the bit stream containing the encoded point cloud data and signaling information can be encapsulated into a file and transmitted.

[0617] In the operation of acquiring point cloud data (71001), the following can be executed: Figure 1 Some or all of the operations of the point cloud video acquisition unit 10001, or can be executed Figure 8 Part or all of the operations of the data input unit 8000.

[0618] In the operation of encoding point cloud data (71002), the following can be performed: Figure 1 Point cloud video encoder 10002, Figure 2 The code 20001 Figure 3 Point cloud video encoder, Figure 8 Point cloud video encoder, Figure 37 Encoding device, Figure 39 Encoding methods and Figure 40 The encoding method operates on some or all of the parameters to encode geometric and attribute information.

[0619] The operation of encoding point cloud data (71002) according to the embodiment may include compressing geometric information and compressing attribute information related to the input point cloud data.

[0620] According to embodiments, the operation of compressing geometry information may include compressing the geometry information on a layer group and / or subgroup basis. In one embodiment, a context buffer for storing the context state (or context information) of the encoded data unit may be managed on each data unit. In this disclosure, a data unit may refer to a subgroup or slice. For methods of compressing geometry information and managing context buffers in the operation of compressing geometry information that are not described in detail below, refer to... Figure 49 and Figure 50 The description.

[0621] According to an embodiment, the operation of compressing attribute information includes compressing the attribute information based on the positions where geometric encoding is skipped and / or based on the reconstructed geometric information.

[0622] In this specification, signaling information may include information for partial decoding and / or context buffer management.

[0623] Information used for partial decoding and / or context buffer management may be included in at least one of the SPS, APS, geometric data unit header (or dependent geometric data unit header), and / or attribute data unit header (or dependent attribute data unit header) and sent to the receiving side. For details regarding information included in the information used for partial decoding and / or context buffer management not described below, please refer to [reference needed]. Figures 51 to 55 .

[0624] Figure 61 This is a flowchart illustrating a point cloud data receiving method according to an embodiment.

[0625] The point cloud data receiving method according to the embodiment may include: receiving encoded point cloud data and signaling information (81001), decoding point cloud data based on signaling information (81002), and rendering the decoded point cloud data (81003).

[0626] The operation of receiving point cloud data and signaling information (81001) according to the embodiment can be performed by... Figure 1 Receiver 10005 Figure 2 Transmission 20002 or decoding 20003, or Figure 9 The receiver 9000 or receiver processor 9001 executes this.

[0627] The operation of decoding point cloud data (81002) according to the embodiment can be performed. Figure 1 Point cloud video decoder 10006 Figure 2 Decoding 20003 Figure 8 Point cloud video encoder, Figure 9 Point cloud video decoder, Figure 38 Decoding device, and Figure 41 The decoding method operates in part or in whole to decode geometric and attribute information.

[0628] The operation of decoding point cloud data (81002) according to the embodiment includes decoding geometric information and decoding attribute information.

[0629] The operation of decoding geometric information may include decoding (i.e., reconstructing) geometric information on a layer group and / or subgroup basis based on information for partial decoding and context buffer management included in signaling information.

[0630] In the operation of decoding geometric information, when skipping a specific group of layers (e.g., Figure 49 In layer group #3) and when performing partial decoding, the context buffer can store num_subsequent_subgroups (or num_subsequent_data_units) for each layer group and determine whether the context reference count is satisfied for each subsequent layer group.

[0631] In the operation of decoding geometric information, a context buffer is managed for each data unit. According to an embodiment, for each data unit, the context buffer can be divided into three storage areas, for example, a context state storage area 51010, a reference count information storage area 51020, and a counter information storage area 51030. In this case, the reference count information storage area 51020 and the counter information storage area 51030 can both be further divided into sub-storage areas corresponding to the number of layers in the layer group structure. For methods of managing the context buffer in the operation of decoding geometric information not described below, refer to... Figure 49 and Figure 50 A detailed description.

[0632] The operation of decoding attribute information includes decoding (i.e., decompressing) attribute information on a layer group and / or subgroup basis, based on information included in the reconstructed geometric information and signaling information for partial decoding and / or context buffer management.

[0633] Rendering (81003) according to an embodiment may include reconstructing point cloud data based on reconstructed geometric and attribute information and performing rendering according to various rendering methods. For example, points in the point cloud content may be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at vertex locations, or circles centered at vertex locations. All or part of the rendered point cloud content is provided to a user via a display (e.g., a VR / AR display, a general-purpose display, etc.). The operation of rendering point cloud data (81003) according to an embodiment may be performed by… Figure 1 Renderer 10007 Figure 2 Rendering 20004 or Figure 9 The renderer 9011 is executed.

[0634] Each of the above components, modules, or units can be a software, processor, or hardware component that performs a series of processes stored in memory (or storage unit). Each of the steps described in the above embodiments can be executed by a processor, software component, or hardware component. Each module / block / unit described in the above embodiments can operate as a processor, software, or hardware. Furthermore, the methods presented by the embodiments can be executed as code. This code can be written to a processor-readable storage medium and thus read by a processor provided by the device.

[0635] In this specification, when a component “comprises” or “includes” an element, unless otherwise stated, it means that the component also includes or comprises another element. Furthermore, the term “…module (or unit)” disclosed in this specification refers to a unit for performing at least one function or operation, and can be implemented by hardware, software, or a combination of hardware and software.

[0636] Although embodiments have been explained with reference to each of the accompanying drawings for simplicity, new embodiments can be devised by incorporating the embodiments illustrated in the drawings. If a person skilled in the art designs a computer-readable recording medium containing a program for performing the embodiments described above, it may fall within the scope of the appended claims and their equivalents.

[0637] The apparatus and methods are not limited to the configurations and methods of the above embodiments. The above embodiments can be configured by selectively or wholly combining with each other to achieve various modifications.

[0638] Although preferred embodiments have been shown and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be made by those skilled in the art without departing from the spirit of the embodiments claimed in the claims, and these modifications should not be understood in isolation from the technical concept or views of the embodiments.

[0639] Various elements of the device according to the embodiments can be implemented by hardware, software, firmware, or a combination thereof. Various elements of the embodiments can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiments, components according to the embodiments can be implemented as separate chips. According to the embodiments, at least one or more components of the device according to the embodiments can include one or more processors capable of executing one or more programs. One or more programs can perform any one or more operations / methods according to the embodiments or include instructions for performing them. Executable instructions for performing the methods / operations of the device according to the embodiments can be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or can be stored in a transient CRM or other computer program product configured to be executed by one or more processors. Additionally, the memory according to the embodiments can be used to encompass not only volatile memory (e.g., RAM) but also non-volatile memory, flash memory, and PROM. It can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, the processor-readable recording medium can be distributed across computer systems connected via a network, allowing processor-readable code to be stored and executed in a distributed manner.

[0640] In this document, the terms “ / ” and “,” should be interpreted as indicating “and / or”. For example, the expression “A / B” can mean “A and / or B”. Furthermore, “A, B” can mean “A and / or B”. Additionally, “A / B / C” can mean “at least one of A, B, and / or C”. Also, “A / B / C” can mean “at least one of A, B, and / or C”. Furthermore, in this document, the term “or” should be interpreted as indicating “and / or”. For example, the expression “A or B” can include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term “or” in this document should be interpreted as indicating “additionally or alternatively”.

[0641] The various elements of the embodiments can be implemented by hardware, software, firmware, or a combination thereof. The various elements of the embodiments can be executed by a single chip, such as a single hardware circuit. According to the embodiments, the elements can be selectively executed by separate chips, respectively. According to the embodiments, at least one of the elements of the embodiments can be executed in one or more processors including instructions for performing the operations according to the embodiments.

[0642] Operations according to the embodiments described herein can be performed by a transmitting / receiving device including one or more memories and / or one or more processors according to the embodiments. One or more memories may store programs for processing / controlling operations according to the embodiments, and one or more processors may control the various operations described herein. One or more processors may be referred to as controllers, etc. In the embodiments, operations may be performed by firmware, software, and / or combinations thereof. Firmware, software, and / or combinations thereof may be stored in a processor or memory.

[0643] Terms such as "first" and "second" can be used to describe various elements of the embodiments. However, the various components according to the embodiments should not be limited by the terms described above. These terms are used only to distinguish one element from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should not be interpreted beyond the scope of the various embodiments. Both the first user input signal and the second user input signal are user input signals, but they do not mean the same user input signal unless the context clearly specifies otherwise. The terms used to describe the embodiments are used only for the purpose of describing particular embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and the claims, the singular form includes the plural of the referred objects unless the context clearly specifies otherwise. The expression "and / or" is used to include all possible combinations of the terms. Terms such as "comprising" or "having" are intended to indicate the presence of graphics, quantities, steps, elements, and / or components, and should be understood to not exclude the possibility of the additional presence of graphics, quantities, steps, elements, and / or components.

[0644] As used herein, conditional expressions such as “if” and “when” are not limited to optional cases and are intended to be interpreted as performing a related operation or interpreting a related definition in accordance with a specific condition when that condition is met. Embodiments may include variations / modifications within the scope of the claims and their equivalents. Those skilled in the art will understand that various modifications and variations may be made to this disclosure without departing from the spirit and scope thereof. Therefore, this disclosure is intended to cover modifications and variations thereof, provided they are within the scope of the appended claims and their equivalents.

[0645] [Mode for public use]

[0646] As described above, the relevant content has already been described in the best mode for performing the embodiments.

[0647] [Industrial Applicability]

[0648] As described above, the embodiments can be applied in whole or in part to point cloud data transmitting / receiving apparatuses and systems. It will be apparent to those skilled in the art that various changes or modifications can be made to the embodiments within their scope. Therefore, the embodiments are intended to provide modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A method for decoding point cloud data, the method comprising: Receives geometric data, attribute data, and signaling information; Decode the geometric data based on the signaling information; as well as The attribute data is decoded based on the signaling information.

2. The method according to claim 1, in, The geometric data is partitioned and included in data units of a layer group structure, and Specifically, the geometric data of the current data unit in the current layer group is decoded by referring to context information related to the data unit of the upper layer group stored in memory.

3. The method according to claim 2, in, The memory is managed for each of the data units. The memory, which is referenced to decode the geometric data of the current data unit, is divided into a context state storage area, a reference count information storage area, and a counter information storage area. in, The context state storage area stores the context information related to the data unit of the upper-level group; The reference count information storage area stores the number of data units for each data unit in the layer group that constitutes the layer group structure, referencing the data unit of the upper layer group; and The counter information storage area stores the counter information updated each time the data unit of the upper layer group is used for each of the layers.

4. The method according to claim 3, wherein, The decoding of the geometric data includes: Since there are no subsequent data units that reference the context information associated with the current data unit, the storage of the context information associated with the current data unit of the current layer group in the memory is skipped.

5. The method according to claim 3, wherein, The decoding of the geometric data includes: Since the current layer group is the last layer group in the layer group structure, the context information related to the current data unit of the current layer group being decoded is skipped from being stored in the memory.

6. The method according to claim 5, wherein, The decoding of the geometric data includes: Since the current layer group is the last layer group in the layer group structure that excludes the layer groups for which decoding is skipped, the context information related to the current data unit of the current layer group being stored in the memory is skipped.

7. The method according to claim 3, wherein, The decoding of the geometric data includes: The release of the context information associated with the data unit of the upper-layer group is determined based on the values ​​stored in the reference count information storage area and the counter information storage area of ​​the memory.

8. The method according to claim 2, wherein, The signaling information includes: Information used to identify the data unit of the upper group of geometric data referenced to decode the current data unit.

9. The method according to claim 2, wherein, The signaling information includes: Information related to the number of reference data units in each of the layer groups in the reference count information storage area for storage in the memory. The information is transmitted individually using a signal for each of the layers.

10. An apparatus for decoding point cloud data, comprising: A receiver configured to receive geometric data, attribute data, and signaling information; A geometry decoder configured to decode the geometric data based on the signaling information; as well as An attribute decoder is configured to decode the attribute data based on the signaling information.

11. The apparatus according to claim 10, in, The geometric data is partitioned and included in data units of a layer group structure, and The geometry decoder decodes the geometry data of the current data unit in the current layer group by referring to context information related to the data units of the upper layer group stored in memory.

12. The apparatus according to claim 11, in, The memory is managed for each of the data units. The memory, which is referenced to decode the geometric data of the current data unit, is divided into a context state storage area, a reference count information storage area, and a counter information storage area. in: The context state storage area stores the context information related to the data unit of the upper-level group; The reference count information storage area stores the number of data units for each data unit in the layer group that constitutes the layer group structure, referencing the data unit of the upper layer group; and The counter information storage area stores the counter information updated each time the data unit of the upper layer group is used for each of the layers.

13. The apparatus according to claim 12, wherein, The geometry decoder determines the release of the context information associated with the data unit of the upper-level group stored in the context state storage area based on the values ​​stored in the reference count information storage area and the counter information storage area of ​​the memory.

14. A method for encoding point cloud data, the method comprising: Encode the geometric data; Encode the attribute data; as well as Transmit encoded geometric data, encoded attribute data, and signaling information.

15. An apparatus for encoding point cloud data, comprising: A geometry encoder, configured to encode geometric data; An attribute encoder, configured to encode attribute data; as well as A transmitter configured to transmit encoded geometric data, encoded attribute data, and signaling information.

Images